The Sex Pistols star told Sky News that the ITV1 programme represented a "mug's paradise" for upcoming stars.
"I cannot hate it - I love pop music," he said. "The more the merrier, the more variety you've got the better. But it's the endless repetitive deluge of similar-sounding things that kind of peeve me.
"It's a bit like a mug's paradise really - you've got nowhere to go after that. Once you've won, what have you won?"
Rotten's comments come ahead of tonight's final, in which Joe McElderry and Olly Murs will battle it out for this year's X Factor crown.
Stacey Solomon, who took third place after being voted off last night's programme, said that she has had "the time of [her] life" taking part in the competition.