The singer, who was kicked off the contest in 2011 after breaking one of its rules, told The Sun that he was a "novelty" contestant on last year's show.
> 'The X Factor': Are the wrong acts going through? - poll
"It's all about the contestants," Cocozza said. "The first live show we had, which was the first time Gary [Barlow] and Tulisa [Contostavlos] were on it, we had 14 million viewers.
"What is it now, like 7 million? The show's f**ked. It's made a s**tload of money but they've f**ked it."
He added: "People are fed up with it. Last year, if I was watching myself get through every week, I'd be like, 'How s**t is this?' I knew that when I was on there.
"Other contestants were putting on class performances. I'm pony and I'm the one going through."
> 'X Factor': Should Simon Cowell come back to the UK?
Cocozza, who last month called the show a "set-up", continued: "Look at this year, with Kye Sones. I'm sure he's got thousands of fans and they're not going to watch if they see Rylan [Clark], who's pony - funny to watch but s**t - and poor Kye has got to go home.
"How old is Kye? 30? He might not get another chance. The novelty thing is bollocks. It's about time they stopped it."
Asked if he was a novelty act last year, he replied: "Yes. My performances were s**t. I knew they were s**t, I don't blame people at all for comparing me to Rylan.
"I just wish I was allowed to sing what I wanted to sing. I didn't want to sing Black Eyed Peas. Gary knew that. Producers made me sing those songs in order to get a negative reaction from the judges."
> 'Strictly Come Dancing' Dani Harmer: 'The X Factor is staged'
Cocozza said that Barlow tried to help him after his X Factor exit and that Contostavlos was a "sweet... normal girl" who he liked, but he was not a fan of Louis Walsh.
"Louis was just brutal," he claimed.
"He caned me on stage and backstage he was a w**ker too. I'd say, 'Alright, Lou', and he'd completely blank me. I know a lot of people hated him when they were on the show."
Of whether he would return to the show as a guest performer, Cocozza said: "I'm not sure if they'd ask me now. But if they did, I'd have to make sure I did something that would piss off Louis Walsh."
Shock X Factor exits - photo gallery: