The ex-Signed by Katie Price contestant said that he was "enraged" by how personal Barlow got during the controversial elimination of Carolynne Poole, arguing that the judge has failed to recognise the entertainment value of his fun-filled numbers.
"To be perfectly honest, Gary ain't really of concern to me in the slightest," he told Heat. "Some things I do think he goes a bit far with.
"There's a difference between being critical and having a dig. Why would you kick me in the mouth as I'm about to leave?"
He continued: "What Gary doesn't remember is when he first started out he was the one doing photo shoots with his arse out and stuff. Everyone's got to start somewhere.
"Gary doesn't dance around on stage. Anyone who can sing and dance at the same time... in my eyes, that's more of a talent than just standing there."
Clark also admitted that he didn't expect there to be such a strong backlash against his success on The X Factor.
"I'm not a murderer, I've done nothing wrong," he declared. "There are some times when you're in a bit of a state and you feel upset.
"If other people got a millionth of the messages I've been getting, it would probably kill them... to the point where they wouldn't leave the house."
Speaking to Digital Spy yesterday (October 15), eliminated X Factor contestant Melanie Masson admitted that she feared Barlow would quit the show over Clark.
Watch Melanie Masson talk to Digital Spy below: