Kerr was voted off the show yesterday evening in a quadruple elimination, but the soldier claimed that he was angered by the fact that Brucknell remained in the competition over acts such as himself and Amelia Lily.
"To think that Amelia is no longer in the competition and that Kitty is... that's just weird," Kerr told Digital Spy.
"She should have been the first person to go. I'm not just saying that because I was the first person to go. My first role in life is a dad and that's what I care about. I got further than I could ever have expected.
"But as a viewer, I would definitely have been annoyed that Kitty's still there and you'll never see Amelia again. Amelia has so much to offer and with Kitty, it's the same dramatic crap, every week."
The Birmingham singer admitted that rumours about discontent behind the scenes between him and Brucknell were true, revealing that they had a row in a taxi over photos of the ex-Lady GaGa impersonator.
"I don't know who told the paper that... it was probably Kitty herself. It said that we were arguing in the taxi, which is 100% true," he said.
"She reacted like a child and was crying like a baby over some ridiculous pictures and I just told her to stop acting like that. Everything is for attention and dramatic with her. I just thought, 'I can't listen to this'. We're the Overs category, do you know what I mean?
"Everyone in the house is f**king sick of her, crying and moaning about everything. What can you say? That's why she can't win it. She's too much.
"I don't think she knows herself what's wrong with her. She's just a little bit strange."
He added: "It's tough to know I've been kicked out so Kitty can stay there to annoy the country. Because that's basically what they're doing.
"It sounds like I'm bad-mouthing her because I went, but I have said it before. She's been kept in as a novelty act to wind people up.
"Johnny is extravagant and will be liked by people and Sami's a serious contender, but you know, it's one of those things. Maybe I should have tried harder, but I don't think it would have made much difference on Saturday night if I'd sung like Stevie Wonder."
The X Factor continues next Saturday at 7.45pm on ITV1.
> 'X Factor' Amelia Lily will shock the world, says Kelly Rowland
> 'X Factor' Kitty Brucknell denies working as stripper