The singer, who was eliminated on Lady GaGa vs Queen week earlier this month, told Heat that she is no longer upset by the remark.
"I was asleep in the dressing room once and I heard one of them say, 'Kitty's got the big production number this week, but it's okay because she's a joke act'," she said.
"I won't tell you [who said it]. But I think everyone has their own insecurities, so I'm not bothered about it now."
Asked about the tension between herself and Misha B, Brucknell said: "We're both very ambitious and we're both very determined people. We had to agree to disagree and not talk.
"We just said, 'Look, we're both really determined to win, so let's be competitors, and then after the show we can see if we can be friends."
Of her alleged racist remark to Derry Mensah of The Risk, she added: "It was just an argument that got totally blown out of proportion. I didn't say what was reported and it got resolved.
"I can't even remember what we argued about, and the whole race thing was hurtful. It was totally resolved within a day."
Following her X Factor exit, Brucknell said that Lady GaGa gave her "the biggest pep talk ever".
The singer has admitted that she would have slapped herself on watching back her first televised audition for the show.
> Kitty Brucknell interview: 'I was this year's villain'
> Kitty Brucknell poses naked: 'I'm too weird for X Factor'