On Saturday night, the N-Dubz star commented on how the singer was "very competitive", adding that she had seen a "different side" to her backstage.
Contostavlos also revealed that some of the other acts had said that Misha B had been "quite mean" to them. However, Destiny's Child star Rowland was unhappy with the comments, arguing that the backstage antics should not be discussed.
With the two female judges not on speaking terms, an insider told Digital Spy that Contostavlos is "very bewildered that Kelly is taking it all so personally".
"At the end of the day, Tulisa was just being honest and saying something because she felt Misha had been mean to her acts, it was nothing against Kelly," they said. "Judges comment on the backstage actions of acts all the time - it has happened so many times in the show's history - Cheryl, Simon and Dannii used to do it loads.
"In fact, the panel - including Kelly - have spent this year commenting how lovely Marcus and Sophie are backstage."
The pair remained divided when discussing the incident on The Xtra Factor, while Louis Walsh also apologised for branding Misha B a "bully".
"It seems a bit hypocritical to get upset and say 'what happens backstage stays backstage' - when everyone has been bringing up backstage already," the insider added. "Perhaps it's only fine when it's positive it seems!"
Misha B, who won praise for her performance of Prince's 'Purple Rain', said that she had not spoken to Contostavlos, but would thank her for the "constructive criticism" if she did.
The X Factor continues on Saturday at 8pm on ITV1.
> 'X Factor' Tulisa Contostavlos: 'I'm not playing games'
> 'X Factor' Rock Week results show - In Pictures