Contostavlos and Louis Walsh both accused Bryan of being a bully on Saturday's Rock Week show and there were tabloid reports of her having backstage spats with Sami Brookes and Rhythmix.
However, according to The Mirror, the real source of tension between the N-Dubz star and Bryan was text messages sent to Contostavlos's bandmate and lover Fazer.
The rapper allegedly gave Bryan his contact details backstage because he is a fan and the 19-year-old was then accused of "bragging" about it backstage on the ITV1 show.
"Word quickly got back to Tulisa who was not amused. In fact she was livid," a source told the tabloid.
"Misha needs to pipe down or she will never get to work with Fazer. Tulisa will always be Fazer's number one priority. He is not interested in Misha romantically, and Misha shouldn't be making enemies if she's going to have longevity in the music industry."
Louis Walsh apologised on Sunday's X Factor results show for labelling Bryan a bully, but the relationship between Contostavlos, Bryan and her mentor Kelly Rowland remains frosty.
> X Factor Sami Brookes: 'Misha B rows due to time of the month'
> X Factor Misha B accused of bullying by Louis Walsh, Tulisa