The Destiny's Child singer is said to have "flipped" after Bryan, who performed Adele's 'Rolling In The Deep' on last week's show, was rude to vocal coach Anne-Marie Speed and other production staff.
Rowland also apparently reprimanded the X Factor hopeful after she was heard saying: "I've already won."
A source told The Sun: "Misha is out of control and Kelly just snapped. She told her that unless she packs it in, she'll be sent packing herself.
"It's one thing to give it the big 'I am' with the other wannabes, but it's totally out of order to be rude to people who work on the show. Kelly is a lovely woman but when the s**t hits the fan she gets tough - and she got very tough with Misha."
X Factor staff have allegedly branded Bryan "the diva from hell". The source also claimed that she was "shell-shocked" when her mentor Rowland shouted at her and forced her to apologise to everyone.
Meanwhile, neighbours of the X Factor finalists' £6 million Hertfordshire mansion have reportedly complained to show bosses about the noisy parties that occur at the property.
> X Factor hopeful Kitty Brucknell to quit over bullying?
> X Factor's Amelia Lily invited back as Kelly Rowland's "guest"
Watch Misha Bryan performing 'Rolling In The Deep' on The X Factor below: