Cowell joked about Brock's size after his performance of 'I Believe I Can Fly' in last night's movie-themed episode of the show.
"Yeah, here's the problem, Jason," he said. "I like you, but I don't believe you could fly. Maybe jump a couple of inches off the ground, but definitely not fly."
Cowell also criticised Brock's outfit, saying: "The problem is you do have a good voice and you're very likeable, but the way that you're dressed, you look like a singer in an Italian restaurant, seriously, who's working there who's just decided to sing 'I Believe I Can Fly'. So I think there's a lot of work that needs to be done with you."
suggesting that it was "bullying" and another describing it as "mean as hell" - Brock's mentor LA Reid defended his act.
"It's not up to them, it's up to America," he said. "And I think you did an excellent job. All you have to do is spread your wings because you know what, Jason? You can fly, my friend."
The X Factor USA continues tonight (November 8) on Fox.
Watch Jason Brock's performance on The X Factor USA below: