The Scottish singer was a surprise victor over Ronan Parke in the final of 2011's show. His debut album Believe was released last month, but only reached a peak of 54 on the chart.
Speaking following the launch of Britain's Got Talent series six on Friday (January 20), Cowell confessed that he did not see the former support worker as a "genuine star".
"I was disappointed with Jai. He wasn't what I was looking for," The Sun quotes him as saying.
"You want a Diversity or Susan Boyle, someone who is a genuine star. He is talented but the show can do better. None of the finalists were up to scratch."
Watch Digital Spy at the Britain's Got Talent launch below:
Cowell's admission has sparked suggestions that McDowall could be axed from his record label Syco after just one album, following in the footsteps of X Factor winners Steve Brookstein, Leon Jackson and Joe McElderry.
The 25-year-old was not offered a record deal until August, two months after his BGT victory. Parke, meanwhile, was signed up immediately after the final.
> Jai McDowall interview: 'I wanted to record heavy metal for new LP'
Watch Jai McDowall sing 'With Or Without You' on the 2011 Royal Variety Performance below: