The Syco boss said that he thought the Echoes singer was "obnoxious" when he auditioned for Pop Idol in 2001.
Recalling the first time he met Young, he told ITV1's Talent Show Story: "I thought he was obnoxious, full of himself.
"I thought this will be interesting, you don't like me, I don't like you... and nothing's changed particularly."
Cowell further confessed that he felt "really, really disappointed" that Young's Pop Idol rival Gareth Gates did not beat him in the reality show final in 2002.
The X Factor judge added: "I still to this day can't watch the final back because I was really, really disappointed. I really liked Gareth and wanted him to win.
"I remember feeling it in my stomach, I felt sick, that's how much it affected me; it shouldn't but it did."
The Talent Show Story starts on Saturday at 8.30pm on ITV1.
> Young reveals respect for Cowell, X Factor
> Will Young: 'Cowell fight helped career'
Watch Will Young being crowned Pop Idol winner below: