The singer, who took part in the ITV1 show last year, described Cowell as a "superstar" and a "good guy".
"When he walks in a room he has an aura about him. You know he's there before you have even seen him because people are whispering about it," Drage told DS.
"I think it must be stressful for him having so many people clawing at him everyday. I just wanted to work with him and progress musically, I didn't want to be his best friend and get something out of him, like so many people do."
He added: "Simon's a good guy. He's rich beyond most people's wildest dreams, he's a superstar around the world, he's big in America and Europe and he has a massive following. So you always feel slightly taken aback when you meet him."
Speaking about this year's X Factor contestants, Drage said that he rated Over-25s Olly Murs, Jamie Archer and Danyl Johnson. He also tipped Louis Walsh's girl group Miss Frank to do well in the competition.
"Miss Frank will do really well if they can gel as a group," he said. "The moment they forget they are three solo singers, they could be wicked. I actually think that Graziella should have got in just on her own, but maybe it's something to do with her look. The girls do have a Sugababes look."
