Danny Egan was asked to leave auditions after he was filmed in a dressing room by hidden cameras. However, the singer told The Sun that he had not used cocaine as previously suggested.
Producers told Danny and his wife Keeley to leave the Manchester set before they had the chance to perform for the judging panel, which includes Gary Barlow and Tulisa Contostavlos. The 28-year-old insisted that Keeley had not used the drug, which he identified as mephedrone.
"I set up lines in front of the mirror," he told the paper. "I'd no idea my face was six inches from a camera.
"I'm sorry and feel really bad for my wife. She didn't do anything wrong."
X Factor staff did not call the police, while a spokesperson said that "appropriate action" had been taken.
Auditionees are being recorded by secret cameras after bosses decided to feature more behind-the-scenes footage in the ITV1 show.
> 'X Factor' live shows 'will be longer than ever'
> Tulisa Contostavlos: 'I smoked weed, had sex at 14'