According to The Sun, the 21-year-old told producers that he would not perform 'Last Friday Night (T.G.I.F.)' because he believes the track's lyrics glorify drinking alcohol.
"Bosses are desperately trying to find a new song for this weekend's group ensemble after Jahmene's refusal," an insider said.
"The six remaining acts were supposed to perform 'Last Friday Night' but Jahmene has said no. The song is about partying and drinking and Jahmene is teetotal.
"He has made it clear from the start that he won't sing songs about drinking and when he heard this track he was adamant he wouldn't perform it.
"Bosses have respected his wishes and it's all hands on deck to get a new song ready in time for Sunday."
Douglas, who has previously spoken of his dislike for clubbing, was said to be noticeably uncomfortable with the alcohol at previous X Factor events.
"When all the contestants went out for Rylan Clark's birthday party he really didn't like it," the source added.
"Even at Judges' Houses, when he was given a glass of champagne to hold, he refused to even have it in his hand."
Photo gallery - Rylan Clark's birthday party:
Copyright: Rex Features Copetti/Photofab/Rex FeaturesDouglas, who was recently named the new bookies' favourite to win the series, last month discussed what types of music he would refuse to sing on the show.
"Anything degrading to women or [with] swearing," he revealed. "I can't see what's around in the charts for children... I think a lot of people are yearning for that soul back."
The X Factor continues at 8pm tomorrow (Saturday, November 17) on ITV1.
Watch Jahmene Douglas perform Robbie Williams's 'Angels' on The X Factor below: