Simon Cowell does not want the 2010 winner, who is no longer on his label, to perform his material on this year's live shows, the Daily Star reports.
Cowell is also said to have ruled out Cardle appearing on spinoff show The Xtra Factor in support of his new album The Fire.
A source close to Cardle said: "It's a real slap in the face. Regardless of what's gone on, you'd think the show would celebrate the success of its winners.
"Matt would love to go back and perform but sadly won't ever get the chance. He's not sure what he's done to upset Cowell."
Cardle won Cowell's final UK series two years ago under the guidance of mentor Dannii Minogue. As the winner, he was signed under a joint deal with Syco and Columbia. He was dropped earlier this year.
His debut single 'Run For Your Life' hit number six in the charts last year and debut album Letters reached number two. It sold over 70,000 copies in its first week of release and has shifted 270,000 as of August this year.
Speaking about a possible return to the show, Cardle admitted he would love the opportunity.
"If they ask me then I would definitely do it but they won't ask me, I can promise you that," he said.
Cardle has spoken out on a number of occasions about his time with Cowell's label. He has claimed that he was not a priority compared to other acts like One Direction.
Watch Cardle's new music video for 'It's Only Love' below: