The 21-year-old hopeful was initially put through to the final 24 after making it through the Boot Camp stage of the competition. However, he was then told that he was going home after Cowell decided to offer a lifeline to Liam Payne, 15, instead.
Viewers saw Payne being given a second chance in Sunday night's episode of the ITV1 talent series, but the implications for Rikki were not made clear.
"I am gutted. Simon decided to dump me in favour of someone else at the last minute," Rikki told the Daily Star. "He couldn't even look me in the eye when he told me it was over. Cheryl [Cole] and Dannii [Minogue] weren't happy with this and that's why they were arguing."
He continued: "What annoys me is that none of this got shown. They really bigged me up on Saturday's Boot Camp show, giving viewers the impression I'd do really well. But on Sunday I barely featured. People have been saying: 'If Liam was brought back, then who got axed?' Now everyone will know it was me."
An X Factor spokeswoman said: "The judges' decisions are final. Not everything that happened at Boot Camp could be shown within the time constraints we had for the programme."