The 22-year-old was eliminated from the contest after the first deadlock of the series, having received fewer votes than his bottom two companion Amelia Lily.
Colton selected 'Will You Love Me Tomorrow' by The Shirelles as his safety song, while Lily performed Lady GaGa's 'You & I'.
Gary Barlow was first to vote after the sing-off, where he accused Lily of having "shouted [her] way through" the performance and backed his own act Colton.
"Craig, I thought your [song] was beautiful, understated and absolutely gorgeous," he added.
Kelly Rowland immediately defended Lily from Barlow's accusation, assuring her: "My darling, you did not shout your way through that song. All you have to do is sing it, which is exactly what you did tonight."
She too decided to save her own act and send Colton home, but not before telling him: "I adore you. I absolutely love your voice, I think you [have] incredible vocals."
Tulisa Contostavlos was next and after admitting that she was "torn" between the pair, opted to save Colton for having "moved" her with his performance.
Louis Walsh then chose to save Lily and send the vote to deadlock, thereby sealing Colton's fate.
"I've loved you from day one Craig, but we are looking for a ready-made popstar," he said. "And I think Amelia Lily is a worldwide ready-made popstar."
Speaking to host Dermot O'Leary after the decision, Colton commented: "I've had the time of my life these last few months and [Barlow] did it all with me... I've done myself proud."
Barlow added that Colton had been "amazing" on the show, declaring: "Everybody in here heard the might of your voice. Good luck for the future... anything you ever need from me, all you've got to do is ask."
The X Factor continues next Saturday (November 26) at 8pm on ITV1.