The 50-year-old impressionist and his partner Erin Boag finished bottom of the leaderboard after judges' scores and viewer votes for their cha cha cha were combined.
Nancy Dell'Olio and Anton du Beke also finished in the bottom two for a second consecutive week.
When asked by hosts Tess Daly and Claudia Winkleman to pick his favourite Strictly moment, Bremner joked: "Well, a close second was shimmying next to [audience member] John Prescott in the quickstep, I loved that - it was just the best dance of my life."
Turning to Boag, he added: "Thank you for choreographing it, and thank you for dancing with me."
"You've been completely and utterly wonderful," she responded. "You've been so wonderful, and I want you to hold your head high, because you've done great."
Edwina Currie became the first person to exit this year's Strictly, while Daybreak presenter Dan Lobb was voted out last week.
Strictly Come Dancing continues next Saturday (October 29) at 6.30pm on BBC One.
Watch Rory Bremner's final Strictly Come Dancing performance below: