With rumours swirling about a possible return for ex-judges Simon Cowell, Sharon Osbourne and Cheryl Cole, none of this year's panel can claim that their seat is secure. Even Louis Walsh.
Plus, with Gary Barlow heading back into the studio with Take That and Tulisa suddenly out of favour in the charts, maybe this year's mentors won't want to come back for another spin on the X Factor roller coaster.
Digital Spy wants to know who you guys want to see return and who you'd happily see ditched.
Is Gary Barlow still the No.1 Head Judge for you? While others were losing their heads over Rylan Clark's sub-TOWIE nonsense, Mr Barlow remained firm and didn't budge. And with Christopher Maloney, he has proved that he can take one of the most unlikely acts all the way to the final.
Do you really want Tulisa back in 2013? She may not have had so much success with her acts this year, but she's stood for no nonsense from the other judges and grown in confidence on the live shows. In many ways she's become the natural successor to Cowell as she has a bigger mean streak than her rivals.
Would it not be the same without Louis Walsh? The cheeky Irish mogul may have gone yet another year without a winner and he may have been slightly less barmy (for his standards) this year, but surely it wouldn't be X Factor without him screaming "WESTLIFE!", "DERMOT!" and "NEXT... BIG... BOYBAND!" like a drunk uncle at a wedding.
Or maybe Nicole Scherzinger is your new queen of X Factor? Whether she's dressing up in an ASDA outfit, thrusting in a catsuit, waffling on about cheeseburgers and Coco Pops or singing 'Horny' in Gary Barlow's face, the other judges have rarely had the limelight since Nicole came to town.
Who is the best X Factor judge? Who do you most want back in 2013? Vote in our poll and let us know below!