The Xtra Factor host, who was a contestant on the show back in 2009, said that the veteran judge would be a hard act to follow.
"Who's going to replace Louis? That's a big question! After he has been there for such a long time and done such amazing work," Murs told Heart West Midlands.
"I mean, it's a job a lot of people will want to take. Maybe they will take someone who's been on The X Factor, done really well in their career successfully, and take someone like that to do it."
He joked: "I don't know who that could be, it could be anyone."
Murs recently admitted that he fancies new X Factor judge Nicole Scherzinger.
Watch Olly Murs's video for 'Oh My Goodness' below: