The Radio 1 DJ and T4 presenter was apparently confirmed for the role last week, after he impressed bosses with his hugely popular online spinoff The F Factor.
However, it was last night announced that Murs and Caroline Flack will host the new series of the ITV2 show.
"In typical Simon Cowell fashion it was all very last minute," a source said. "He left it so late that on the call sheet the presenters were named as Jane and John Doe.
"Simon was keeping everyone in the dark, saying it was going to be a surprise. Well, he's not kidding. It's been the craziest week."
A show insider added: "Matt has been knocked for six. He thought the gig was his."
Speaking about his new job, Murs said: "The X Factor is the show that put me where I am so I'm chuffed to be involved again, this time to work on it from the other side and see some new stars coming through."
Flack, who has confirmed that she has quit I'm A Celebrity... Get Me Out Of Here!'s ITV2 companion show, added: "I'm a massive fan of the X Factor and can't wait to get started."
The new X Factor judges and Xtra Factor presenting team will begin filming auditions in Birmingham today.
> More X Factor UK news