The This Morning presenter, who was replaced by Christine Bleakley in November, revealed that she watched the first episode of the new-look ITV skating contest with Ant & Dec in January.
"When the music started I thought, 'What am I doing? I should be there'," she confessed, according to The People.
"It is hard to watch when you're not on it anymore but I do still switch on. I loved it. But it was time for something different."
The 31-year-old was speaking while promoting The Voice UK, which she is hosting alongside Reggie Yates tomorrow (Saturday) from 7pm on BBC One.
Willoughby recently told Digital Spy that coaches Sir Tom Jones, Jessie J, will.i.am and Danny O'Donoghue are "badass"-enough to make The Voice a success.
> Holly Willoughby to front revamped Surprise, Surprise
> The Voice UK: How does it work?
Watch Willoughby and co-host Yates talking to Digital Spy below: