The actress, who is a judge on Simon Cowell's other show Britain's Got Talent, branded the singing competition "boring".
"I never normally watch it until right at the end, I find it boring," she told the Daily Star.
Holden added: "This year I think I know one of the contestants so I'll have to make sure I watch it now."
The X Factor has been attracting significantly less viewers this series. Last night's (September 23) show logged an audience of 8.86m - 400k down on last week.
The episode revealed the final 24 contestants and saw Cowell inform Gary Barlow, Tulisa Contostavlos, Nicole Scherzinger and Louis Walsh which categories they will be mentoring.
Photo gallery - The X Factor Final 24: