Filmed auditions start in Liverpool on Wednesday but bosses are still searching for a permanent replacement for Kelly Rowland after Dannii Minogue rejected a £1m offer to return.
> Katy Perry, Lily Allen, Noel Gallagher: X Factor replacements in pics
Alicia Keys will join Williams, Stefani and Rod Stewart as a guest judge, The Sun reports.
Williams often performs with the X Factor contestants and last year helped Take That bandmate Gary Barlow at Judges' Houses, emerging in a Sinitta-style leaf outfit.
Meanwhile, Katy Perry, Rihanna and Nicole Scherzinger have all turned down the chance to be guest judges, according to the Daily Star Sunday.
"At this rate the fourth judge may not be signed until much later in the series," a source said. "Dannii turning them down was a massive blow because as far as ITV1 and The X Factor were concerned it was a done deal.
"The back-up plan is to have a host of guest performers take part in the regional auditions and possible Boot Camp before revealing a permanent name at the Judges' Houses in August. Everyone involved knows how important the judges are to the success of the show.
"Numerous high-profile singers have come forward wanting the job. But bosses fear they aren't big enough names to wow viewers. So it's now panic stations to make sure they can sign up a big enough name in time."
While a permanent judge is unlikely to be found before Boot Camp, former Spice Girls Geri Halliwell and Mel B, The Saturdays singers Frankie Sandford and Rochelle Wiseman and Britain's Got Talent judge Alesha Dixon are all said to be in the frame.
Barlow, Louis Walsh and Tulisa Contostavlos will return to The X Factor, as well as host Dermot O'Leary.
Watch Dermot O'Leary discuss the new series of The X Factor below: