The Logie Award-winning TV presenter quit the Channel Ten chatshow last year in order to concentrate on her children and launch her radio career. Her departure is said to have "damaged" the reputation of the series, according to industry insiders.
The Circle's ratings have plummeted in 2012, averaging only 56,000 viewers compared to 187,000 on the Seven Network's Morning Show and 104,000 for the Nine Network's new Mornings series.
Media analyst Steve Allen told the Herald Sun that The Circle is suffering because Channel Ten does not yet have a breakfast lead-in show like Seven's Sunrise and Nine's Today.
"[There's] major damage, really major damage, at this point to The Circle," Allen said. "We can now see it desperately needs the Breakfast lead-in to make it less vulnerable."
Ten's new programme Breakfast, hosted by Andrew Rochford, Paul Henry and Kathryn Robinson, is scheduled to start later this month.
Ten bosses recently revealed that they are in "no rush" to replace Swan permanently.
> 'The Project' to air at 6pm in order to save Ten's viewer ratings
> Channel Ten boss remains positive about 'The Project' ratings defeat