The X-Files star was offered the role of Lady Cora Crawley - eventually played by Elizabeth McGovern - in the ITV period drama, according to TV Guide.
Anderson has appeared in a number of other period dramas, including the BBC's Bleak House and last year's adaptation of Great Expectations, which will air in the US on PBS on April 1.
A third series of Downton Abbey is currently filming and is expected to air in late 2012 on ITV and in January 2013 on PBS.
It was recently reported that stars Maggie Smith (the Dowager Countess), Dan Stevens (Matthew Crawley), Siobhan Finneran (O'Brien) and Jessica Brown Findlay (Lady Sybil) may not appear beyond the third run.
Hugh Bonneville (Earl Robert Crawley) has also denied rumours that a film version of Downton is in the works.
The show's creator Julian Fellowes has since written a four-part mini-series charting the voyage of the Titanic and will next adapt the 1957 memoirs of burlesque dancer Gypsy Rose Lee into a film.
> Downton Abbey casts Cara Theobald, Lucille Sharp as maids
Watch the stars of Downton Abbey discuss series three with Digital Spy below: