Odds on the actor leaving have been cut from 6/1 to 9/2. David Morrissey - who appeared as Jackson Lake in the 2008 Who Christmas special - is favourite to replace him at 4/1. James Corden - who made a guest appearance in the most recent series - has seen the biggest rise, with his odds changing from 100/1 to 25/1.
However, William Hill's spokesman Rupert Adams backed Smith to return.
"The viewing figures were disappointing but Doctor Who was up against the World Cup and sunny skies, so we think Matt will survive the disappointment." he said.
Odds of 66/1 were also given on former Who star Christopher Eccleston returning to the series. Other actors in the bookmaker's list include Chiwetel Ejiofor at 5/1, James Nesbitt at 12/1 and Stephen Fry at 50/1.