Speculation about a multi-Doctor episode as part of the show's 50th anniversary celebrations in 2013 have been swirling around for a long while, and Smith was asked at the fan event who he would potentially like to work with if such an episode ever did get made.
Watch the first Doctor Who panel event at San Diego's Comic-Con:
"I think as an event if it did ever happen - I don't know how feasible it is - to get David [Tennant] and Chris [Eccleston] back would be amazing," he said.
"But I think it would be very difficult to make it happen."
When offered the chance to work with any former Doctor, alive or dead, he told fans: "I love Patrick Troughton. I think he was marvellous and I would like to look him in the eye as a Doctor."
Smith, Karen Gillan and Arthur Darvill answered several light-hearted questions at the event such as, 'What is the plural of TARDIS?' and a 6-year-old girl quizzing Smith over whether he was scared of the Weeping Angels.
Gillan claimed that her favourite show to do a crossover episode with would be Mad Men, so she could "meet Jon Hamm", while Darvill proposed a Sesame Street and Who special.
Smith took a question on the possibility of a Sherlock and Who crossover, suggesting that the two lead characters wouldn't get along.
"If Sherlock and Doctor ever met, they would both find each other ridiculous," he said.
The new series of Doctor Who will air in the US and UK in mid-August. The first screening of the premiere 'Asylum of the Daleks' will be at the London BFI on August 14.
Photo gallery - Comic-Con 2012 stars: