Smith explained that The Doctor doesn't give enough thought to how his companions' lives are affected by travelling with him through space and time.
"He's such a nomad, and he does his thing.... He doesn't quite engage in the way that other people do. He's his own man," the actor told SFX.
Smith added: "When you think about it, he's actually quite selfish. Because he picks these people up - often hot women - and then cruises around, drops them off, and goes, 'See you later. Someone else's turn!'"
The star also revealed that he has several favourite Doctor Who episodes from his tenure so far, saying: "I love the [Weeping] Angels episodes, 'The Time Of Angels'. I love 'The Eleventh Hour', I love 'Day of the Moon', I love 'The Big Bang', I love 'Vincent And The Doctor'… It's so tricky. I love them for so many different reasons because I have so many memories associated with them."
Smith recently insisted that Doctor Who's 50th anniversary in 2013 must be "an event".
Doctor Who returns to BBC America in the US on August 27 and will air on the same day on BBC One in the UK.
> Doctor Who Matt Smith: 'Let's Kill Hitler is best episode yet'
> Doctor Who Steven Moffat: 'The Doctor will never star in Torchwood'
Watch a trailer for Doctor Who below: