Fox's alleged comments came after Tina Fey jokingly warned Swift at the Golden Globes to "stay away from Michael J. Fox's son".
[Michael J Fox with wife Tracy Pollan and their son Sam Fox]
Since the January 13 awards event, where Fox's son was seen assisting with awards presentations in his role as 'Mr Golden Globe', Vulture caught up with the actor at a book party to ask if he would approve if his son dated Swift.
Fox is quoted as saying: "I don't keep up with it all. But Taylor Swift writes songs about everybody she goes out with, right? What a way to build a career."
[Taylor Swift with ex-boyfriend Harry Styles]
Fox also explained that if Swift were to come to a family dinner, "I wouldn't even know who she was".
When asked about a hypothetical Sam and Taylor break-up song hitting the airwaves, the Family Ties actor quipped that he would likely remember Swift then, saying: "'Sam, You Piece of S**t.' Oh … that was the girl you brought home!"
> Taylor Swift 'writes five songs about Harry Styles relationship'
Watch a clip of Tina Fey warning Taylor Swift off Sam Fox at the Golden Globes 2013 below: