The former Beatle appeared on Friday Night with Jonathan Ross in 2008 and was asked if he missed anything about Merseyside, to which he replied: "Er, no."
Speaking to the BBC, Starr, who is preparing for a gig at the Liverpool Empire as part of his UK tour, said that he is sorry his response caused outcry in the city.
A topiary image of the drummer was beheaded by vandals in the wake of the incident.
"I apologise to those people [who were offended], as long as they live in Liverpool, not outside," he commented.
However, he added: "No real Scouser took offence - only, I believe, people from the outside."
Reflecting on his feelings for Liverpool, Starr continued: "I was brought up there, I went to school there, all my childhood, my youth was there.
"It's silly that whoever took offence took offence."
Starr recently said that The Beatles were "lucky" to have him in the band, as he had a higher public profile when they recruited him.