Widdecombe has regularly finished at the bottom of the judges' leaderboard during her time in the competition, but is yet to appear in the bottom two after the public vote.
Wogan urged the former Conservative MP to follow the example of John Sergeant, who stepped down from the show in 2008 after the public consistently saved him at the expense of better dancers.
"Ann Widdecombe would be sensible to withdraw from Strictly Come Dancing," Wogan told the Daily Mail. "She should do as John Sergeant did. He knew he couldn't dance for toffee and he made way for people who could. Ann's even worse than him. She can't dance at all. She should fall on her sword."
Wogan said of the public: "They're not basing their votes on whether she can shimmy across the floor. They're basing their opinions on whether they like the look of the contestants or on what they’ve read in the tabloids. And, on occasion, they sometimes want to contradict the judges.
"Having said that, when it gets down to, say, four people - three of whom are half beating themselves to death trying to dance to the best of their ability and one of whom is Ann Widdecombe - I like to think the public will display a little common sense."
Judge Len Goodman recently revealed that he does not expect Widdecombe to pull out of the competition any time soon.