The political journalist, 64, has never been voted into the bottom two on the dance contest, despite repeatedly receiving lowest marks from the judging panel.
In a statement, Sergeant said: "I am sorry to say I have decided to leave Strictly Come Dancing. It was always my intention to have fun on the show and I was hoping to stay in as long as possible.
"The trouble is that there is now a real danger that I might win the competition. Even for me that would be a joke too far.
"I would like to thank Kristina and all those viewers who have been rooting for me through the series."
Arlene Phillips complained earlier this week that Sergeant needed to work harder on his training because it was unfair on his competitors if he continued to knock out superior dancers.
Bruno Tonioli said he was "appalled" at the public's decision to save Sergeant over Heather Small and Rachel Stevens two weeks ago.
"We are very sad to see him go," said BBC One controller Jay Hunt. "We would have liked him to stay but we respect his decision to leave. John and Kristina will be performing a farewell dance this Saturday on the show."