The ITV4 show followed the former England cricketer as he took on a variety of extreme challenges across the globe.
Although Flintoff himself found the adventures "really good fun", he confessed that Versus The World would not return due to his wife Rachael's fears for his safety.
"That show was really good fun, although I did manage to break two ribs," he told The Sun. "ITV want me to do a second series, but the missus is not that happy about it. She only knew what I'd been doing when she watched it on TV, and just sort of said, 'Why?'"
A family friend added: "Rachael lets Freddie get on with his own stuff, and because he does lots of media work she didn't really know what the last series was about.
"When she saw Freddie Versus The World for the first time, she was really concerned that he shouldn't be doing something that dangerous.
"When the option came up for the second series, Rachael made it perfectly clear that neither she nor the children want to see him putting himself at risk again."
Flintoff last month signed up for a BBC documentary that will explore mental illness within sport.
He also fronts Freddie Flintoff's World Of Sport, which launched on BBC Radio 5 live back in May.