Segel has confirmed that he will not be writing the next film, and added that he may not appear in it at all.
The next Muppets film was announced last week, at which time it was stated that the actor's involvement was uncertain.
"My goal was to bring The Muppets back and I did that, leaving them in very good hands - my writing partner [Nicholas Stoller] and James Bobin, the director," Segel told Collider.
"All I wanted to do was to set the stage for them to do whatever they wanted. I did what I set out to do, and now I wanna pursue more human-related projects. I'm sure I'll return in some capacity here and there, but that was half a decade of my life. Five years of hard work. I'm ready for a little puppet break."
Segel is a great fan of the franchise and was heavily involved in securing its return to the big screen.
He previously admitted that his Muppets collection was the reason he is single.
Segel's departure from the franchise is said to be "totally amicable".
> 'Muppets' Jason Segel on-set interview: 'These puppets were my idols'
Watch the Muppets trailer below: