© Rex Features
The comic, who is famous for characters such as Kevin the teenager and Tim Nice-But-Dim, worried that Gervais's style of comedy in The Office was the death knell for his brand of sketches.
He said: "I thought we were past it, because everything was so cool. With Ricky coming up with his amazing, just so well-observed, comedy. I think what we did was well-observed too, but he got the ultimate thing to observe - an office - that I just thought it was much cleverer.
"It didn't have any jokes, it was just painfully truthful and embarrassing and really, really funny so I thought, 'That's it, I'm over.'"
According to PA, Enfield credits Matt Lucas and David Walliams's Little Britain with encouraging him to continue in comedy.
He continued: "Then Little Britain came along, and that was fantastically uncool and everyone loved it, so I thought, 'Oh good,' so we got back into it.
"I love Little Britain, I wouldn't have started doing all this again if it wasn't for Little Britain."
A new series of Harry and Paul begins tomorrow at 9.30pm on BBC Two.
