The character, who first appeared on BBC Two in 1995, would gatecrash functions and red carpet events to ask the rich and famous controversial questions.
Speaking at the premiere of his new film Cass, Kaye told the Press Association: "No, I was a different guy really - I couldn't do it again.
"I used to hang around here for about two years solid. It was fascinating being on that side and being with all the paps and it's a very good insight to that whole world. But having done it kind of intensely for that period of time, no, I wouldn't go back."
Kaye added that playing Pennis was like having "a second childhood".
"I only ever made three half-hour shows so I'm always amazed that people remember it," he said. "It was good fun, it was like a second childhood, a second adolescence, but I wouldn't want to go back to it."