In an interview with This Is Exeter, the actor explained how his time on the soap came to an end.
"After ten years, it was time for me to leave. I told the producers I wanted to have some time off but they said no, so I booked myself in a panto anyway," he stated.
Williamson added: "I'm honestly happy there's no prospect of me going back. And if there was and they asked me, I'd say no."
Despite this, the 43-year-old said that he was more than prepared to let Barry's spirit live on in Ricky Gervais's Extras.
"I still don't know how Ricky got my number. He told me that everyone was going to get the fun taken out of them on his new show but I didn't care," he said.
Williamson explained: "It was a really different experience to my work on EastEnders. Ricky and Stephen Merchant both directed as well as starring in the show, so we worked really quickly and they would change things as we went along."
The actor went on to say that he does not mind mainly being remembered for his soap character.
"If I have been typecast as Barry I've brought it on myself by letting it run on with Extras," he acknowledged.
"But I think it's better to be typecast than not remembered. People do shout 'Barry' at me in the street. But he was a popular character who got all the comedy lines among the murder and mayhem, so any attention has been pleasant."