The 48-year-old comic was alleged to have used the word "mong" - a derogatory term which refers to Down's Syndrome sufferers - in a demeaning fashion.
However, writing on his website, he announced: "The Down's Syndrome Association contacted my PR on May 6 explaining they had had a report about my routine about Susan Boyle and voiced their obvious concerns.
"I sent them an email [in which I wrote], 'I clearly explain that words change and that at no point am I referring to anyone with Down's Syndrome'."
He added: "'Not only am I not referring to people with Down's Syndrome I also explain that I am not associating the word with its old derogatory meaning. I also do it as part of a routine about comedians taking responsibility for everything they say. I hope this is acceptable'."