The stand-up suggested to ShortList that people can avoid material that offends them if they choose.
Of the recent storm over Ricky Gervais's use of the word 'mong', Carr said: "I would describe it as being his turn this year. It was my go two years ago, and it was Frankie Boyle's last year."
He continued: "I don't think boundaries are there to be pushed, I just think that's his sense of humour. People have the right to be offended.
"But then, he's also a pretty easy guy to avoid if you don't like what he's saying. It's not mandatory to follow him on Twitter."
Boyle was criticised earlier this year for his material about Katie Price's disabled son Harvey.
Asked if he ever feels guilty about telling a joke, Carr said: "No. I wrote it, I rehearsed it and then I said it.
"Some thought has gone into that. You don't say things by accident as a comedian - you've got to stand by what you say."
Watch Jimmy Carr tell the 'most offensive jokes' below: (CONTAINS SWEARING)