The Salford-born entertainer believes that fellow stand-up comic Carr has made "enough" jokes at other people's expense in his career to weather the storm.
Speaking on radio station Magic 105.4, he opined: "Jimmy's definitely dished it out enough over the years, so I think he can definitely take it."
Manford, who worked with Carr on 8 Out of 10 Cats between 2007 and 2010, appeared to question the taste of the Channel 4 panel game.
When asked whether he missed being a regular guest on the programme, the former One Show host said: "I'm not a comic who likes a victim."
Carr's £3.3m tax avoidance revelations incurred the wrath of David Cameron, but the 10 O'Clock Live star reportedly later hit back at the Prime Minister during a gig in Stockport.
Watch Jimmy Carr's 8 Out of 10 Cats roasting below: