The prime minister took issue with Carr reportedly sheltering £3.3 million a year in a Jersey-based scheme.
"I think some of these schemes - and I think particularly of the Jimmy Carr scheme - I have had time to read about and I just think this is completely wrong," The Guardian quotes Cameron as telling TV reporters.
"People work hard, they pay their taxes, they save up to go to one of his shows. They buy the tickets. He is taking the money from those tickets and he, as far as I can see, is putting all of that into some very dodgy tax avoiding schemes."
He added: "That is wrong. There is nothing wrong with people planning their tax affairs to invest in their pension and plan for their retirement - that sort of tax management is fine. But some of these schemes we have seen are quite frankly morally wrong.
"The government is acting by looking at a general anti-avoidance law but we do need to make progress on this. It is not fair on hardworking people who do the right thing and pay their taxes to see these sorts of scams taking place."
However, Cameron's spokesperson previously defended the OBE given to Take That's Gary Barlow, who allegedly avoided tax in a different scheme.
Barlow, together with manager Jonathan Wild and bandmates Mark Owen and Howard Donald, were reported to have put £26 million into the scheme.
Lawyers for the bandmates involved insisted that they thought the investments were legitimate enterprises and not tax avoidance schemes, and said that they all pay "significant tax". There is also no suggestion of any illegality.
A spokesperson for Prime Minister David Cameron told The Daily Telegraph "it was important that people pay the right level of tax".
They added of Barlow's OBE: "The Diamond Jubilee celebrations were a great success and Gary Barlow played a big part."