Kavanagh said that the tabloid was "not a swamp that needed draining" following calls that it should be shut down in the same way that sister title the News of the World was scuttled at the height of the phone hacking affair.
At the weekend, eight people, including five journalists from The Sun, were arrested under the Operation Elveden investigation into allegations of illegal bribes paid to police and public officials by the media.
The arrested journalists included various senior figures at the newspaper, including deputy editor Geoff Webster, chief reporter John Kay and picture editor John Edwards. A Surrey Police officer, a member of the armed forces and a Ministry of Defence employee were also arrested.
In a The Sun today, Kavanagh admitted that money sometimes changed hands while reporters were investigating stories, but said that this was standard practice. He also claimed that the "witch-hunt" against journalists had put Britain "behind ex-Soviet states on press freedom".
The Sun's former political editor said that at any other time the treatment of journalists in this way would have been met with uproar within parliament, along with human rights campaigners and those fighting for civil liberties.
He said that journalists were subject to the "biggest police operation in British criminal history - bigger even than the Pan Am Lockerbie murder probe", and there are 171 officers involved in the Operation Elvedon investigation, along with the Operation Weeting and Operation Tuleta probes into phone and computer hacking respectively.
Kavanagh claimed that one raid involved two officers being "pulled off an elite 11-man anti-terror squad trying to protect the Olympics from a mass suicide attack", while journalists had also been "needlessly dragged from their beds" and "humiliated" as their intimate possessions were searched.
"This inquiry has even begun to disturb those of our critics who have been at least partly responsible for what many see as a 'witch-hunt'," he said.
"The Guardian has raised questions about freedom of the press. Its media analyst, Steve Hewlett, says that when it comes to paying for stories, no newspaper - 'tabloid or otherwise' - is exempt. Yet in a quite extraordinary assumption of power, police are able to impose conditions not unlike those applied to suspected terrorists.
"Under the draconian terms of police bail, many journalists are barred from speaking to each other. They are treated like threats to national security. And there is no end in sight to their ordeal.
"Their alleged crimes? To act as journalists have acted on all newspapers through the ages, unearthing stories that shape our lives, often obstructed by those who prefer to operate behind closed doors.
"These stories sometimes involve whistleblowers. Sometimes money changes hands. This has been standard procedure as long as newspapers have existed, here and abroad. There is nothing disreputable about it. And, as far as we know at this point, nothing illegal."
The Sun editor Dominic Mohan has said that he was "shocked" by the arrests but pledged to continue to lead the paper during the current scandal.
But James Murdoch, the embattled executive chairman of The Sun publisher News International, told MPs last November that he would not rule out closing down the tabloid if it was found guilty of illegal behaviour or "wrongdoing".
> News International offices searched in police payments probe