Speaking to The Guardian, Hunt reiterated his support for maintaining the corporation's independence from government.
He also said that the coalition is "committed to the principle that the BBC should have a ring-fenced pot of money over a multi-year period".
However, Hunt believes that the changing way people are watching TV, particularly in terms of the growth of online viewing, could soon make the annual licence fee obsolete.
"We support the principle of the licence fee and always have done," he said.
"But we also recognise, as technology changes, we may need to adapt the way it's collected. It is not going to be possible to have a tax every time anyone buys a computer."
Hunt, who was critical of the BBC's pay to its top executives before the election, said that the public just wants "value for money" from the corporation.
"All I can do is advocate changes at the BBC while respecting editorial independence upon which the success of the BBC rests," he said. "I can't do anything that requires the BBC to pay certain people certain amounts."
Former BBC director general Greg Dyke said last year that the TV licence should be scrapped to save around £100m a year in administration costs.
Dyke made the claim while conducting a review of media policy for the Tories, but Hunt said that he has not yet seen the final report.
"I hope Greg will deliver that report. I never saw sight of it before the election and I shall look forward to reading it," said Hunt.
He added: "If you've got any ideas [on what to do about replacing the licence fee] pop them on a postcard please. I seriously don't know. It's think it's a really difficult problem."
Hunt's department of culture, media and sport is currently attempting to cut back on aspects of regulation which date back to a "pre-internet era".
A particular area of focus is the contract rights renewal (CRR) for advertising airtime on ITV1, which was introduced in 2003 to protect advertisers from the loss of competition when Carlton and Granada merged to form ITV plc.
"CRR is an example of micro-regulation that we can do without. Telling people the price at which they can sell airtime is an example of regulation inherited from the pre-internet era," said Hunt.
"It's very important if we are going to have, not just a strong BBC but strong competition to BBC, that we look at whether that regulation is appropriate."
Last month, ITV chief executive Adam Crozier called for an end to the CRR to help "sustain a vibrant independent broadcasting sector that can rival the BBC and compete on a global stage".