The talent contest's viewing figures have fallen from a high of 10.7m to 6.6m.
© BBC/Wal to Wall/Guy Levy
"I don't get massively engaged week by week in whether we are up or down," Cohen told The Daily Telegraph.
"The way I look at it, we have launched the most successful BBC One entertainment show in over a decade. We didn't expect such high ratings at the start - shows like X Factor started on around five million."
Cohen insisted that he "couldn't be happier" with the reaction to The Voice UK.
"There are always moments when the competition is strong - sometimes that helps us, sometimes it doesn't," he suggested. "But... it is a quality series and audiences are responding to it."
The Voice UK coach Jessie J was recently rumoured to have angered producers after describing elements of the show as "lame".
