The ratings for the BBC One show have fallen since its launch, with the initial blind audition stages gathering up to 12 million viewers at its peak.
However, the live shows have seen it recently fall to around five million or less.
Dutch creator De Mol - who has also been behind the hugely successful Deal Or No Deal and Big Brother - explained that the format will have to be changed for the British version's second series.
"We'll have a meeting with the BBC to come up with some new ideas to make the live shows more exciting," he told Radio Times.
"The role of the coaches can be bigger. In the semi-final, the coaches were just watching the talent perform. They had no influence. That's something we should change.
"On Saturday, it was 100% up to the audience."
He added that the show's positive image in comparison to other talent series may need to be altered, saying: "It's something we must work on. More competition."
However, De Mol also said that he wanted the four coaches to return to the series, and that the British contestants were some of the best of all the global editions of the show.
"We'll talk about the coaches with the BBC right after the final this Saturday. As far as I'm concerned we have four great characters but it's too early to say."
Simon Cowell recently joked that any X Factor rejects should audition for The Voice instead.
Photo gallery - The Voice semi-final results:
Copyright: BBC Wall To Wall