Waterman claimed that the acts on the show are "irrelevant", saying that all Cowell is concerned about is viewing figures.
"I think everybody's got the show wrong, there is only one star in the show and that's Simon Cowell. It's the Simon Cowell show and that's what people watch, all the rest is irrelevant," the record producer told Live From Studio Five.
"I believe that you know, when the producer or director talks to Simon ten minutes before he goes off air, the director says come up with something that we can keep running until Saturday night and he just drops it in."
Speaking about Irish brothers John & Edward, Waterman said: "When Simon gets that 18 million audience and he says thank you very much lads, it will be see you later, you've served your purpose.
"Then they'll go round the clubs, for you know three grand, they'll make a good living. They've got no talent, they can't sing, they can't dance."
Waterman, who worked with Cowell on two series of Pop Idol, also claimed that the 50-year-old used to be jealous of all his fan mail.
"I spotted in Simon a ruthlessness which I've never seen anywhere," he said. "Simon has one fixation and that's being successful and he'll cut through anything.
"Everybody forgets that in the first series of Pop Idol I was the bad guy and got all the fan mail and then suddenly Simon did a quick change when he saw how much fan mail I was getting."
Live From Studio Five airs weekdays at 6.30pm on Five.