The 60-year-old judge told The Guardian that bosses have somehow got to find a way of stopping the 'cheesy' Liverpudlian from succeeding.
Walsh said Maloney is also being accused of faking nerves: "The other contestants call him 'shake and fake'. He keeps getting really big votes, though."
It's thought that Simon Cowell is furious Maloney is still on the show as bosses don't think he will sell records if he wins.
On the official X Factor website, Walsh said: "Maloney is bland and boring. For some unfathomable reason, the public still like him. He really suits his new nickname; shake and fake.
"Chris can really sing though, I'll give him that."
Walsh added to The Guardian of this year's series: "We're probably missing Simon Cowell and maybe Sharon Osbourne."
However, he is in no rush to get rid of Tulisa, who he has grown close to: "She works hard, she's honest. I did say she's a chav in a tracksuit, but she's a good chav in a good tracksuit. She's a lovely girl.
"I was dreading meeting her. I thought she was going to be a right little knacker. You know, a real toughie."
Of the two girls he'd least like to meet on a dark night in an alley, he said: "Oh Cheryl."
Walsh continued to say that he isn't too bothered by Gary Barlow or Nicole Scherzinger's return next series: "Well, [Gary] has a lot of work to do. He has a tour to do. You need two girls. You could have Tulisa and Sharon."
Walsh also revealed that there was friction between him and Girls Aloud when he managed them at the beginning of their career - and later on with Cheryl Cole.
He said: "You want the real story? OK, I'm going to tell you. There was a girl in Ireland called Nadine Coyle, and I said, go for this show.
"She is an amazing vocalist. A real, real singer. So I pushed her in the band, and I said Nadine is the lead singer.
"From then on, there were problems. I never worked with them very well because they kind of scared me.
"And from the moment I said Nadine was the best singer, the others alienated me.
"Well, she is the best singer! By a mile. But she didn't marry a footballer. If she'd married a footballer she'd have been a big star."
On friction with Cheryl while she was on the X Factor panel with him, he admitted: "Yes. Cheryl knows how to work it. Cheryl is all about Cheryl."
On his hair transplant, he explained: "Simon said, 'Darling you're losing your hair', and I said, 'Darling, I'm not'.
"Then he showed it to me on camera and I was. He said, 'Darling, you should get your hair done'. Everything is 'darling' with Simon. So I said, 'OK darling, I'll get it done'."
But plastic surgery - apart from having his bags taken away - is a no-go for him: "No, and I've never done the gym in my life. I'm a fat f**k."
On pal Simon Cowell's preening, he said: "That's his own hair. But I'd say everything else has been done. He's very vain, and he's looking younger. That's all I'm saying."
Defending this year's X Factor, he said: "I think the press are trying to slag it. It's like the Manchester United of TV programmes. Everybody wants to slag it off, but they're still watching it."
Speaking about being falsely accused of molesting 25-year-old Leonard Watters last year in a toilet, he said: "It ruined me. I'll never get over it, I'll be quite honest with you.
"I'm wary of everyone now, of everything I say, everyone I meet, always looking over my shoulder – I was never like this before.
"It was the worst thing that happened to me. All I have in this business is my name, and I would have lost everything. People would have said there's no smoke without fire."
On being suicidal, he said: "I had the sleeping tablets. I have to be honest. I was in this daze. By the way, the guy's in jail now. I hope he's reading this in jail.
"The fact that somebody set out out to destroy me is a real strange thing to deal with. I've blocked this whole thing out of my mind. I hate talking about it."