Clark's auditions have divided viewers throughout the series and many fans complained when he was selected over acts such as Joseph Whelan and Jake Quickenden at Boot Camp and Judges' Houses.
However, Walsh insisted that he would have made the same pick as Nicole Scherzinger if he had the Boys category.
"I would have picked him, I would definitely have picked him. Everyone is talking about him!" Walsh told Digital Spy.
"He's a household name already. Not everybody is going to love you, but that's okay. You are never going to please everybody."
Meanwhile, fellow acts Lucy Spraggan and Jade Ellis have also defended Clark's spot on the ITV1 show.
Watch Tulisa and the X Factor Girls talking to Digital Spy below:
"Rylan is one of the greatest and best people I have ever met in my life," said Spraggan.
"He is amazing, he is hilarious. He is exactly the same on-screen as he is off-screen."
Ellis added: "He is talented and he does deserve his spot. He should be given a bit of a break. He's gone through the same procedure as us and it's gruelling. He's not a joke."
The X Factor Live Shows start on Saturday at 8pm on ITV1.