Barlow was performing in East London when he criticised the flamboyant contestant, according to The Sun.
The X Factor judge is quoted as asking his fans: "I'm serious. What are we going to do about Rylan?
"I might have to be honest on Saturday night and go, 'Rylan, I thought that was absolute s**te'."
When he asked the audience what they thought of Clark, they reportedly booed and chanted "Get him out", to which Barlow responded: "I'm glad you agree!"
The Take That star has had a series of clashes with Clark throughout the series, with Barlow repeatedly stating that the 24-year-old has stayed over better singers in the competition, including his own acts.
Clark recently suggested that the pair should perform a duet if he reaches the X Factor final.
Barlow, meanwhile, received his OBE from the Queen at Buckingham Palace yesterday (November 21).
Photo gallery - 'X Factor' hate figures:
Copyright: Rex Features