The Take That star admitted that he thought Halliwell was "sweet", but "tried a bit too hard" to win the producers over with her antics.
"She's crackers," he told The Sun. "How you see her on TV is exactly how she is in real life.
"The crowd really didn't like her. She got booed all day long. She was a bit over-excited."
The 'Patience' star also revealed that Halliwell believed she was being trialled for a permanent place on the judging panel.
"She'd been told she was auditioning for the role and, bless her, I think she tried a bit too hard," he said.
"I do love her for being her, though, we had fun backstage. She's sweet but mad."
When asked if he thought the former Spice Girl would return to the show, the 41-year-old added: "I doubt it. And if she is, I may be going."
Halliwell joined the X Factor panel as a guest judge in Liverpool. Her bandmate Mel B later appeared at the Manchester leg of auditions and won praise from Barlow for her honesty with contestants.
"Mel was brilliant," he said. "It actually gave me a bit of a holiday the day she was there.
"I'm normally the one who delivers the bad news but she was doing it all so I could just sit there, going, 'Oh, that's awful, what she said' but secretly loving it."
The X Factor airs tonight at 8.10pm on ITV1.