Tulisa began criticising Barlow and his act Christopher Maloney after he performed Cutting Crew's hit '(I Just) Died in Your Arms'.
She explained: "I thought your vocals were very strong," before turning to Barlow and saying: "How many of these '80s classics are you going to let him keep destroying? Every week he's getting the same critique from all of the judges. I even feel bad having to say the same thing."
Barlow then responded to Tulisa, saying: "I don't know what's offended me more. What you've said or your fag ash breath."
Tulisa, looking aghast, replied: "Just a note for Gary - lay off the red wine because I can really smell that as well."
Judge Nicole Scherzinger described Maloney's performance as like "an '80s Halloween pop opera", with Louis Walsh added that he was reminded of classic singer Tony Christie while watching the 34-year-old on stage, to which Maloney replied: "Thank you."
Walsh also said: "I feel like we're looking at a future cabaret or panto act."
When asked by presenter Dermot O'Leary how he responded to the criticism, Maloney said: "I mean, all I'm doing is my best. I'm working my socks off and doing my best every single week."
Prior to his performance, Barlow had said that his mum was "a big fan" of Maloney's.
Photo gallery - The X Factor week four live show:
Copyright: Ken McKay/Thames/Rex Features