The boyband, who were voted out in last night's (December 2) semi-final, said they are more focused on "getting their first song right" than trying to beat their rivals, who themselves finished third in X Factor's 2010 series.
Asked by Digital Spy what they have that One Direction do not, Josh Cuthbert replied: "We don't have the record sales and the millions of fans. You can't even compare us to One Direction, they've done so well.
"We've got to concentrate on getting the first song right."
The group also touched on former X Factor contestant Ella Henderson's recent comments hinting at a romance with Cuthbert, insisting that they are all merely "her big brothers".
"It was a bit blown out of proportion... things that she said were taken the wrong way," Jaymi Hensley said.
"We're all close to Ella and Josh is just one of those charming boys that make you feel good about yourself, and I think that's what she liked."
Union J went on to describe their mentor Louis Walsh as "the best thing that happened to us" and insisted they would be remaining close with the 60-year-old.
"He's been a mentor to us on the show and I'm sure we'll look at him as a mentor forever," Hensley said.
"He's been a great part of our lives and our journey with Louis hasn't ended. He's been a great friend and that's something that will last forever."
Union J declared that this week's X Factor final is "anyone's game", arguing that James Arthur's "angst" and Christopher Maloney's "power" could see either of them finish above current favourite Jahmene Douglas.
On the controversy surrounding cruise ship singer Maloney's progress, Hensley stated: "What is quite upsetting is... aside from all the stories and aside from [if] he's relevant or not, there's no denying Chris has got an awesome voice. He's got a massively powerful voice."
Photo gallery - X Factor latest results show:
Copyright: Ken McKay/Thames/Rex Features