The 26-year-old, who admitted during his audition that he has been sleeping rough for the past six years, said he found it hard to fit in with the other contestants.
"I find it a lot more comfortable to be by myself and just chill out," he told Dermot O'Leary backstage, before leaving fellow singers Jake Quickenden and Adam Burridge to practice by themselves.
He is later seen asleep on a sofa, with his hat covering his face.
When they were called to the stage to perform, the remaining boys were clearly unimpressed with Hance's attitude.
"Robbie has decided to go for like a sleep which kind of stopped us from three hours of practice," said a frustrated Quickenden.
"It messes up our chances as well," explained Burridge. "We've got to perform it as a group.
"It looks like he doesn't care about it. It could be that he's just covering up in case he gets booted out or in case he doesn't make it through."
Quickenden said: "If he messes his part up, it could jeopardise everyone. I mean he's a top lad but... who knows... deep down he might want it more than anyone, just might be a front."
They added: "Hopefully it sorts itself out and hopefully it goes well."
On stage, the trio are seen explaining what had happened earlier in the day to Gary Barlow.
"How have the rehearsals been going?" the Take That star asked.
"We rehearsed well last night and today Rob's decided to go for a power nap," replied Quickenden
"He went missing a few times today, Rob, didn't ya?" added Burridge.
"Do you feel prepared?" asked Barlow.
Quickenden and Burridge said they did, but Hance replied: "To an extent but who does?"
As they begin to sing, it soon becomes apparent that Hance's absence in rehearsals has affected his performance. He is seen forgetting the words to the song and - assuming he has messed up his audition - placing his microphone on the floor and walking off stage.
After Hance disappears, Quickenden and Burridge are left to finish the performance and await the judges' verdict.
The X Factor Boot Camp stage, which includes the conclusion of Hance, Quickenden and Burridge's group performance, airs tonight and tomorrow (September 22, 23) on ITV1.