The support worker impressed the judges on Saturday's show, but reportedly left at the Boot Camp stage for her sibling, who is battling a drug addiction.
© ITV / THAMES TV/SYCO
© ITV / Thames/Syco
"My younger brother Gareth has been on hard drugs for many years, and he's lived with me for most of those years," Cartwright is quoted as telling The Sun.
"I made a deal with him - he'd keep off drugs and I would go for it on The X Factor. But he broke his promise just before Boot Camp. He was in a bad way and having lots of problems.
"I had reservations about going, but I went along. When I was in Liverpool, I had an anxiety attack and I knew I couldn't do it. I couldn't focus on anything. I was worried about how it could all affect Gareth and wanted to support him. His wellbeing is more important to me than fame or money."
Photo gallery - 'The X Factor' 2012, episode 6:
Copyright: ITV THAMES TV/SYCO
Cartwright added that once he heard that she had quit the show for him, her brother overdosed on heroin, but has since recovered."He said he just wanted to fall asleep and not wake up again. He did that because he felt he was a hindrance to me," she recalled.
"This is not a sob story - I don't want that, I love my life. Now we have to look forwards, not backwards. I'll definitely give X Factor another go next year."
Fellow X Factor contestant Robbie Hance has reportedly gone missing since his successful audition.
Watch Tammy Cartwright's X Factor audition below:
