The reality TV star announced that she would be leaving the ITV2 show during its Christmas special last month.
Bright explained that she is looking forward to "to [getting] away from James to start afresh".
She told Reveal: "The show would be forever setting up scenarios where they'd send James to wherever I was in order to create drama.
"It made it unbearable."
Bright is currently dating fellow TOWIE star Tom Kilbey.
She added that she was concerned no other man would love her again following her relationship with Argent.
"I didn't think any man would want me after James," she explained.
"I was like the tarnished girl. No-one would bother going on a date with me, even if they liked me, because they knew there would be chaos.
"I was dreading my past relationship coming back to haunt me like this."
Back in September, Lydia Bright told Digital Spy that she had no plans for leaving the reality series.
Watch Lydia Bright and Tom Kilbey talk to Digital Spy below: