The actress was dropped from the CBS crime drama in 2010 and shot a comedy pilot for NBC, but was later made to return to Criminal Minds full-time for its seventh season as a year was still remaining on her contract.
However, once her contract was up, Brewster quit her role as Emily Prentiss in February of this year.
"I'm not saying anything to denigrate Criminal Minds; that's a great show," she told TV Guide. "I just didn't appreciate it anymore. I appreciate those people, but I realised my heart wasn't in it and I needed to go because plenty of people would rip their arm off to be on that show, so they should be."
Brewster added that being forced to return to the series following her initial departure "did sort of sour the experience" of her brief return.
"I wasn't going to dig in my heels and say, 'Go ahead, sue me. I'm not going back. It's a horrible place'," she explained. "I don't know if I would have left if that hadn't happened, but it was heartbreaking."
However, the 43-year-old admitted that she does still miss being a part of the Criminal Minds cast, which recently welcomed new series regular Jeanne Tripplehorn.
"I'm not going to lie - I'm jealous of Jeanne Tripplehorn, which is ludicrous because I decided to leave," she said. "You hear, 'Oh we're doing this this week and these are our guest stars', and there's a little tinge of regret. But I don't ultimately regret it."
Criminal Minds returns to CBS tonight at 9/8c. The series airs on Sky Living in the UK.
> Mandy Patinkin regrets Criminal Minds: 'That show was a big mistake'
Watch a video recap of Criminal Minds's seventh season below: