The actor - whose new show Cult premieres tonight (February 19) on The CW - previously played Alaric on the network's paranormal teen drama.
"I had my creative differences along the way with the character, and while [exec producer] Julie Plec and I are dear, dear friends, she knew that," Davis told TV Guide.
"I'm very opinionated and when you're put in a position where you're getting the storylines that are not necessarily what's cranking at the back of your mind, or digging at the potential of your character and you have to sit on your hands, that's frustrating."
Davis explained that he resented being "a secondary character" on The Vampire Diaries, motivating his decision to join Cult.
"Julie saw a really great way to wrap up my character, which was in alignment with the end of my contract, and I couldn't have hoped for a better execution for winding down Vampire Diaries and segueing into Cult.
"I called my agents right away and I said, 'I'll be a free agent soon enough. Get me back out on the market'. That was on a Friday and they called me on a Monday with an offer for this pilot Cult."
Cult will air on Tuesdays at 9/8c on The CW - the series is yet to secure a UK broadcaster.
> Prison Break star Robert Knepper joins The CW's Cult
> Supernatural star Alona Tal joins The CW's Cult
Watch Matt Davis and the stars of Cult talk to Digital Spy below: