Murray was convicted of Jackson's involuntary manslaughter on Monday and will remain behind bars as he waits to be sentenced on November 29.
The physician was found guilty of administering a lethal dose of the anaesthetic propofol to Jackson in the moments before his death on June 25, 2009.
Jermaine was in the courtroom with several members of the Jackson family as the verdict was read this week and watched as Murray was led away in handcuffs.
The Jackson 5 singer told The Ellen DeGeneres Show that Murray's maximum sentence of four years behind bars simply isn't enough punishment for contributing to a man's death.
"I think the sentencing [is] very soft. I mean, you would probably do more time if you stole," Jermaine said.
He continued: "[If] you went into the hospital and stole cases of propofol you would probably do more time… To give him two years for taking a life. Forget about Michael Jackson but taking anyone's life?"
Jermaine's sentiments about Murray's sentence were echoed by his brother Tito Jackson, who commented that Murray should spend the rest of his life in jail.
Sister Rebbie Jackson also admitted that she felt no "relief" from Monday's verdict, but is happy to see Murray held accountable for his actions.
Jermaine said that although the trial has concluded, the Jackson family don't necessarily have the closure they need to move on from Michael's death.
"There were a lot of questions that the family had - and we're still waiting for answers," Jermaine said.
Jermaine's full interview on The Ellen DeGeneres Show airs on Wednesday.
> Michael Jackson doctor Conrad Murray found guilty - trial in pictures
Watch a preview of Jermaine Jackson's Ellen DeGeneres Show interview below: