Carey signed a contract to become an Idol judge last month, filling one of the seats vacated when Jennifer Lopez and Steven Tyler decided to leave the show.
However, Carey hung up the phone when she was told that Minaj is a frontrunner to join the American Idol panel, TMZ claims.
Sources have alleged that Carey is unhappy as she was led to believe that she would be the only woman on the new American Idol panel.
The website also says that Randy Jackson's future on the show is unclear and he may need to be replaced by another new judge. Other sources have suggested that even if Jackson does stay, a second male judge could be hired to even up the panel.
Brad Paisley, Keith Urban and Enrique Iglesias are among the names thought to be in the mix for a job on American Idol.
Nick Jonas has previously confirmed that he is being considered for the role, with reports claiming that Carey had the final say on whether he could join the panel.
P Diddy and Adam Lambert are among the other stars who have previously been linked to the American Idol judging spots.