The executive producer of the series will reportedly tell the singer that she will not be asked back "within weeks", according to The Sun.
A source close to the production team claimed: "Quite apart from her performance on last year's show, you can't have a singer whose album only reached number 35 in the charts sitting in judgement on other singers.
"Tulisa has been on the panel for two years. She had a good run, but all good things come to an end."
Meanwhile, Cowell is said to be fighting "tooth and nail" to keep Louis Walsh on the panel, who other producers also reportedly want to axe.
An insider said: "He will fight tooth and nail to keep him on the show.
"It hasn't reached the shouting stage yet — but if it does, Simon is prepared for the fight."
An X Factor spokesperson said: "No formal decisions have been made on the panel for 2013's X Factor as yet."
Tulisa was previously rumoured to be facing the axe back in January, but she denied the reports.
Robbie Williams is the latest star to be linked with a permanent spot on the judging panel.
Watch Tulisa's video for 'Young' below:
Tulisa's 'TFB for Bank' pop-up stall at Westfield Shepherd's Bush - photo gallery:
Copyright: Eamonn McCormack/Getty Images for TFB by Tulisa