The ballroom contest has so far lost out every week while going head-to-head with its ITV1 rival on Saturday nights.
In the most recent schedule clash, The X Factor pulled in 3m more viewers than Strictly. The talent competition had an audience of 10.86m between 8pm and 10pm, while BBC One's dance offering attracted 7.84m from 7.35pm to 9.45pm.
A BBC source told the Daily Mail: "We're holding our nerve about going head-to-head with The X Factor for the moment, but it is getting increasingly difficult when we see the ratings on Sunday mornings.
"I'm not sure how much longer Strictly will be on for so long against The X Factor. It's not doing the show any good."
The BBC faced criticism last month after announcing that Strictly would have a later slot in competition with The X Factor. Simon Cowell and Strictly dancer Brendan Cole were among those who condemned the move.