According to the Daily Star, the actress, 29, will not join Williams at his new £7 million Wiltshire home because she wants to focus on her acting career in Hollywood.
"It's been a real blow for Rob," said an insider. "He felt that he had everything - the chance to move home, a rekindling of his relationship with Take That and all this with the love of his life by his side.
"Now Ayda has really thrown it all into confusion. She's staying in California. She's made that very clear, but she also wants Rob to be there most of the time.
"She's told friends that he must visit often if they want to work as a couple and really she thinks he should commute from their home in LA."
The source added: "It's hard for [Ayda] to really understand the emotional weight of the Take That reunion."