The 'Candy' singer recently moved to Wiltshire in the UK with wife Ayda Field and their baby Theodora Rose, but still owns his US property.
"Robbie has a fleet of staff in LA, people to look after his plants, his terrapins, his dogs, his paintings as well as cleaners, butlers, chauffeurs and security," a source close to the family told the Daily Star.
"It's like a giant luxury world class hotel with swimming pools, lake and gardens but no guests.
"The staff are doing their chores but with no-one living there it's pretty sad."
Williams and Field welcomed their first child together last month, and are looking to settle down in the UK to raise her.
However, the singer reportedly plans to keep their LA home in case he ever wants to have a change of scenery.
The source added: "Ayda is happy to get rid of the LA house but Robbie hates the UK weather and wants to be able to go back to California at the drop of a hat, so it's worth it to him."
Robbie Williams has been tipped as a guest fifth judge on The X Factor alongside his Take That bandmate Gary Barlow.
He has also revealed that he intends to send his daughter Theodora to a private school.
New! Digital Spy breakfast show - watch Anna Williamson discuss the day's biggest stories including The X Factor, Homeland and Leona Lewis: