The 25-year-old gave birth to her son Luca in March 2012 but told Life & Style that it is "hard to be away" from him.
Duff, who has been married to hockey player Mike Comrie for two and a half years, explained that she had a tough time while filming a guest appearance on Fox comedy Raising Hope.
"I was so happy to be on a set again," she said. "But it was hard. I also had a day of, 'I need to get home - what am I doing away from my baby?'"
She continued: "Honestly, it's such an adjustment. It's a huge change and it took a few months for me to be like, 'OK, this is me now, this is who I am now, this is how I do things now'."
Duff has previously said that she is not yet ready to expand her family, suggesting that she will wait until Luca is a little older.
View all of today's best celebrity pictures - Photo gallery: