The TV presenter, who has two sons with the late Big Brother star, told the Huffington Post that he recently had to put his foot down when Jackiey Budden decided to spoil her grandchildren.
"If any of you have experienced separation where kids are involved, you'll know the numerous pitfalls and stumbling blocks placed in our way when it comes to maintaining an amicable relationship," he wrote in a blog post.
"Well, recently I have been re-introduced to the feeling of 'them and us' by Jade's mum's little oversight. The boys love her so much and I truly believe she makes them feel closer to their mum than I ever could... but this always seems to come at a slight cost.
"She hadn't seen them for over 14 months so I guess she was going all out to compensate. I got a phone call from an excitable Freddy telling me he was now the proud owner of a hamster!
"As I explained to Jackie, it would have been nice to have been asked because now I've been put into a position where I have to be the mean parent who says no and ruins the fun. This has happened before with mobile phones.
"I can't fault Jackie when she's got the boys, I just wish we could work together a little better. I think I'll invite her round for dinner before she goes back, shows a little bit of willing I guess and if that reinforces the kids perception that the adults are all getting on fine then it'll be worth the effort."
Goody died in 2009 after a battle with cancer. Budden herself suffered a miscarriage last year.
> Jeff Brazier: 'Jade Goody legacy could hurt our children'
> Jeff Brazier: 'Son Freddie sees mum Jade Goody at night'