The Australian fashion designer, who is also a mentor on Australia's Next Top Model, was initially accused by TV star Ajay Rochester of "hating fat women" but called her claims "reprehensible".
Perry was then labelled a 'fattist' after describing size eight Next Top Model contestant Alissandra Moone as "overstuffed luggage" in an episode of the show last week.
Braybrook has now claimed to the Herald Sun that Perry and fellow contestants on the reality series would often make jibes about curvy people on set.
"On the show he was always like, 'I hate fat people'," Braybrook said. "He used to say it all the time."
Perry has hit back at all the rumours, calling them "completely and utterly false". Last week, he said: "Somehow everyone thinks I hate fat people... I have somehow become the new skinny ambassador. I can dress a big girl better than anybody. When we started fashion week, I have always used girls like Megan Gale and Erika Heynatz... who are tall and have always been curvy."
Watch TV host Kerri-Anne Kennerly discuss the weight debate with Alex Perry on her morning show below: