The Australian fashion designer, who caused controversy earlier this week when he referred to a size eight Australia's Next Top Model contestant as "overstuffed luggage", said that his comments were taken out of context and he was talking specifically about her modelling shot instead of her body shape.
Perry told the Herald Sun: "Somehow everyone thinks I hate fat people... I have somehow become the new skinny ambassador. I can dress a big girl better than anybody. When we started fashion week, I have always used girls like Megan Gale and Erika Heynatz... who are tall and have always been curvy.
"It was the fashion editors who were always saying, 'Uh, he is using those girls again', and not heroin chic, who they were using at the time."
Perry reminded his fans that his clothing label goes up to size 16 and his custom-made designs will fit size 24 people, before insisting that he will be using plus-size models in his next runway collection.
"I am going to do a beautiful show and we will use girls who wear plus sizes," he said. "I hate terms 'curvy', I hate 'plus', they are just models. The minute you say 'plus size' people just say, 'Uh, that means fat, right?' So I don't say it."
Perry admitted that he can make "acidic" comments, but added that he has "never intimated a girl was fat" in seven years on Australia's Next Top Model.
Aussie TV star Ajay Rochester recently accused Perry of "hating fat women". Perry later called her claims "reprehensible".
Watch Kerri-Anne Kennerly discuss the weight debate with Alex Perry on her morning show below: