The director has criticised the veteran action star following the film's critical panning and failure at the box office.
He told The Times of India: "[Producer] Dharam Oberoi told me Sanjay wanted Kangana to be dropped and since it's a predominantly male-dominated action film, I didn't want to make an issue.
"The problems were innumerable, ranging from Dutt's coming late, to last-minute cancellation of shoots and schedules, to erratically wanting to change the content of scenes in the name of creative discussions that are not to be confused with mindless interference.
"The difference being Aamir's suggestions and observations came from a long and intense deliberation on his part and not like Dutt who was blurting out without thinking, encouraged by the likes of Dharam who pander to him for their personal benefits."
He added: "I am not blaming anyone except myself. The biggest mistake I made was to go to an actor as unprofessional and as unimaginably erratic as Dutt."
Varma recently said that item songs are more important than the content of films when it comes to publicity.
Dutt has commented that Department marked a change of direction for him because he has never done such a "hard-hitting, realistic, raw, gritty action-drama".