The Anti-Counterfeiting Trade Agreement (ACTA) was rejected by 478 MEPS in the vote, while just 39 supported it and 165 abstained.
The outcome was considered to be likely after the Committee on International Trade INTA also voted against the legislation last month by 19 votes to 12.
INTA is a body that formally recommends how the European parliament should vote.
ACTA had been signed by 27 member states, including the UK, but today's vote means that the European Parliament will not actually ratify the legislation at this stage.
The agreement covered the counterfeiting of physical goods, but campaigners hit out at the punitive anti-internet piracy proposals, such as fines and imprisonment for online copyright infringement.
There have been fears that the act would restrict internet freedoms and also trample over the civil rights of EU citizens.
Martin Schulz, the president of the European Parliament, said the vote against ACTA was because the legislation is "too vague" in its current form.
"The majority in the European Parliament is of the opinion that ACTA is too vague, leaving the room for abuses and raising concern about its impact on consumers' privacy and civil liberties, on innovation and the free flow of information," he said.
Ever since the initial draft of the bill was released on Wikileaks in 2008, campaigners warned that it represented a potential threat to freedom of speech online.
It was also alleged that the important issue had been debated by lawmakers in secret.
Its defeat marks another blow for the global media industry, after the Stop Online Piracy Act (SOPA) was dropped in the US earlier in the year after a broad protest campaign.
MEP David Martin, a member of INTA, said after the committee's vote last month that the legislation just left too many "questions unanswered", such as what role ISPs should play in policing the web.
He also said that some of the proposed sanctions for breaches of copyright were "disproportionate".
Jim Killock, executive director of the Open Rights Group, welcomed the EU's decision and urged lawmakers not to revisit the legislation at a later date.
"This is a tremendous victory for the movement, for democracy and for every European citizen that has demanded that their rights be respected," he said.
"ACTA must be abandoned. The Commission must drop its calls to try again. ORG would like to thank the thousands of activists from the UK that helped persuade MEPs to stand up for democracy."
However, EU commissioner Karel de Gucht has said that the commission would await the decision of the European Court of Justice (ECJ) on the act, before deciding whether to revive the legislation.
The ECJ was asked in February to assess whether the treaty is compatible with "fundamental rights and freedoms of EU citizens".
"This attempt to delay this issue further and pull the wool over our eyes will fail so long as the Open Rights Group and activists like you remain vigilant," said the ORG in a message to its supporters.