The new law will mean that sugary drinks larger than 16oz cannot be served in restaurants, cinemas, stadiums and arenas, Metro reports.
Soft drink companies including Pepsi and Coca-Cola opposed the prohibition, saying that consumers should be free to select the size of drink they want.
The purpose of the ban, proposed initially by Mayor Michael Bloomberg, is to fight obesity.
"This is the single biggest step any city has ever taken to curb obesity, certainly not the last step that lots of cities are going to take," Bloomberg said.
"We believe it will save many lives."
Businesses have six months to stop selling oversized drinks, after which they will face fines of up to $200 (£123) for failing to comply.