The Rock of Ages star was confronted by paparazzi when leaving the city's marriage license bureau with fiancée Hilaria Thomas.
Baldwin is said to have physically lashed out at New York Daily News freelancer Marcus Santos after asking him to step away, while a second photographer for the publication caught the ensuing altercation.
"I knew he was going to attack me," Santos said in the Daily News. "He comes after me, starts shoving and punching me - one time, right in the chin.
"He was like crazy, you know. There was an eruption of mad. You're going to get a marriage license, you should be happy. What happened with this guy?"
Baldwin's representatives have denied that he punched Santos.
The 30 Rock actor later addressed the incident on Twitter, writing: "A "photographer" almost hit me in the face with his camera this morning. #allpaparazzishouldbewaterboarded.
"I suppose if the offending paparazzi was wearing a hoodie and I shot him, it would all blow over..."
Baldwin became engaged to Thomas in April and the couple will reportedly marry in New York City on June 30.
Watch the trailer for Rock of Ages below: