The Sopranos star will play Leander Whitlock, a contemporary of Commodore Louis Kaestner (Dabney Coleman), according to TV Guide.
Boardwalk Empire creator and former Sopranos writer Terrence Winter claimed that Chianese will be "almost unrecognisable" in his new role.
The actor played 'Junior' Soprano from 1999 to 2007 and has also appeared in episodes of Law & Order and Damages.
Chianese will appear in six episodes of Boardwalk Empire's second 12-part run.
The Untouchables star William Forsythe has also joined the cast of the period drama as gangster Manny Horvitz.
The new season of Boardwalk Empire begins on September 25 on HBO. It will air in the UK on Sky Atlantic in October.
> Boardwalk Empire season 2: First Look
Watch the new trailer for Boardwalk Empire season two below: