The actor will become a series regular in the HBO drama's third season, Deadline reports.
His character Gyp Rosetti is described as a charismatic but "ruthless" gangster.
Gyp ends up becoming a rival for Nucky (Steve Buscemi).
Cannavale recently finished filming a role on Showtime's Nurse Jackie and has previously appeared on shows such as Third Watch, Ally McBeal, Will & Grace, Cupid and Cold Case.
Boardwalk Empire, which is expected to return for its third season on HBO later this year, recently won two honours at the Screen Actors Guild Awards.
> Boardwalk Empire creator explains finale twist