He will star opposite Scarlett Johansson in the romantic movie, reports Variety.
The film follows a struggling singer-songwriter (Johansson) who moves to New York City in the hopes of launching her career.
After being dumped by her boyfriend, an unlucky music producer (Ruffalo) helps her with her career and slowly falls in love with her.
The pair are due to star in Joss Whedon's Avengers in April.
Writer/director John Carney is known for his work on the musical romance Once. Formerly a member of Irish indie-rock band The Frames, Carney directed his one-time bandmate Glen Hansard in the movie.
Carney will also direct Steve Carell in drama Dogs of Babel, which is currently in pre-production.
> Ten Things About... Scarlett Johansson
Watch the extended Super Bowl trailer for The Avengers below: