Kacey gave birth to the couple's first child Kline Olivia Diaz on Monday (July 30), according to People.
Eli said of the baby's birth: "They tell you about this feeling that is beyond comparison... and the moment Kacey and I laid our eyes on little Kline we understood it's absolutely the coolest thing ever!"
Kline Olivia Diaz weighed in at 6 lbs, 15 oz and was 20¼ inches long at the time of her birth.
Eli Young Band recently scored back-to-back US country number one hits with 'Crazy Girl' and 'Even If It Breaks Your Heart'.
Their latest LP Life at Best hit number six on the US pop charts in 2011.
Watch Eli Young Band's 'Even If It Breaks Your Heart' music video below: