According to The Sun, the singer and husband Sam Cooper welcomed a daughter named Marnie Rose Cooper on Tuesday (January 8).
"Marnie is a real cutie," a source told the paper. "Sam and Lily are ecstatic about adding another girl to the family.
"A little sister for Ethel is brilliant news to lift the January blues for the family."
However, the couple are yet to make an official announcement confirming the news.
Allen had tweeted last week about trying to bring on labour by eating spicy food, writing: "3rd curry this week... nothing #cosywomb".
Cooper and Allen welcomed Ethel in November 2011.