The actor, whose final scenes as Charlie Slater air next week, was only aware of his character's role in the plotline until the full details of Ronnie Branning (Samantha Womack) and Kat Moon's (Jessie Wallace) scenes were announced last November.
Speaking on This Morning today, Martin explained: "I'm one of those actors - I don't know how many others do it - but when I get the scripts, I only read my bits. So I looked [and saw], 'Charlie leaves his grandson upstairs, goes downstairs, gets on the drink, gets drunk - the baby's dead'. So that's terrible guilt obviously - because I feel as though it's my fault that he's died.
"I was surprised when I saw the story come out - because I don't read the others' [lines] - that it was a baby swap."
Discussing the controversy surrounding the plot, the 77-year-old said: "Don't forget, there's millions of people - there's 60 odd million here, and they've all got different opinions. Every subject, everything that comes up… everybody will have opinions about that. So therefore you have to accept that's the public - some are for, some are against, and that's the way it is."
He added: "Over the years, don't forget, EastEnders have always had subjects. When [Charlie's] brother had raped Kat and we didn't find out until later on, that was a very strong storyline. And when Little Mo was bashed by her husband Trevor. These stories are all part of real life."
Earlier today, it was officially confirmed that the baby swap storyline will be cut short. Reports suggest that it could end as early as April.