Taylor, who recently came second in the BBC's 'Sports Personality Of The Year' award, was criticised by Sky commentators Sid Waddell and Stuart Pyke for his below-par performance in the first round match against Gary Mawson.
The darts legend said that the subdued style of his 3-0 victory in the match should have been balanced against the fact that it took place four days after his grandson Jack was born nine weeks premature.
"They were going on about my darts, my flights, something else. They should do their homework a bit, not jump on me straight away," Taylor told reporters.
"I probably had about two hours sleep in four days. It was a very worrying time for us. My daughter's very fragile as it is, without them saying things.
"They really don't know what they are talking about at times. I didn't watch the match back on TV, I got feedback from my family and friends. It was very upsetting for the family.
"What do they (Taylor's family) think of them? They hate them. Not so much Sid Waddell, mostly Stuart Pyke. I'm telling the truth. I didn't like the things they came out with, it was very hurtful. At times they say things they shouldn't."
Taylor, whose grandson is now out of intensive care, added: "I love Sid [Waddell] to bits but I'll give credit where credit's due and where it's deserved. If he says the wrong thing, I'll tell him. He does it with me as well.
"Not only has he been one of my biggest fans over the years but also one of my biggest critics. I don't mind that. It's fine and I agree with it. But they should have wound their necks in a bit and gave us a fair shot."
According to the Daily Express, Waddell was surprised by Taylor's reaction, while Pyke believes that his commentary was fair and balanced.