Comedian Conley took exception to the heavyweight boxer's careless attitude when he disturbed some burning logs by poking around in the campfire.
After jumping up to collect the errant embers, Conley warned: "No, that's fire and fire's dangerous."
Not understanding the 51-year-old's frustration, Haye made the situation worse by telling Conley to throw the hot wood over towards the vegetation at the back of the camp.
Haye later told the Bush Telegraph he was just "bored and being boisterous" when he flicked the fire. He added: "I seemed to really bug [Conley]. I really got under his skin."
Meanwhile, Conley branded Haye "out of order" in a conversation with fellow camper Eric Bristow. The darts player agreed, labelling Haye as "f**king stupid".
When Conley attempted to tackle the subject with Haye, the 31-year-old South Londoner joked: "You don't think that would be funny? A bush fire might spice things up."
The run-in led to Conley sitting alone on a rock crying, branding the boxer a "b**tard".
Former Doctor Who star Colin Baker later told the Bush Telegraph: "Brian tried to tackle him about it. David being David was unrepentant. He is very beguiling and funny but he is a force of nature and the end result was that Brian went off upset.
"Brian went off on his own and I followed him and there's not many talents I have in here but calming down upset people I hope is one of them. And I told him I'd be upset too and he'd be alright."
> 'I'm a Celebrity' Helen Flanagan 'demanded cameras be turned off'
> 'I'm a Celeb': Limahl, Rosemary Shrager spend night in hut with insects
> 'I'm a Celeb': David Haye, Helen Flanagan wash in the bathing pool
After returning to camp, Conley apologised for getting upset after having "had a go" at Haye, while Haye admitted he thought it had been a joke.
Conley then quipped about himself and Made in Chelsea star Hugo Taylor: "At least we cry. We're men. Real men cry."
Photo gallery - I'm a Celebrity's top buff bodies:
Copyright: Rex Features ITV/Rex Features