Conor took home 50% of the show's prize money in the task, bluffing his rival Luke Scrase.
Digital Spy readers described the twist as "disgraceful" following Conor's behaviour with Deana Uppal earlier in the series, when he claimed that he would happily "punch" her and made sexually aggressive comments about the Miss India UK winner.
Fans also comnplained about the selection of Conor to take part in the task and the fact that he had won cash without any public vote of support for him.
However, host Dowling claimed that the White Room twist had been one of his series highlights.
"Conor winning the 50 grand, people say, 'Oh no, he shouldn't win', but he won it fair and square," Dowling told Telly Mix.
The former Big Brother champion did admit that he wouldn't have taken the prize money personally, claiming that he "wouldn't have that".
Big Brother ends on Monday, August 13 at 9pm on Channel 5.