Argent made the gaffe during an interview with actor Al Pillay on The Late Show with Joanne Good last week.
Cabaret performer Pillay, 52, has an Irish-Jamaican mother and his father is of South African, Indian and Spanish descent.
"Have you ever been to Essex before?" Argent asked while co-hosting the BBC London programme. "I think you'd fit right in. You've got this whole spray tan thing going on. You're browner than me."
Even after Pillay informed Argent that he was "born brown", the reality star continued: "The girls would love you with that tan."
Pillay later urged his Twitter followers to listen again to the show and help him clear up Argent's motive for the remarks.
He asked: "Was it a subtle racist remark when Arg commented on my spray tan or was he just being thick not realising I was born with brown skin?"
However, Pillay returned to the site after receiving a bouquet of flowers from Argent and clarified that he did not believe the TOWIE star had meant to cause offence.
It was reported earlier this month that Argent has decided to quit The Only Way Is Essex.
> 'Only Way Is Essex' Lydia, Arg to take relationship to 'next level'