He will miss two weeks' filming after arriving two hours late on the most recent occasion. Producers of the E4 show have also issued a formal warning to the rest of the cast over their "extremely unprofessional" behaviour.
The diamond-mining heir also failed to turn up for a photoshoot to promote the upcoming third series, which premieres later this month, The Mail On Sunday reports.
Producers have apparently tired of the Made In Chelsea stars arriving on set drunk or hungover and sleeping in between takes.
Quality Street heiress Millie Mackintosh was suspended from the show last summer after going on holiday while filming was still in progress.
Production manager Heidi Birkett emailed the Made In Chelsea cast last week, saying: "It has been brought to my attention that some of you have been turning up late as well as not being in a fit and professional state for filming. This is totally unacceptable and extremely unprofessional.
"As we discussed with all of you before Christmas this level of behaviour will not be tolerated and would result in action being taken. We and the Channel are taking this very seriously and unfortunately I'm sad to say that this has now become reality with one member of the cast.
"As many of you now have outside interests as a result of Made In Chelsea I would urge you to think how this could have a knock-on effect to such things as personal appearances, magazines, other TV projects, branding etc. I would also ask all of you to think whether you would act in such a way in any other job without expecting similar repercussions."
Boulle said that he has agreed to take time off from Made In Chelsea "by mutual consent" after his "business life" had impacted on his involvement in the programme.
Channel 4 confirmed the news, with Made In Chelsea executive producer Sarah Dillistone adding that unprofessional behaviour "will not be tolerated".