The show was likely written to entertain - and to that end it's a relative success. You can cherry pick and throw in confirmation bias all you like to support whatever world view you hold. For example, the show could be condemned for cultural appropriation - those are regency upper class dresses that solely belonged to white europeans - yet Caroline of Brunswick was being played by a woman of colour. If I can find someone, anyone who is offended by that - shall we cancel the show on those grounds? It's a legitimate question.
In short, throwing buckets of carefully crafed ideology over works of fiction is disingenuous and pointless. It wasn't a treatsie on race relations from 1811 - it was a work of fiction, developed by one of the best black writers on the planet. It was a racially diverse cast and an interesting story that feels new and fresh in an otherwise staid and uninteresting genre. Going over and over it with identity politics means you can poke holes in it - but so what? If you reverse all the roles, then you'd have a black duchess raping a white duke.... and you'd have racism again. You'll always have racism if that's what you're looking for, it's how confirmation bias works.
Progress is being made. Slowly. This is a definite progression from all white productions of Jane Austen classics. Decide whether you want to throw yourself behind it, or you want to be part of a group who ensure that something like this never gets made again. Sure, it wasn't perfect, and bits of it were problematic but If you want a perfect TV show, write one - just don't be surprised when other people pull it apart for being racist.