I wouldn't call 15% of the population a small amount of people. Autism rates are around 1 in 100 as you've said and the incidence of ADHD has been on the up, but in Canada (according to this analysis https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9406225/ - the figure levels out around 3% of the population. So let us presume you've got 100 Canadians.. I can say with some certainty that 1 of them has a diagnosis of Autism. 3 of them have a likely diagnosis of ADHD. We still have 11 people to account for. In a country Canada's size, population 38.25 million - those 4% diagnosed people are around 1.53 million individuals.
I'll give you the 45,000 Downs Syndrome Canadians and the 1 in 10,000 births that are Prader-Willi syndrome (another 20 or 30 individuals per year). You're suggesting there are a further 3.8 million people or so with 'neurodivergence'? These people have what? That's a large cities worth of people who have clinically identified themselves as what precisely?
My point is that if 1 in 100 people have a diagnosis of autism, but then 12 other people decide they also have it.... and those 12 people decide 'autism isn't really that hard' then, in effect, it makes it much harder for the 1 in 100 individual. Likewise, if you suddenly lump in dyslexia, dyscalculia and dyspraxia (and these are the likely 3.8 million people) into the same bucket as Down Syndrome or Prada-Willi, you're drawing attention away from where society needs to place it.