Just what are we assessing?
To situate you a bit of biography:
I’m in my 40s. I have a BA and a MA and a teeny bit of another BA. I have taken a lot of tests and done a lot of other kinds of assessment over the years.
The MA is in Educational Technology. I have been working in the field for a bit over a decade now. I have written a fair number of tests and other kinds of assessment over the years.
I have a kid, he is 12. Short hand time: the assessment (there is that word again) we had done on him when he was 8 showed that he is “gifted” (a term I am not entirely in love with). It also showed he has processing speed issues and dysgraphia (TL;DR version: his handwriting is a mess and takes a long time).
So you combine those two things and a very bright kid suddenly looks like a dolt (mindful of its limitations, I think of this a lot “Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid.”).
Now his school gives him extra time on tests (which, having worked in Universities, I can tell you most of them do for students with issues as well), has learning advisers, allows him to type what makes sense to type, dictate stuff that is appropriate etc etc. So this is good.
But it does make one think: what are we assessing? I mean, the classic learning design is: learning objective (what we want them to know), Assessment (how we will prove they know it) and content/activities etc (the stuff we are going to teach them). Read that back, nowhere does it say anything about speed. Clearly in “real life” some things have to be done with a pace (I really don’t want a surgeon who hangs out and thinks for a bit as a bleed out), but realistically aren’t most time constraints we place on assessment more about fitting the test to a class period or (in a professional development setting) not interrupting the work day too much?
Anyway, the boy had a test today. The learning adviser helped him out by typing for him, he got extra time. The learning adviser just emailed us to tell us how proud of him she was and how hard he had worked and how much knowledge he managed to get out of his head.
So questions aside, today was a good school day.
Cameron's Pondernatings 
Assessment is often not what what we think it is, as you’ve pointed out. A comprehension or recall question can become a spelling or handwriting test instead. I suspect a lot of “no good at maths” is really “couldn’t understand the question”, too.