An F-Grade for the Government? Consultation and the A-Level Fiasco
Today is results day for A-level students in England. It’s usually a day for endless media pictures of delighted students leaping joyously into the air, clutching the pieces of paper that represent the pathway to their bright new future. This year however, it seems the more apposite picture may be the one from the bottom of the same news articles with a student looking forlornly down at their own grades. Unusually though, the cause for this breach in procedure is not the failings of students, but the failings of those in charge of assessments.
Because of coronavirus and school closures the 2020 exam series has been largely cancelled and grades awarded on the basis of teacher assessments. These assessments have then being standardised using a statistical model. The end results, it’s fair to say, have not met with universal approval. Almost 40% of teacher assessment grades have been downgraded, with the axe falling most heavily on the most disadvantaged pupils in the country. Conversely, those in independent schools have seen the biggest rise in grades.
The procedure has drawn criticism from across the spectrum and has left big questions hanging in the air about how fair the model is, and how many students have had their dreams torpedoed in the midst of a year that already has its fair share of challenges.
The Education Secretary, Gavin Williamson has leapt into action to encourage those students who believe they’ve been hard done by to appeal their grades, which ignores the fact that by that point many of them will already have missed out on their university offers. He has insisted that he will not be forced into a similar u-turn as his Scottish counterpart was last week, although did make an eleventh-hour concession last night that mock exam results (which not all students will have and aren’t standardised) will be allowed as a ground of appeal.
This morning, he insisted to Sky News that the system had been consulted widely upon, and that the consultation returned responses in favour of the proposals. The consultation he refers to, “Exceptional arrangements for exam grading and assessment in 2020”, ran for two weeks in April, and garnered over twelve and a half thousand responses. It resulted in nearly all of the Government’s proposals being adopted, with only minor variance from their original forms, even where there were finely balanced numbers of responses.
Interestingly, the statistical model itself did not form part of the consultation, although various aspects of it were consulted upon. This raises a very interesting point for consultors when using statistical modelling. If the statistical model is consciously or unconsciously biased, then the results will be skewed. In this case, one of the end results of this seems to have been that those attending private schools, where sixth form sizes are fairly small, have been given a large grade boost when compared to sixth form colleges which will have larger class sizes. This disparity is further enhanced when, as in this case, the Government has decided not to allow for appeals with the operation or outcome of the standardisation model as a ground. Consultation is not a numbers game, but it is perhaps worth noting that in the matter of appeals, 46% of respondents disagreed or strongly disagreed with the Governments proposals.
There are potential equalities issues here. The reinforcement of systematic disparities through the use of a standardised model that it would seem not only preserve inequalities, but render them more pronounced is surely distasteful. Though it may not be direct bias, the grade inflation at independent schools, and the seeming deflation for those schools where disadvantaged and minority students are prevalent seems like an expression of inequality of the highest order, and certainly counter to the spirit of the Equality Act.
Whilst no statistical model should be used to artificially inflate grades beyond what students deserved it is difficult not to wonder if there might not have been a better way of modelling this that would ensure fair consideration across the board. For us the bigger question is whether this failing which, in retrospect feels fairly obvious, might not have emerged, had the standardisation model itself been subject to a more open and rigorous consultation process.
Although the anger over this is unlikely to survive more than a week in the popular press, you can guarantee that in the minds of those students who have suffered disappointment and loss because of what has been widely criticised as a deeply unfair system, the anger will not dissipate so quickly. We certainly wouldn’t be surprised to see this arising as an issue before the High Court, most likely on equalities grounds.
For consultors there are perhaps a few salient reminders that we can take from this chaos. One is to ensure that you are open about what you are doing, be transparent with your models and how you plan to use them. Secondly, be mindful of equalities, even if there is not an immediately apparent impact. Thirdly, be clear. Particularly when dealing with something as important as this, people want clarity. Emotions will be running high, so you must have clear messaging to cut through the confusion.
For now, we wait to see if Mr Williamson will follow the path of the Scottish Education Secretary and make a u-turn on grading. Though he maintains that there will be no volte-face, with criticism coming in from all sides it may not be long until this position becomes unsustainable. For students and parents disappointed with their results, the confusion is sadly not over quite yet.