Grade (un)reliability - the full story

Over the last few weeks, I've been working on a document that tells the full story about grade (un)reliability, including

* why grade reliability matters

* the problematic nature of the quest for the 'right' mark

* the distinction between 'accuracy' and 'reliability'...

* ... and how grades can be unreliable even when marking is technically perfect

* the 'lottery-of-the-first-mark'

* the inherent 'fuzziness' of all marking

* how to measure fuzziness

* the evidence about how (un)reliable grades actually are

* some simulations of 2018 A level Biology and 2018 GCSE English Language

* a description of 21 possible ways to deliver reliable grades (some of which work better than others...)

* ...and an analysis of why double-marking doesn't work at all

* some implications of the various possibilities

* a full analysis of the statistics of marking and re-marking.

Much of the content has appeared elsewhere, in bits; this document brings (I think) all the main things together into a single place.

The document is long (!!!), but I trust well-structured; please do be in contact of you have any thoughts, questions or suggestions, and especially if you disagree with anything - and indeed if you find any mistakes (which are my fault, and for which I apologise in advance!).

To read or download the document, please click on "School exam grades are unreliable" below...

School exam grades are unreliable, web version, revised 10 march 2023.pdf

Download PDF • 2.35MB