Grade (un)reliability - the full story
Over the last few weeks, I've been working on a document that tells the full story about grade (un)reliability, including
* why grade reliability matters
* the problematic nature of the quest for the 'right' mark
* the distinction between 'accuracy' and 'reliability'...
* ... and how grades can be unreliable even when marking is technically perfect
* the 'lottery-of-the-first-mark'
* the inherent 'fuzziness' of all marking
* how to measure fuzziness
* the evidence about how (un)reliable grades actually are
* some simulations of 2018 A level Biology and 2018 GCSE English Language
* a description of 21 possible ways to deliver reliable grades (some of which work better than others...)
* ...and an analysis of why double-marking doesn't work at all
* some implications of the various possibilities
* a full analysis of the statistics of marking and re-marking.
Much of the content has appeared elsewhere, in bits; this document brings (I think) all the main things together into a single place.
The document is long (!!!), but I trust well-structured; please do be in contact of you have any thoughts, questions or suggestions, and especially if you disagree with anything - and indeed if you find any mistakes (which are my fault, and for which I apologise in advance!).
To read or download the document, please click on "School exam grades are unreliable" below...
