There’s a problem with inconsistency in the A-level grading system

17 Jan 2019

About a quarter of all grades awarded at GCSE, AS and A-level are inconsistent, research has revealed

In a blog post on think tank Higher Education Policy Institute’s (HEPI) website, Dennis Sherwood explains the findings of the comprehensive study by Ofqual published last November in the report, Marking consistency metrics – An update.

Ofqual refers to the Office of Qualifications and Examinations Regulation, a non-ministerial government department that regulates qualifications, exams and tests in England.

In its study carried out over several years, exam submissions from a large number of schools were blind double-marked. This means they were marked twice: once by an examiner from the general pool and another time by a senior examiner for a mark designated as “definite” (or what we might call “right”). These marks are then assigned their corresponding grades and compared.

1 school exam grade in 4 is wrong. Does this matter? – https://t.co/VEmprNbtJV pic.twitter.com/5PJHovE5X7

— HEPI (@HEPI_news) January 14, 2019

As explained by Dennis Sherwood in HEPI’s blog, this is in essence what the study found: “…on average, across all subjects at both A-level and GCSE, about one grade in every four, as awarded every year, is wrong.”

Not every subject is created equal, the study found. Grades for mathematics have the highest probability (96 percent) for having the school exam grades corresponding equally with the “definitive”/right grade.

Other subjects, however, have much lower chances of having grades marked right. The reliability of economics grades being marked right is 74 percent, whereas for history, it’s about 56 percent.

This means 26 percent of grades in economics are likely to be marked wrongly. In history, that figure is significantly higher, at 44 percent. Overall across the 13 popular subjects analysed (French, Spanish and art were excluded), the average comes to about 75 percent right, 25 percent wrong.

Sherwood, who runs the Silver Bullet Machine consultancy, notes some of the important consequences of this:

“Every candidate sitting four A levels is likely to be awarded one wrong grade.
Every candidate sitting eight GCSEs is likely to be awarded two wrong grades.”

In the context of the hundreds of thousands of grades awarded at A-level and GCSE last August, it is estimated that about 200,000 A-level and more than 1.3 million GCSE grades were likely to be incorrect.

With university admissions so heavily reliant on grades, many applicants could have found themselves rejected from their top choices for not making the grade and worse still, having a borderline mark that ended up a grade lower.

“The school exam grading system is broken, no longer fit-for-purpose, and must be changed. Grades as currently used are not the only way to recognise a candidate’s achievement: there are several other totally practical possibilities, one being no longer to award grades, but for certificates to show the candidate’s standardised UMS (Uniform Mark Scale) mark, associated – importantly – with a measure of the subject’s ‘fuzziness’,” Sherwood wrote.

“Fuzziness” refers to the acknowledgment that all marks are fuzzy (eg. it’s hard to distinguish between a 54 and 55 even for the best of teachers) yet one mark can make all the difference between making a grade or not.

According to The Telegraph, Mike Buchanan, Executive Director of the Headmasters’ and Headmistresses’ Conference, describes the findings as “extremely worrying”.

Hundreds of thousands of young people are affected every year, Buchanan said, pointing out that girls, who dominate humanities subjects, are particularly disadvantaged, as this is where unreliability is the most extreme.

“The implications are grave, as a questionable grade can have a significant effect on a pupil’s life chances in a high stakes exam-focused environment.

“Hundreds of thousands of children are being forced to retake English and Maths GCSE or may be missing out on university places or jobs because they did not get the grade.”

The difference of one or two marks can make or break student futures. Source: Shutterstock

In addition to Sherwood’s suggestion to use certificates that explain the range of a university applicant’s exam marks, former Chief Examiner Neil Sheldon advocated for “crude” grades to be removed.

The ex-Vice President of the Royal Statistical Society said: “I don’t think any candidate wants to be told you have got a B but it might be a C or an A. I don’t think any university would be happy admitting anybody on a similar basis.

“Crude” grades should be scrapped and candidates marks should be given alongside a margin of uncertainty instead, Sheldon explained.