//
you're reading...
Assessment

DfE/OFQUAL’s Exam Replacement Problem

Within the Department for Education/Ofqual’s consultation on this year’s exam replacement there are a whole series of distractions and much confused, woolly thinking.  The key challenge that requires addressing – and was missed last year in the need to move rapidly to a centre assessed grade – is the comparability between grades awarded in different schools/ centres.  This is the basis for having a national examination system and is totally missed in the consultation in favour of some ill-defined “external quality assurance by the exam boards” in June. 

When we talk about examinations being fair; what we mean is there is comparability in standards between the grades awarded to students in different centres rather than fair to individual students per se.  Students will have had sixteen years of vastly different life experiences before they walk into the examination hall.  Examinations essentially ignore the different opportunities, or lack of them, children and young people have been afforded over time.

In looking to create a far from perfect system of comparability in grades between schools, I would suggest that all schools (not individual pupils) are given an expected grade range, for each subject, based on their current Year 11.  This must happen prior to the awarding of any centre assessed grades.  The grade range would be based on two separate calculations.  What is particularly interesting is the assumptions within and hence potential implications of each calculation.  In different schools either of the two calculations might form the upper or lower limit of the range.

Calculation 1 – A straight forward calculation per subject based on the prior attainment of the current Year 11 group, using their Key Stage 2 results.  This will lead to groups that usually underperform at GCSE and schools containing a large cohort of disadvantaged pupils, particularly those who are long term disadvantaged, being treated more favourably than the current examination system.  In essence, moving all schools/departments towards a Progress 8 score of zero.

Calculation 2 – A statistical calculation using a three year subject based progress score from years 2017-2019.  This locks into the system schools’/departments’ prior performance.  Whilst being rather dated (2017-2019 seems a lifetime ago and much has changed) this does allow previously higher performing departments/ schools, often with more advantaged intakes, not to be overly penalised by this year’s system.

The two calculations would produce a grade range. This would need to be sent to schools by the 9th April 2021.  The timescale is tight. However, like many others I’ve been calling for a Plan B for months and this is the consequence of the inactivity and lack of planning. 

The information sent to schools would provide a grade range for each subject for: grades 4-9; grades 9-8; 7-6 and 5-4.  Trying to set a range for each grade is likely to be too unreliable as statistical models are less useful as sample sizes fall.  Where school cohort sizes are small the reliability of these proposals would be reduced.  This would have a significant impact if a similar model was used at A-level.  As an aside, I would suggest that students’ examination certificates should carry the GCSE grade range awarded; that is, grade 8-9; grade 6-7; grade 4-5 or grade 3.

Schools would have until the 30th April 2021 to either accept the grade ranges for a subject or appeal them directly to Ofqual.  Any appeal would need to be submitted alongside objective data that has national comparability and excludes results from Summer 2020. Appeals would be heard during May 2021 and determinations made.  Parallel to this process, schools would award grades (either 8-9; 6-7; 4-5 or 3) to individual students/ rank students, using the internal data available to them. Awarding a range reduces some of the cliff-edged decisions and potential inaccuracies. I’ve argued elsewhere that the need for ten grades at GCSE is overkill.  Grades awarded must be within the overall range agreed by the school and Ofqual.  There is no further moderation process required.  Post release of grades, pupils may appeal their grades to the examination boards/ awarding body. Performance tables and all other related documents would need to be shelved for another year.

My suggestion above attempts to give teachers and schools a guide to the number of grades that should be awarded within each range.  It would be up to teachers and schools to determine which students were given which grade.  I hope that there are far better ideas than mine but remember it is the comparability between grades, awarded by different schools, that the Department for Education/Ofqual should be seeking to address and have utterly dodged in the consultation.

Proposing teachers give students a GCSE grade based on school-based evidence is nonsense.  It suggests that grades are criteria referenced as opposed to a construct based on the comparability of a student’s performance, relative to their peers, on a particular set of examination papers sat over a number of days.  A moderation process after teachers have awarded grades is going to lead to merry hell breaking out and a blame and counter-blame game played out. There will be the almost inevitable U-turns that renders the previous work done as useless and an absolute waste of time. 

Proposals to produce “mini-exams” with centrally set papers that do/don’t have to be sat and can be selected from as a school sees best is a well-meaning but hopeless suggestion.  By all means provide national exemplar questions and papers for teachers/schools to use as they see fit.

Thanks to @sdupp

Appeals during the summer holiday period, after two exhausting academic years and in contravention of teachers’ terms and conditions of service will be given short shrift by unions.  I’m not convinced teachers/schools remarking their own work makes any sense either.  An external independent view is required.

And since the vast majority of the work is being done in schools, I’d expect a massive reduction or reimbursement of examination fees. As with all of these consultations you sometimes have to ignore the biased questions and say what you really think.

Discussion

7 thoughts on “DfE/OFQUAL’s Exam Replacement Problem

  1. This is a bold suggestion, and there’s much to agree with here, especially the slimming down of grades awarded and more rigorous steps for comparability between centres — which *could* have been done better in 2020.

    I’m not completely sure I agree with the statement “When we talk about examinations being fair; what we mean is there is comparability in standards between the grades awarded to students in different centres rather than fair to individual students per se.” Rather, it seems to me that the *reason* we seek comparability in standards between centres is to be fair to individual students. Though ‘fair’ is a difficult concept to pin down, and I agree with the broader point that for the key purpose of measuring readiness for further study in a subject, it’s attainment that matters, not some form of relativised attainment.

    When you say ‘all schools (not individual pupils) are given an expected grade range, for each subject, based on their current Year 11’, I presume you mean a distribution across the ranges you suggest (8-9; 6-7; 4-5; 3)? I’m not sure I completely see how the school appeal to Ofqual on grade ranges you’re suggesting would work. You say “Any appeal would need to be submitted alongside objective data that has national comparability and excludes results from Summer 2020” — this seems to suggest appeals would be on the basis of past attainment at the centre rather than current Y11 attainment, which seems problematic from the perspective of individual fairness.

    “Proposing teachers give students a GCSE grade based on school-based evidence is nonsense. It suggests that grades are criteria referenced” — this is a key issue, and goes back to the question of what these qualifications are for. If it’s for the purpose of measuring readiness for further study in a subject, then *shouldn’t* they be criteria referenced? Time for a big rethink!

    My thoughts on the 2021 consultation are here: https://constantinides.net/2021/01/16/gcses-and-a-levels-in-2021/

    My analysis of the 2020 algorithm failures is here: https://constantinides.net/2020/08/15/a-levels-and-gcses-in-2020/

    My response to the 2020 consultation, also suggesting grade ranges (but done in a different way) is here: https://constantinides.net/2020/04/16/award-of-gcses-and-a-levels-in-2020/

    Thanks for the thoughtful post & I hope this time we will be listened to.

    Posted by George Constantinides | January 17, 2021, 8:05 am
  2. Super stuff, thank you. And for some further evidence of why “ten grades at GCSE is overkill” (and, to my mind, seven for A level likewise), and building on the (very gentle) point you make in your previous blog that “the reliability of marking in some subjects is relatively poor”, might I mention this HEPI blog https://www.hepi.ac.uk/2021/01/16/how-to-be-innovative-in-school-exam-assessment-fewer-grades/. And alternative 2 in this blog might be of interest too – https://www.hepi.ac.uk/2020/07/23/hindsight-is-a-wonderful-thing/.

    Posted by Dennis Sherwood | January 18, 2021, 12:28 pm
  3. Further on your point regarding criterion-referencing, I note these grade descriptors still seem to be active for Grade 8/5/2: https://www.gov.uk/government/publications/grade-descriptors-for-gcses-graded-9-to-1.

    Posted by George Constantinides | January 19, 2021, 4:15 pm
    • That is a very interesting discovery, George. I’d not seen these grade descriptors. Are they too general, I wonder, or could they form the basis of centre assessments for <2, 2-4, 5-7, 8-9 type outcomes

      Posted by kal hodgson | January 20, 2021, 12:22 pm

Trackbacks/Pingbacks

  1. Pingback: Avoiding Examination Chaos the Summer | @LeadingLearner - January 23, 2021

Leave a Reply to kal hodgson Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Educating with Purpose – The Heart of What Matters

Liminal Leadership

FOLLOW ME ON TWITTER

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 32,075 other followers

Follow @LeadingLearner on WordPress.com

Blog Stats

  • 1,469,351 hits

COPYRIGHT LICENCE

%d bloggers like this: