Measuring Pupil Progress in an Uncertain World

Over the past twelve months, I’ve never sat in as many meetings where I’ve said “we just don’t know”.  This week was no exception; we were trying to think what GCSE English & Maths data we could usefully collect at a whole school level during this academic year. 

Photo Credit: mederndepe via Flickr cc

There may not be much we agree on in education – though opposition to expanding grammar schools is looking fairly solid – but uncertainty around the standards expected for a score or grade may well unite us across all phases.  We are entering a retrospective world where scaled scores and the value added associated with them are determined after the event.  As such, many primary schools, leaders and teacher are still coming to terms with last summer’s results.  The shock in some quarters was palpable.

At a secondary level, a new grading system for GCSE English and Mathematics has left many teachers and school leaders uncertain and anxious around in-year standards and measuring progress in Year 11.  This uncertainty will move to a whole raft of E-Bacc subjects next year and other subjects the year after that.  No subject or teacher will be untouched.  The key whole school performance indicator, Progress 8, will also be calculated retrospectively until at least 2019.

Schools are going to take different approaches to grading GCSE English & Mathematics this year.  One of my favourites, at the moment, is the “Building on Sand Approach”.  This involves a dense statistical approach to determining grades and scores which overwhelms teachers, forcing them into submission.  Anyone who sees the dense spreadsheet believes it.  It just looks so good that you become transfixed and decide to fall into line.  The numbers may well add up but the underlying assumptions pushing the numbers along are actually made up.  It’s invalid but comes at a huge cost in terms of time and effort.

Photo Credit: drpavloff via Flickr cc

We gave up trying to predict future performance of pupils a couple of years ago and moved to a current grade system.  Scaled scores for Year 6, new 1-9 GCSE grades, new AS and A-levels with limited papers and no grade boundaries is persistently leaving holes in our attempts to collect meaningful in-year data.  In this week’s meeting,  we decided just to collect percentages attained on sample papers for Y11 students in GCSE English & Mathematics; we did the same last year in Year 2 and 6 except for writing.  We collected data twice in the year and looked for signs of progress through pupils attaining higher scores.  Not exactly rocket science or complex.

Don’t Fret; Sweat the Small Stuff

Our post levels approach I’ve explained in detail here – Data & Feedback Informed Teaching and Learning or as I like to say, “Find out what they don’t know and teach them it.”  The first step is making sure you have really strong schemes of learning, that is, ones that are challenging with the learning appropriately sequenced in a clear progression.  From this, pre-assessments (find out what they already know) are developed; then you can plan your teaching and use a series of assessments to check what pupil have and haven’t learnt compared to what you have taught.

It is the learning gaps, the bits children don’t know, that I refer to as the small stuff, in the above heading.  This isn’t small in terms of importance, anything but.  It’s small in terms of grain size compared to the aggregated data – scores, grades, levels – beloved of data managers and leaders.  It’s in the former we find certainty and a potential way forward; re-teaching the parts of the learning progression pupils didn’t grasp first time around.

One thing missing from our approach is an external perspective.  If, despite your best efforts and intentions, your scheme of learning is pretty mediocre you may end up effectively delivering a right load of tosh.  I’m musing about whether we need to look at introducing a few nationally referenced assessments into the middle of Key Stage 2 and somewhere in Year 8 or 9.  I won’t rush to a decision but if pupils aren’t making progress I want to know sooner rather than later.  Waiting until the end of Key Stage 2 or 4 seems rather negligent.

Let’s Be Honest, We Don’t Know

Uncertainty can be seen as a weakness; a ditherer’s disease that shouldn’t affect teachers or leaders.  In the liminal assessment and standards world we find ourselves in, uncertainty is the only certainty.  In moments like this, sometimes all you are left with is trying to find serenity in the madness.

A bit of an aside, it’s time for performance management target setting in many schools, targets that will be monitored and reviewed during the year.  I wish you well with that.

Postscript on external perspective:

David Didau suggested looking at Proof of Progress