Why Accountability in Our Schools is a Total Shambles #HTRT2016

The accountability system for primary schools is so broken it could take a decade to fix.  In secondary schools the fix could be quicker; the problem is we are currently measuring the wrong thing.

Having the opportunity to engage with Dr Becky Allen & Dave Thomson has been not only beneficial but a great professional development opportunity.  With a conference call booked for later this month with Professor Rob Coe’s Durham University, CEM Team, to discuss baseline assessments, I think my professional development may take yet another big step forward.

Over the summer I’ll be working, alongside other Headteachers’ Roundtable colleagues, on a set of policy proposals for the accountability strand.  For me it’s the key one.  If we could get accountability right it would unlock the potential for deep cultural change within our schools; recruitment and retention up, toxic cultures out, work life balance improved and a positive spiral initiated.

There seems to be three questions we need to answer (I may have missed one or two):

1. How effective are individual schools?  Are they providing a good education for their pupils?
2. How good is the education system as a whole in England?  Is it improving over time or getting worse?
3. Are there any groups of pupils (by gender, ethnicity or disadvantage) who are performing less well?  Are policy initiatives or efforts closing the gap and increasing educational equity?

My brain can only cope with one of these at a time; a school’s effectiveness needs to be based on a multi-year, contextualised value added measure.  All measures have a degree of variance from one year to the next; measuring over time increases the reliability of the measure and as such the validity of the conclusions we can draw from the data.

We need a contextualised value added measure to make sure no school is advantaged or disadvantaged, in accountability terms, due to its intake.  The use of attainment measures (5+A*-CEM, Attainment 8, Expected Standard in RWM) tells you more about the ability of a school’s intake than how good the quality of the education is within the school.

Primary Accountability is a Total Fuster Cluck

The biggest shambles we are facing is assessment and accountability in the primary phase.  My own view is we have probably reached the point at which we should throw the whole lot out and start again.  The use of teacher assessment is hugely problematic and open to massive gaming.  An unscientific, statistically invalid poll on my own blog suggests that quite a few people thing the same.  It’s far too open to manipulation or gaming and fails to produce a measure at the granular size needed to enable a value added measure to be calculated.

The aborted attempt to use three different baseline tests; only for the Department of Education to belatedly admit (or possibly understand) that they could not be used as the basis for assessing a school’s effectiveness – the value it adds to pupil’s learning or the progression made by pupils – leaves us bereft of any baseline except possibly Key Stage 1.  The Key Stage 1 results, however, even if they weren’t based on teacher assessments are useless because instead of a point from which to measure the value added you have a “band of attainment”.  This year’s “expected standard” or “greater depth”, like the levels and sub-levels which went before, are insufficiently granular; pupils attaining a 2b have a range of attainment.

Statistically Correct but Politically Inconvenient

Due to the brilliance of Datalab (thanks again Dave & Becky) we’ve been able to pull together some statistics for the past three years to look at how many schools have consistently been below a given threshold.  The threshold used was the -0.5 that will be the new Progress 8 floor standard; they’ve also had a look at how many schools are above the +0.5 threshold by way of some balance.

Acknowledgement: Produced by Education Datalab

For a political system that likes to lambast schools they perceive are failing and laud those who they favour the rather unfortunate reality is it looks like 90% of schools sit within a very narrow range around the average.  If I had used the Attainment 8 score and identified those schools +0.5 or -0.5 for three consecutive years the numbers are almost negligible.What is very clear is that there are precious few outliers when you look at the data in this way.  Taking a three year average (as opposed to below the threshold for three consecutive years) and using the Best 8 GCSE point scores as these give the greatest number of outliers we get the following; from 2013 to 2015 just 5.7% of secondary schools are more than half a grade above expectation and 5.1% are more than half a grade below expectation.

Is now the time for a total revision of the accountability system?