In January 2018, the Department for Education released their latest update on Secondary School Accountability measures. It seems a number of people missed the significant change to Progress 8 that a number of groups, including Headteachers’ Roundtable, had been calling for.
Whilst all schools have their Progress 8 score affected by negative outliers; those with particularly small cohorts are disproportionately affected. The text in italics below is taken straight from the January 2018 publication.
Pupils with extremely negative progress scores – change from 2018
“The department has received some feedback about the disproportionate effect that a small number of extremely negative scores can have on a school’s average. Such extreme cases tend to occur where a pupil was a middle or high achiever at key stage 2 and goes on to achieve much worse at key stage 4, for reasons beyond the control of the school (e.g. long term illness), than the national average for others with similar prior attainment. This has created concerns that in a small number of cases some progress scores calculated for individual pupils can be so largely negative that they can distort the overall picture of performance for a school.
The department has listened to this feedback and is refining the methodology for 2018 in order to reduce the disproportionate impact of the most extreme pupil level progress scores only. The refinement we are making introduces a limit on how negative a pupil’s progress score can be when calculating the school average. These pupil progress scores will still be large negative scores (to reflect that the pupils have made much less progress than other pupils in the same prior attainment group as them), but the disproportionate effect these have on a school’s score will be reduced.
We are not setting a maximum limit on how positive a pupil’s progress score can be as there are much smaller numbers of extremely positive progress scores that have a disproportionate impact than extremely negative ones progress score can be when calculating the school average.
The limit will mean that there is a minimum progress score that is assigned to pupils within the prior attainment groups (PAGs) where extremely negative scores exist. Where a pupil’s score is more negative than the minimum score, the minimum score will replace the pupil’s original progress score when calculating a school’s Progress 8 average. The minimum score for each PAG will be determined based on the variation in pupil progress scores for pupils across the country within that PAG (as measured by the standard deviation). The minimum scores will be fixed at a set number of standard deviations below the mean so that approximately 1% of pupils are identified nationally (we anticipate this will normally be no more than 1 or 2 pupils in any school). As such, predicting which pupils will, and will not, have their score affected by this methodology change, in advance of Progress 8 scores being made available, will not be possible. The exact minimum progress scores will be confirmed in the autumn, once we have the 2018 progress data.
Rather confusingly the School Performance Tables will display two Progress 8 figures; one with the negative outliers removed and one with the negative outliers in. I’m not convinced this is necessary, helpful or useful in terms of transparency. It will probably confuse more people that it helps. Despite this the limits placed on negative outliers is to be welcomed; not often I do this but *tips his hat to the DfE*. We appreciate you listening and making the necessary changes.
Now we need to maintain the dialogue to ensure off rolled pupils are proportionately placed back onto a school’s roll and a three year contextualised rolling average is agreed as the headline school effectiveness measure.
Once again top performing schools are still advantaged as there is no limit on the outliers at the top end. So the extremely high performing pupils will nicely keep the top performing school’s Progress 8 out of reach of any ordinary comprehensive. It’s illogical to cap the negative outliers but not the positive; unless of course there is other agendas at play as usual.
So let the gaming begin. Switch your ‘less motivated students’ to IGCSE’s and widen their curriculum to take in subjects that are not counted in the performance tables. These students then become ‘Outliers’ and not counted in the P8 value for the school.
Doesn’t quite work that way; the outlier isn’t removed rather their negative score is reduced slightly. It will still impact on a school.