Assessment: It’s a Curriculum Not a Data Issue

It has taken time but I sense some of the fog is clearing in the post-levels World.  The title is a simple statement I make when speaking at various conferences about assessment; you could almost see the penny drop with attendees.  Far too many schools replaced levels with something that looked like levels and was equally as flawed and meaningless. 

Assessment is best designed backwards; that is, think first about what conclusions you would like to come to?  How might you best approach gaining the information/data required to do this?  Whilst the tide of assessment and data delusion is turning there is still a long way to go.  In some schools senior and middle leaders are still clinging to six weekly data drops of meaningless information to help them make invalid and inappropriate conclusions.  Here are a few questions that you might want to answer as a leader or a teacher.

What do pupils actually know now that I have taught them during this topic/scheme?

There are lots of issues here around the ability to immediately recall or work through a particular process versus the longer term retention of the same.  There will be a minute by minute check needed; this is purely for the teacher and pupils’ benefit; isn’t worth recording and will need a means of quickly assessing all pupils e.g. use of white boards as opposed to a hands up approach where one pupil gives an answer.

This agile assessment needs teachers to be responsive in class, in real time, to adapt their teaching in response to the feedback gained from pupils.  I’ve seen great practice from Early Years to Sixth Form.  The context is very different but the core process is identical; intervention in the classroom, close to the point of first teaching.  Don’t allow gaps in learning to appear and grow.

Using the concept of spaced practice, as a guide; there needs to be some larger assessments conducted further away from the point of first teaching.  I tend to use the phrase “common and cumulative” to describe what is required.  The cumulative aspect of the assessment requires pupils to be assessed on content that was taught earlier in the year; it’s a month by month check.  Pupils are asked to recall, through an age appropriate assessment, content they have been taught during the year or key stage.  This tends to need a more formal recording, analysis and planned response.  The analysis needs to be done at a grain size (think a single useful piece of assessment data) that is age and subject appropriate.  Too many school level assessment policies pay scant regard to these differences is search for an unnecessary and ill-informed conformity.

Interestingly, this data also helps answer another question, “What do I teach well and where do I need support to become an even better teacher?”

Key Points to Bear in Mind:

• Avoid aggregating data (turning it into some sort of grade); when you do this you lose all the finer level information teachers and pupils need to improve
• Avoid insisting that all subjects and phases/ages have an identical; the process is identical but subjects and phases require flexibility within this
• Avoid excessive recording and monitoring; this kills the goose (read teacher here) who lays the golden egg (read day in day out great learning in the classroom)

Is our curriculum sufficiently challenging?  Are our pupils making sufficient progress (relative rather than ipsatic)?

Too many schools still try to use the minute by minute or month by month assessment approach to answer this second set of questions.  This soon destroys the potential benefit of the data collected for the first set of questions whilst also not producing valid data for to answer the second set.  If you want to know about the challenge of the curriculum being taught or the progress pupils are making; you are looking at a year to year data collection process; twice a year I can live with but six times a year is madness.

We’ve started using three different data sets to help us find out about the progress pupils are making between the national data sets produced at the end of EYFS, Key Stages 1 & 2 and GCSE/A-level.  The long time span of Key Stage 2 and then up to GCSE without some form of progress monitoring is still too scary for me.  Different data sets play a part at different times of the year/school phase.

Whilst the use of national referenced assessments isn’t without problem – the environment they are sat in matters significantly (Professor Becky Allen is brilliant on this); like all methods of assessment they are not perfect but can be useful.

Credit: Professor Becky Allen’s presentation at ResearchEd Durrington captured by Oliver Caviglioli

You may be familiar with the suite of assessments from CEM in Durham, GL Assessment or Fisher Family Trust.  Whilst they are not cheap they provide useful information that will help you answer the critical questions posed above with far more reliability then trying to guesstimate GCSE grades in Key Stage 3 using teacher assessment; akin to trying to solve algebra by chewing gum.  This nationally referenced assessment data will help  you answer whether you are delivering an inadequate curriculum rather well or a challenging curriculum particularly badly or a challenging curriculum rather well; internally generated data on its own won’t.

More recently a number of examination boards have started to produce sample papers.  Students sit the papers and teachers mark them with the marks for each pupil fed back into a database; once the sample size is statistically large enough then the school is sent a report telling it how many of its pupils are in the top 3%, 20%, 50%, 80% etc.  Whilst not giving a GCSE grade it uses the same methodology that will be used when the final examination papers are set.

Finally we occasionally have mocks; pupils sit a full set of examination or test papers from one examination series.  The papers are marked using the examination board mark schemes and grades given using the relevant grade boundaries; we term it a current grade and then tell students that the final grade they get will depend on how hard they work.  If you want a bit of a workload buster this year we paid a couple of examination markers to mark sets of mock papers; the markers appreciated the money and the teachers appreciated the time to spend on other things.  Each marker gave feedback on strengths and areas that needed developing once they had marked the papers.

Teacher assessment sits at the pivotal point between teaching and learning; that should be its primary focus.  Tracking progress and the level of challenge of the curriculum requires a comparison with all pupils nationally or a sufficiently large sample.  Confuse the two at your peril; assessment needs you to use the most appropriate data to answer the question you are asking.