Redesigning Classrooms: Is Your Summative Assessment Trustworthy?

Are we about to enter a national assessment black hole?  We may soon be facing an assessment system with a terminal exam at the end of Year 11 and Year 13.  A five year gap, from the beginning of secondary schools to GCSE, without any nationally recognised or externally accredited examinations to affirm for us how our students are performing. 

In contrast, when I started at St. Mary’s we had end of Key Stage 3 SATs, the option to buy in SATs papers for the end of Years 7 & 8, module assessments throughout Key Stage 4, coursework at GCSE and the new modular A-level with coursework modules available.  These have already or are about to disappear and to confound the matter further the safety blanket of levels may soon be ripped from around us as well.

We are going to need to get much better at assessment including the summative element by understanding key principles and developing some highly effective practice.  This blog post is built on the symposium led by Dylan Wiliam that I have previously written about titled, “Redesigning the School’s Curriculum: Find your Compass.”

Below is some actual summative data from three different students, in three different subjects produced in response to our assessment system.  It formed the starting point for a Thursday afternoon staff CPD session that started with the simple question, “So, what do you notice?”

Staff not only notices things but began to explain them.  First up were the drastic differences in between some consecutive assessment cycle grades – students B went from a 6a to a 5c, nearly a two year regression in about half a term is some going.  Also staff picked up the often significant drop as students moved from Year 7 to 8, look at both students A & B.  The tendency to use one test/assessment piece to produce the summative grade, the narrow set of skills being assessed and the lack of transfer of assessment data from Year 7 to 8, even though it is all available in the SIMS, were raised as issues.

It presents the fundamental question, is our summative assessment system trustworthy?  Do students and parents have faith in the outcomes?  Is it actually telling us which of our students are more able mathematicians, linguists, scientist, historians, geographers etc?

Principle one: Is the school’s system to be trusted (do stakeholders have faith in the outcomes)?

The challenge was set to look at current practice in relation to the set of principles that would guide the building of a reliable summative assessment system.  Are assessments:

Principle Two: Distributed (so that evidence collection in not undertaken entirely at the end)

Principle Three: Synoptic (so that learning has to accumulate)

Principle Four: Extensive (so that all important aspects are covered)

Interestingly, we may find some help from an unlikely source as it is often the Cinderella subject in the school’s curriculum if it is there at all – Drama.  Drama has operated in the Key Stage 3 Curriculum black hole for an eternity.  The following was written by Cathy Lloyd, Head of Performing Arts, and it’s interesting to reflect on what lessons are transferable and applicable.

“Currently there is no National Curriculum for Drama. Because of this, it was decided to create a syllabus based around the skills of an actor, namely technical ability, interpretation of a character and knowledge (of both the theatrical setting and Drama terminology such as hot seating, improvisation, teacher in role). The information for this was developed from the syllabus of one of the top Drama schools and its external examinations (LAMDA).

In each year group, we have structured Schemes of work that focus on one of these skills at a particular level (although the latter is flexible). Therefore, throughout the course of a year, students will develop their skills in each of the three key elements. It was felt that the current assessment schemes are extensive (they covered our key areas of learning) and distributed (we took multiple snapshots during the year) but that there is not a synoptic element. 

With this in mind, we intend to change the final Schemes in each of the three year groups from Assessment cycle 4 to Assessment cycle 6 (wht will be the new assessment cycles 3 to 4) and to create mini projects that would culminate in a finished production.  This final production could be showcased to parents/other students. This production could be either a Devised or Scripted play and is excellent preparation for a GCSE in the subject. It would also allow students to focus on ALL of the skills as they would each have a character and be equally responsible for the staging and production of the piece which would develop both skills and knowledge. This would ensure that at the end of the year the grade for the student would include a synoptic reflection of their overall ability and be an excellent indication of general ability for the new teacher/student and indeed parent. It would also be useful information for those students intending to study the subject at GCSE level.”

What interests me in this approach is that Drama has been able to use the opportunity afforded by not having a national curriculum to devise a coherent and integrated curriculum and assessment model.

The curriculum has content (knowledge and understanding), a procedural dimensions (how actors go about their business and the habits of mind that need to be developed) and whilst not apparent from the above it also has the development of the learner (metacognitive) as part of the overall package.

The assessment programme, used for summative purposes and grades reported, is distributed, extensive and now synoptic.  The department intend to “roll over” the synoptic element from Year 7 to 8 and 8 to 9 to use in the mid-year summative grades that are reported to prevent the beginning of Year 8 drop seen in the examples above.

The approach leads to a really well though through programme that will keep the wolves away from the door and would be good enough for my children – key touchstones when making decisions.

Other departments all had various ideas for strengthening their assessment processes and making the outcomes we report and use to target interventions more valid and hence trustworthy:

• Religious Studies have already developed an assessment system that is both distributed and synoptic. It consists of a series of end of module tests which are “cumulative”, that is they include in each module test questions from the previous module(s). However, they intend to focus on skills based assessment at Key Stage 3 as they considered the results would be far more consistent  than knowledge based assessment. Since fifty percent of the Religious Studies GCSE marks are for questions that assess reflection and reasoning skills, i.e. evaluation, these skills should be developed and the focus for assessment in lower school RS. Their assessment programme will now be more extensive.
• English were concerned that they were using individual assessment pieces as the basis of reporting and intend to move to a more distributed model where they use a number of pieces to determine grades for reporting purposes.  This matches well with the portfolio approach that they already have that builds up a student’s English profile.  As a spin off of the more distributed approach they can move away from a written assessment, prior to an attainment grade being entered for a student, and increase the amount of speaking and listening which further secures the extensive nature of their assessment programme.

And now the final principle:

Principle Five: Manageable (so that costs are proportionate to benefits)

We currently have six assessment cycles per year.  This was madness before we introduced the Marking Policy this year and attempted to integrate the two.  Staff at St. Mary’s are big hearted and always up for a challenge but this particular “spinning wheel” challenge is getting them down and not yielding enough benefits for the students.   The teachers are honest, we needed to improve the quality and quantity of our marking as a school, but as a leader I also need to be honest and accept the system is unmanageable.  The proposal for September is that we move to four assessment cycles for subjects with ten percent or more curriculum time and two for subjects with less than ten percent.

Our draft proposals for Assessment, Recording & Reporting at St. Mary’s next year

This is a fine tuning of the system as are many of the proposals out of yesterday afternoon’s CPD but such is the Redesigning Schools programme.  It is about developing best and next practice not throwing out baby and bath water and starting again.  Much of what we do is already good, we need to have the mindset and determination to just keep doing things a bit better each day, week, month and year.

Talking of doing things a bit better, it’s worth spending a moment reflecting on current plans for GCSE and A-levels and whether they meet the key principles associated with high quality, effective summative assessment.  My own thoughts are that the arrangements will be far more manageable and clearly may contain a synoptic element.  However, the absence of distributed and extensive elements is a matter of concern and will bring the trustworthy element into question.  The new examination system may restrict what we do but it must never define us.  It will change and hopefully evolve to something that is far more fit for purpose.  There was a reason why we moved away from the O-level and A-level end of course examination system of my youth.  Maybe that lesson should be included in the new History Curriculum.