Assessment

Redesigning Classrooms: Is Your Summative Assessment Trustworthy?

Are we about to enter a national assessment black hole?  We may soon be facing an assessment system with a terminal exam at the end of Year 11 and Year 13.  A five year gap, from the beginning of secondary schools to GCSE, without any nationally recognised or externally accredited examinations to affirm for us how our students are performing. 

In contrast, when I started at St. Mary’s we had end of Key Stage 3 SATs, the option to buy in SATs papers for the end of Years 7 & 8, module assessments throughout Key Stage 4, coursework at GCSE and the new modular A-level with coursework modules available.  These have already or are about to disappear and to confound the matter further the safety blanket of levels may soon be ripped from around us as well.

We are going to need to get much better at assessment including the summative element by understanding key principles and developing some highly effective practice.  This blog post is built on the symposium led by Dylan Wiliam that I have previously written about titled, “Redesigning the School’s Curriculum: Find your Compass.”

Below is some actual summative data from three different students, in three different subjects produced in response to our assessment system.  It formed the starting point for a Thursday afternoon staff CPD session that started with the simple question, “So, what do you notice?”

Staff not only notices things but began to explain them.  First up were the drastic differences in between some consecutive assessment cycle grades – students B went from a 6a to a 5c, nearly a two year regression in about half a term is some going.  Also staff picked up the often significant drop as students moved from Year 7 to 8, look at both students A & B.  The tendency to use one test/assessment piece to produce the summative grade, the narrow set of skills being assessed and the lack of transfer of assessment data from Year 7 to 8, even though it is all available in the SIMS, were raised as issues.

It presents the fundamental question, is our summative assessment system trustworthy?  Do students and parents have faith in the outcomes?  Is it actually telling us which of our students are more able mathematicians, linguists, scientist, historians, geographers etc?

Principle one: Is the school’s system to be trusted (do stakeholders have faith in the outcomes)?

The challenge was set to look at current practice in relation to the set of principles that would guide the building of a reliable summative assessment system.  Are assessments:

Principle Two: Distributed (so that evidence collection in not undertaken entirely at the end)

Principle Three: Synoptic (so that learning has to accumulate)

Principle Four: Extensive (so that all important aspects are covered)

Interestingly, we may find some help from an unlikely source as it is often the Cinderella subject in the school’s curriculum if it is there at all – Drama.  Drama has operated in the Key Stage 3 Curriculum black hole for an eternity.  The following was written by Cathy Lloyd, Head of Performing Arts, and it’s interesting to reflect on what lessons are transferable and applicable.

“Currently there is no National Curriculum for Drama. Because of this, it was decided to create a syllabus based around the skills of an actor, namely technical ability, interpretation of a character and knowledge (of both the theatrical setting and Drama terminology such as hot seating, improvisation, teacher in role). The information for this was developed from the syllabus of one of the top Drama schools and its external examinations (LAMDA).

In each year group, we have structured Schemes of work that focus on one of these skills at a particular level (although the latter is flexible). Therefore, throughout the course of a year, students will develop their skills in each of the three key elements. It was felt that the current assessment schemes are extensive (they covered our key areas of learning) and distributed (we took multiple snapshots during the year) but that there is not a synoptic element. 

With this in mind, we intend to change the final Schemes in each of the three year groups from Assessment cycle 4 to Assessment cycle 6 (wht will be the new assessment cycles 3 to 4) and to create mini projects that would culminate in a finished production.  This final production could be showcased to parents/other students. This production could be either a Devised or Scripted play and is excellent preparation for a GCSE in the subject. It would also allow students to focus on ALL of the skills as they would each have a character and be equally responsible for the staging and production of the piece which would develop both skills and knowledge. This would ensure that at the end of the year the grade for the student would include a synoptic reflection of their overall ability and be an excellent indication of general ability for the new teacher/student and indeed parent. It would also be useful information for those students intending to study the subject at GCSE level.”

What interests me in this approach is that Drama has been able to use the opportunity afforded by not having a national curriculum to devise a coherent and integrated curriculum and assessment model.

The curriculum has content (knowledge and understanding), a procedural dimensions (how actors go about their business and the habits of mind that need to be developed) and whilst not apparent from the above it also has the development of the learner (metacognitive) as part of the overall package.

The assessment programme, used for summative purposes and grades reported, is distributed, extensive and now synoptic.  The department intend to “roll over” the synoptic element from Year 7 to 8 and 8 to 9 to use in the mid-year summative grades that are reported to prevent the beginning of Year 8 drop seen in the examples above.

The approach leads to a really well though through programme that will keep the wolves away from the door and would be good enough for my children – key touchstones when making decisions.

Other departments all had various ideas for strengthening their assessment processes and making the outcomes we report and use to target interventions more valid and hence trustworthy:

• Religious Studies have already developed an assessment system that is both distributed and synoptic. It consists of a series of end of module tests which are “cumulative”, that is they include in each module test questions from the previous module(s). However, they intend to focus on skills based assessment at Key Stage 3 as they considered the results would be far more consistent  than knowledge based assessment. Since fifty percent of the Religious Studies GCSE marks are for questions that assess reflection and reasoning skills, i.e. evaluation, these skills should be developed and the focus for assessment in lower school RS. Their assessment programme will now be more extensive.
• English were concerned that they were using individual assessment pieces as the basis of reporting and intend to move to a more distributed model where they use a number of pieces to determine grades for reporting purposes.  This matches well with the portfolio approach that they already have that builds up a student’s English profile.  As a spin off of the more distributed approach they can move away from a written assessment, prior to an attainment grade being entered for a student, and increase the amount of speaking and listening which further secures the extensive nature of their assessment programme.

And now the final principle:

Principle Five: Manageable (so that costs are proportionate to benefits)

We currently have six assessment cycles per year.  This was madness before we introduced the Marking Policy this year and attempted to integrate the two.  Staff at St. Mary’s are big hearted and always up for a challenge but this particular “spinning wheel” challenge is getting them down and not yielding enough benefits for the students.   The teachers are honest, we needed to improve the quality and quantity of our marking as a school, but as a leader I also need to be honest and accept the system is unmanageable.  The proposal for September is that we move to four assessment cycles for subjects with ten percent or more curriculum time and two for subjects with less than ten percent.

Our draft proposals for Assessment, Recording & Reporting at St. Mary’s next year

This is a fine tuning of the system as are many of the proposals out of yesterday afternoon’s CPD but such is the Redesigning Schools programme.  It is about developing best and next practice not throwing out baby and bath water and starting again.  Much of what we do is already good, we need to have the mindset and determination to just keep doing things a bit better each day, week, month and year.

Talking of doing things a bit better, it’s worth spending a moment reflecting on current plans for GCSE and A-levels and whether they meet the key principles associated with high quality, effective summative assessment.  My own thoughts are that the arrangements will be far more manageable and clearly may contain a synoptic element.  However, the absence of distributed and extensive elements is a matter of concern and will bring the trustworthy element into question.  The new examination system may restrict what we do but it must never define us.  It will change and hopefully evolve to something that is far more fit for purpose.  There was a reason why we moved away from the O-level and A-level end of course examination system of my youth.  Maybe that lesson should be included in the new History Curriculum.

Redesigning the School’s Curriculum: Find Your Compass

The world of education is changing again and like many changes it is bringing us almost full circle.  If you are a teacher of about fifty years or older you may just about be able to remember a time before the National Curriculum and when levels weren’t part of your life (even if like me you are struggling to remember what you had for lunch or where your car keys are).  This is not quite taking us back full circle to pre-1988 as the sharp edge of accountability means you aren’t likely to tell a new Science teacher that you “can teach them (less able Year 10 group) anything you like as long as you keep them in the classroom.”  The latter part of this instruction proved more difficult than I at first thought.

Much has changed in education with respect to the level of informed professional practice of teachers and the degree of accountability we all experience.  The excellent SSAT Redesigning School Symposia are up and running in both Manchester and London.  With Dylan Wiliam leading on the development of new curricula and assessment – the “what” we have to do – and Andy Hargreaves on Professional Capacity – the “how” to do it – we can move with both excitement and no small degree of trepidation into the next phase of education in this country.  It’s time for the profession to take the lead, working with government and other interested parties, to deliver an education fit for the 21^st Century and fit for our students.

Dylan’s classic one liner was “don’t implement things you don’t believe in.”  I wonder what your school’s response was to the E-Bacc a number of years ago.

1. Didn’t make any impact as we already required students to opt for two Sciences, a MFL & either History or Geography
2. Didn’t make any impact as we thought it was a bonkers idea, didn’t fit with our curriculum philosophy and so we ignored it and carried on regardless.
3. Panic – get all students opting for the E-Bacc (or at least the most able as they will get a grade C), options changed rapidly and you’ve either now seen the light of Mr. Gove’s wisdom or wondering what you’ve done and why.

We need to get our curriculum compass out and make sure we know which way we are heading particularly if, after years of being told what to do, we’ve lost sight of why we are doing certain things within the curriculum.  At the symposium we were challenged to think about seven curriculum principles and which where the most important three.  I failed miserably to identify just three but managed to realise it is the tension between the different principles that was going to be key in breathing life into the curriculum at St. Mary’s.

Dylan’s principles for a good curriculum were: balanced, rigorous, coherent, vertically integrated, appropriate, focused and relevant (you need to take care with these terms as they had a technical meaning that doesn’t necessarily relate directly to everyday use and meaning).

In developing our new curriculum what would teachers need to know if they were going to be effective?  Here’s my starter for teachers but you will need to add your own thoughts as well:

1. How their subject knowledge and concepts are vertically organised and integrated – what order do students learn these ideas and so what order should we teach them?
2. What are the habits of mind, attitudes and skills students must develop if they are to be successful learners within the subject?  What gets better when a student’s understanding of your subject increases?
3. How does the subject relate to other subjects and society – what numeracy or Mathematical skills do students need to be successful in Science & Geography in Year 8?
4. How do students learn – can we transmit knowledge and understanding or must they construct it?

What does research say about the most effective teachers and the pedagogy they use?  Hattie goes for: teacher clarity (learning intentions & success criteria), feedback (but make sure the learner has to respond to it by improving his/her work), relationships and peer discussions.  If you have fifteen minutes this is a great summary of Hattie’s work presented by himself on what to do and what not to do.

I had the chance to listen to Dylan a couple of year’s ago at a SSAT Think tank and immediately went back to school to redraft the College’s Curriculum Policy.  After discussions with staff it was written in the style of a handbook and may give you some ideas about what to do and what to avoid.  We are currently challenging ourselves to increase the pace in Key Stage 3 by building programmes of study based on learning intentions written in SOLO Taxonomy style.  We want to make seven sub-levels progress in Key Stage 3 for all students and more with the most and least able.

Now that brings me to the afternoon’s work which nearly made my head explode.  What are we going to do if, as it seems likely, levels disappear?  If you are a teacher under fifty you won’t know life without levels.  Now don’t get me wrong I realise that levels were pretty much made up by subject groups sat in rooms, have been revised, revised again, had sub-levels introduced and by now probably don’t really link to how subject knowledge and concepts are vertically organised and integrated.  I know this but there is something very comfortable about the familiar.  I sense the comfort blanket is about to be taken away and we are going to grow up rapidly as a profession – we are more than capable of meeting this challenge with a little more wisdom from Dylan and some assessment principles.  We focussed on summative assessment.  The principles to guide in building a reliable system are:

1. Distributed (so that evidence collection in not undertaken entirely at the end)
2. Synoptic (so that learning has to accumulate)
3. Extensive (so that all important aspects are covered)
4. Manageable (so that costs are proportionate to benefits)
5. Trusted (so that stakeholders have faith in the outcomes)

The introduction of data and its use in schools over the last decade has taught us much: make sure you know a student’s starting point (teach them from this point), set challenging and explicit goals linked to specific knowledge, understanding and skills that s/he needs to attain and check how they are going during lessons, at the end of topics and periodically through the key stage.  The information generated from assessments can be used to monitor progress and intervene when necessary.  This is great learning for us as professionals.

As we hopefully take the lead in matters of curriculum, pedagogy and assessment there is a need to make sure we build the professional capacity within our school, localities and nationally.  That was day two of the Redesigning Schools symposia with Andy Hargreaves, PC = f [HC, SC, DC], the “how” we can do it.

This is my second ever blog, the first attempt to write one I have managed to lose in the ether.  If you find it please send it back to me.  I’ll gather my thoughts on the building of professional capital, courtesy of Andy Hargreaves and blog again.

Thanks to Dylan Wiliam for a fantastic day and to Sue Williamson who is leading the SSAT onto fertile and crucially important ground.  Interested in Redesigning Schools?  Get involved.