Academic Review: Don’t Blame, Find Out and Fix

It is that time of year when schools up and down the country are looking at the summer’s results.  There is a tendency when things have gone well to smile, congratulate and move on rather than identifying the key strengths in your approach that have led to you success. 

However, what I want to think about is what approach to take when things are a bit more varied:

The Ostrich Approach?

The Headless Chicken Approach?

The Sledgehammer to Crack a Nut Approach?

If I’m honest over the past thirteen years as a Headteacher I’ve probably done all three.  Looking at the pictures above I’m not sure I have quite nailed the right approach yet.  Time for something different, perhaps?

Academic Review

The idea for an academic review came from Serious Case Reviews which involves looking objectively at problems following a significant child protection or safeguarding case.  The focus is on determining what went wrong so improvements can be made rather than blaming people, though I can only imagine the stresses and difficulties involved for people caught up in the process.  We decided to drop the word “serious” as it wouldn’t exactly help staff involved remain calm and objective but use the approach to look at any areas where results gave us cause for concern.

The obvious questions of “who, how, what, where and when” formed our starting point and this is the process we have decided to implement for this year.

The “who” to review will be based on progress measures, for example, ALPS post-16 which is often available a week after A-level results and three levels progress at GCSE rather than simply using raw data.

It was also felt a small team approach would be beneficial partly to have a number of different pairs of eyes on the information and any potential recommendations and partially for professional development of the team members and to allow good practice to be discussed and shared as part of determining and implementing any recommendations.

The following is the introduction to the review paper:

Introduction

The Academic Review Process has been developed to investigate issues relating to student underperformance.  It is a “no blame approach” that seeks to understand why we have, as an organisation, been unable to ensure students meet their full potential.

An academic review will only be initiated where there is evidence of low achievement rather than just low raw results which could be indicative of a cohort’s ability profile.

The use of ALPS, at AS & A-level, and three levels progress against national norms, at GCSE, will be key indicators for the need for an Academic Review.  An ALPS grade of 8 or 9 would represent significant underachievement as would a significant gap between a department’s and national norms of three levels of progress.

It is important that the review is evidence based and supported by open honest and professional discussions if the correct conclusions are to be reached and appropriate recommendations made. 

The process must be completed by the end of September and work alongside the annual analysis and evaluation of results.

The Academic Review will be led by a Deputy Headteacher, nominated by the Headteacher, the Senior Head of Learning House (Assistant Headteacher) and the Head of Department.  None of the team’s members will have been involved in line managing or leading the departments undertaking the review.

The Headteacher may determine an Academic Review, even when there is significant evidence of underachievement, if s/he considers that there are evident and material reasons that are already evident, for example, the long term absence of members of staff responsible for teaching the class(es).  This decision is at the sole discretion of the Headteacher.

We then tried to identify the reasons that may have led to possible variation in performance and the usual reasons for in-school variation are linked to quality of teaching or quality of systems and processes supporting the teaching and teachers.

Accepting all the imperfections of the lesson observation process the gradings from lesson observations gave 85% of lessons graded as good or better in the last academic year.  It is most likely that we need to strengthen processes but which ones?  The following is a series of prompt questions for the head of department and review team to work through to pinpoint the areas for improvement that are recorded in the key recommendations.  Some of the areas may lie within the control of the department and others may link to issues that need to be considered at a whole school level.  The important thing is to evidence

Academic Review Process

The following information and evidence should be collected and reviewed prior to drawing any conclusions:

• Overall quality of teaching within the department.  This should include data from the last two complete Appraisal (Performance Management) cycles and is available from the Headteacher.
  • Is the quality of teaching good enough to deliver consistently good outcomes for students?
  • How effective has marking been in improving levels of achievement?

• Module and coursework/controlled assessment marks.
  • Was there underperformance in any particular modules/elements of the course?  Who taught these modules?
  • Were coursework/controlled assessment deadlines set and adhered to by the department?
  • Was coursework moderated by the examination board?  What reasons were given in the Moderator’s Report for the adjustment?
  • At A-level, individual student performance against ALPS target grade. 
    • Is there a particular sub-group of students, for example bursary students, who underachieved?
    • At GCSE, class residuals and free school meal versus Non FSM comparative data. 
      • Did a particular class/sub-group of students underachieve?

• Students’ projected grades for the full academic cycle should be compared to actual grade achieved. 
  • Were predicted grades accurate and were underachieving students identified early?
  • What internal processes were in place for standardising grading judgements made by teachers within the department? 
  • What processes were in place for moderating internal judgements against examination board criteria or grade boundaries?
  • Were grades entered into the assessment cycles, for individual students, standardised and moderated at a departmental level?  For example use of Examiners’ Reports, on-line results services and exemplar materials?
  • What processes were in place to ensure summative grades were valid & reliable?  To what extent were current and predicted grades based on extensive, distributed and synoptic assessment practices?

• Data on PARS for student being taught in the classes of the subject under review should be collated.
• Evidence of interventions with individual and groups of students should be collated at a departmental and whole College level.
  • Was information entered on PARS in a timely manner in line with the College’s Behaviour Policy for students in the subject under review?
  • What interventions were put in place for identified underachieving students at a departmental and whole school level?
  • What impact did these interventions have and how was this measured and tracked?

• Schemes of work, and at A-level Y12 & 13 transition and revision programmes, should be reviewed.
  • Are schemes of work of sufficient quality to secure the required student outcomes?  Is there an aspect of differentiation to challenge high attaining students and support lower attaining students?
  • Do schemes of work ensure full coverage of the course in a timely manner?  Is time given to each element/aspect of the course roughly comparable to assessment weighting?
  • Do the revision programme support achievement within the course?  How is the work completed monitored and assessed?
  • Do the transition units support achievement within the course?  How is the work completed monitored and assessed?

• Evidence of Other & Extenuating Factors.
  • Are there any other or extenuating factors that help explain the underachievement?
  • To be considered, these factors must be specific to the department and not have affected other departments where students have achieved highly nor the department previously when achievement had been high. 

Recommendations
In determining recommendations we need a “No blame but no excuses” mentality, only reasons supported with evidence are allowed.

The discussion of this review process within the senior team has already led to some really rich and deep conversations with honest appraisal and suggestions that, when implemented, will form a stronger basis for further improvement.  This will lead to consistently good and then hopefully outstanding achievement by our students.  The first round of reviews in September 2013 must set the right tone, challenge and support is a difficult balance.  If we can get people fascinated with the improvement process and possibilities to make things better for our students then we will make massive strides forward. You can’t do this when you are stressed or fearful about consequences.  It’s for our students that I am committed to a “No blame but find out and fix” the issues.  I work with great people who are committed to our shared vocation.  They deserve better leadership that can be provided by an ostrich, headless chicken or sledgehammer.

A copy of the Academic Review Paper is here:

Paper – Academic Review Process v.2 03.09.13

A lot of this thinking influenced the co-construction of the #5MinResultsAnalysis with @TeacherToolkit (Ross McGill).

#5MinResultsAnalysis – Updated 2014

The post and #5MinResultsAnalysis resources may be of use and interest to you.