Comparable outcomes and comparable progress
2017 saw the first results of the new 9-1 GCSE specifications in England in maths, English language and English literature. Last August Ofqual’s comparable outcomes policy ensured outcomes that were very comparable with outcomes in the predecessor A*-G specifications in 2016.
As we all knew in advance, Ofqual had guaranteed that, as long as the national cohorts were ‘similar’, a broadly comparable % of students would achieve 4+ in 2017 as achieved C+ in 2016, and that a broadly comparable % of students would achieve 7+ in 2017 as achieved A+ in 2016.
The raw outcomes (England only) are tabulated below:
There were some differences in the national cohorts, largely because of two Ofqual decisions:
- That the best of English language and English literature would count double in Progress 8 if a student sat both qualifications;
- That IGCSEs would not count for schools in the Performance Tables from 2017.
In 2017, 678,449 students sat GCSE English language compared to 454,339 in 2016. 545,944 students sat GCSE English literature in 2017 compared to 372,621 in 2016. However, there was much less difference in GCSE mathematics as 703,140 sat this in 2017 compared to 698,546 in 2016.
There was another difference in 2017, relating to the prior KS2 attainment of the cohort. There has been a significant shift upwards compared to the 2016 cohort as has been outlined in our ‘2018 - Benchmark Briefing Paper KS4 and KS5’. The removal from 2012 of the KS2 writing scores from the calculation of KS2 levels appears to have had a significant impact on this ‘prior attainment inflation’ as have rising standards in the primary sector.
So, the key question for us at Alps, and for our growing cohort of KS4 schools, has been ‘What impact will all or any of the above have had on value-added progress in the 9-1 linear qualifications in 2017 compared to the A*-G qualifications in the same three subjects in 2016?’.
This question would only be answered during our re-benchmarking process that takes place annually after we receive the validated national data set from the DfE.
Alps methodology will be familiar to most readers, but here are two refreshers:
- An Alps score of 1.00 always means that on average that set, subject, school or college has achieved value-added progress equivalent to matching the 75th% nationally.
- Students make greater progress in some subjects nationally than they do in others, so the Alps scores required to be in the top 25% of each subject nationally varies. Some subjects need a higher score to achieve an Alps grade 3 (top 25% value-added performance) whereas others require a lower score in order to achieve the same Alps grade.
Table 2 below compares the Alps scores required for grade 3 in each of the 2017 9-1 subjects the Alps grade 3 scores in the same subjects when awarded as A*-G qualifications in 2016
So, as can be seen above, the shift in the value-added score needed for grade 3 in mathematics in 2017 is only +0.01 different to the score required in 2016.
There is a bigger difference of +0.02 in English language and of +0.03 in English literature. So, cohorts of students in our schools now need to gain a slightly higher overall grade profile in these two English qualifications than previously, as our new KS4 benchmark reflects changes in national performance.
As can be seen in Table 1, a higher percentage of students achieved grade 4 in 2017 compared to grade C in 2016 and a higher percentage of students achieved grade 7 in 2017 compared to grade A in 2016 in both English language and English literature.
Was this despite of, or because of, the rising tide of students who took home board GCSEs in 2017? Has value-added progress in English literature risen most because schools that solely focused on English language in the 5A*-C including maths and English language era now focus more equally on both when either can count double in Progress 8 (whilst a good grade in the other drops into a student’s Open Bucket) thus nailing 30% Progress 8 through English alone?
Alps clients now have complete access to all our national benchmarks based on 2017 data at both KS4 and post-16; this can be found on your Connect Homepage.
Finally, the key question relating to GCSE in 2018 for schools, and all of us at Alps, is how the examination boards and Ofqual manage ‘comparable outcomes’ in the much wider range of subjects this year. Frustratingly, we will not know the answer until after the DfE statistics come out later in the Autumn. However, we anticipate some flux in the system, especially in GCSE DA - Combined Science with its brand new 17 grade system and Synergy and Trilogy qualifications. Is that the direct successor of both Core and Additional Science or a new qualification that can behave ‘incomparably’ to those qualifications? Will it be taken by the same cohort who took both Core and Additional or by a greater number of students? It is because this is a new qualification that at Alps we are unable to provide a 2018 benchmark for the DA - Combined Science initially.
We are aiming to create a DA - Combined Science benchmark using our client data in the Autumn term, and so we are asking that you do upload your Combined Science Synergy and Trilogy grades using Connect Data on Results Day with all your other KS4 results. Some main indicators (e.g. Red and Blue Teaching) will be impacted by the missing benchmark, however, the progress of your students in DA - Combined Science will still contribute to your overall school QI and QI grades.
Two things all our schools can rely on in the rapidly changing examination, assessment and accountability landscape are:
- Alps will work to ensure that you are able to track students and subjects aspirationally but fairly in Connect Interactive in 2018-19;
- As always, our reports and consultants will continue to point you towards the correct strategic improvement priorities for the year ahead.