Interpretation of KS5 Results on Results Day – approach with caution August 2023

In this article we outline why interpretation and analysis of the 2023 Key Stage 5 outcomes should be approached with caution in terms of the value-added grades assigned. In the blog below, Alps Educational Consultant John Roe writes about the importance of the Alps client benchmarks across 2022, and why full analysis of 2023 should take place once the client benchmarks have been published in Connect and Summit. Note that the current default benchmark in Connect and Summit is the 2019 national benchmark.

High level Summary

1.  In England, value-added grades in Connect and Summit are likely to be less red on results day. The cohort had a higher prior attainment on entry (TAGs) than in 2019, and outcomes are due to return to 2019 standards.

2.  In Wales, value-added grades may be in line with 2019. CDGs mean cohort prior attainment is higher, but outcomes also above 2019 standards.

3.  Client benchmark timeline for publication

• • A level England – Monday 28 August
  • BTEC 2016 England – Monday 11 September
  • A level Wales – Thursday 14 September

Alps KS5 Benchmarks in 2023

At Alps our value-added analysis is based on national datasets provided to us by the DfE. The last national dataset released for KS5, was in 2019 and the DfE have no current plans to release a new KS5 dataset whilst prior attainment is based on Teacher Assessed Grades (TAGs). In our Connect platform, we have used this 2019 benchmark to provide a consistent baseline through the pandemic to support schools and colleges in making judgements on improvement priorities.

In 2022, we provided schools and colleges in England with A-level analysis based on the 2019 DfE dataset, alongside trialling an alternative analysis based on benchmarks generated from 2022 Alps client data. This alternative analysis was made available in November of 2022, via the generation of an additional A-level report and a benchmark toggle in Connect. We did this to support schools and colleges due to the unique nature of A-level outcomes in 2022.

Our analysis showed that in 2022, A-level results were closer to the 2021 (TAG) outcomes than those of the previous examination year of 2019. There was also an uplift in prior attainment for the 2022 A-level cohort based on the awarded CAGs in 2020.

From our analysis of Alps client data containing approximately 350,000 entries, we were able to conclude that in 2022, only minimal differences existed between the A-level analysis based on 2019 DfE benchmarks and our 2022 client benchmarks. This we believe was due to a partial ‘cancelling out’ effect of these two opposing factors, (i.e., the higher awarding of grades and higher prior attainment based on CAGs). For example, when switching the analysis from the 2019 DfE benchmarks to the 2022 Alps client benchmarks, most schools and colleges remained on the same Quality Indicator grade unless they were near to a grade boundary. Fluctuation in subject grades was also minor when switching benchmarks.

After this trial and very positive feedback from schools and colleges in England, we have decided to continue our client benchmarks in 2023. Schools and colleges in both England and Wales will now be able to analyse their 2023 A-level results using both sets of benchmarks. The 2019 DfE benchmark will remain as the default in Connect, however alternative analysis based on 2023 client benchmarks will be available via a benchmark toggle.  We have also decided to make both sets of analysis available in Summit, again via a benchmark toggle.

Our A-level client benchmark analysis for schools and colleges in England will be accessible to users in both Connect and Summit from the week commencing 28 August. Client benchmark analysis will be available for Welsh providers on Thursday 14 September. We also plan to carry out vocational benchmarking where possible in the first few weeks of September.

Results in 2023

Although minimal differences existed between our 2019 DfE and Client benchmark analysis in 2022, we are expecting much greater differences in 2023. This is due to Ofqual stating that A-level results are to return to 2019 standards in England and Qualifications Wales stating their intention for results to be brought down to the midpoint between 2022 and 2019 outcomes. The average prior attainment of students sitting examinations in 2023, is likely to be higher than those students who sat exams in 2019. This is therefore likely to have a negative impact on value-added analysis when comparing outcomes with those of students and providers in the 2019 DfE dataset. As a result of this, we believe that our 2023 A-level client benchmarks may provide a more accurate reflection of your value-added outcomes and be the best benchmark to use when identifying improvement priorities.