Are You Ready for the Reckoning?
How might you account for any differences between your Alps VA and the Performance Tables VA (L3VA)?
Schools and colleges that eagerly obtain their Alps analysis in August wait until November (unvalidated) and January (validated) to catch sight of the L3VA, and how it is represented in the Performance Tables and the Ofsted Inspection Data Summary Report.
They clearly want the two different VA analyses to tell a very similar story – and they commonly do, both for the school / college overall and in terms of how individual subjects are rated.
Alps benchmarks progress against the previous year’s DfE national data, so that schools and colleges can obtain their VA analysis when they need it – around Results Day each August. In contrast, the L3VA / Performance Tables use current year’s data, which may account for minor differences between the two analyses. However, Ofqual’s insistence on ‘comparable outcomes’ means changes in the national data annually are slight.
If you find differences between your Alps and L3VA analysis, here are some quick tips and reasons to help you see why the two interpretations may vary:
1. Alps is a pure A Level VA analysis whereas the Performance Tables (L3VA) includes all AS outcomes taken by students in that cohort if they were not continued to A Level and counts each as a 0.5 A Level.
Action: Check your L3VA for the number of discontinued AS grades that are being counted and whether they are having a positive or negative impact.
2. The L3VA is a cohort analysis so it excludes all students who were not in that post-16 cohort who sat A Level qualifications, whereas Alps includes the uploaded results of, for example, Y14 students.
Action: Check how many ‘out of cohort’ students are in your Alps report and whether they are having a positive or negative impact on your Alps VA.
3. Despite being a cohort analysis, the L3VA can include students who took AS up to 3 years ago. These ‘ghosts’ left after taking at least 1 AS (though commonly 3 or even 4) and have now turned 18 without the DfE finding them in any subsequent educational provider.
Action: Check your L3VA for the number of ‘ghost students’ that are being counted and assess their negative impact.
4. Both the L3VA and Alps baseline progress based on students’ average GCSE scores, however since 2018 the L3VA has aligned GCSE scores with the 9-1 scale. For 2018 and 2019 outcomes Alps has used the 8-1 (A*-G) scale.
The L3VA change was implemented for the cohort who took GCSEs in 2016, when no 9-1 grades were available. See the extract below:
“From 2018 onwards, prior attainment in L3VA reflects the point scale used at KS4 in the current reporting year for all students (e.g. for the 2018 L3VA, the scale used for GCSE and other qualifications will be the points used in the 2018 key stage 4 performance tables). Qualifications in scope will be those approved in KS4 performance tables in any year from 2014 onwards. Simple discounting rules apply, and the best results in any subject will be used when calculating the average point score. As before, re-sits or additional qualifications gained during 16-18 are ignored.”
From 16 to 18 accountability measures: technical guide for measures in 2018 – October 2018, page 73
Note: Alps has adopted the 9-1 scale for monitoring in the Academic Year 2019.20 and the 2019 examination results will reflect this.
Action: Compare the students’ prior attainment in Alps and the L3VA.
5. The L3VA applies confidence intervals, and the Lower Confidence Interval (LCI) has to be above the score of 0.00 for the progress to be graded as ‘Above Average’ (Sig+) and below the score of 0.00 to be graded Sig-.
Action: If a subject is red-hot and not Sig+ check that the L3VA score is positive and whether the Lower Confidence Interval not being above 0.00 has made the VA ‘not significant’. If a subject is blue and not Sig- check that the L3VA score is negative and whether the Lower Confidence Interval not being below 0.00 has saved the VA from being ‘Below Average’.
It is worth noting that the two systems are set up to do different core tasks. The differences described above do not impact on your core Alps analysis and your ability to use Alps for real, timely performance improvement and intervention plans.
We have a technical document on our Knowledge Base which compares the main differences between the two systems. In the Knowledge Base search for ‘Alps and the L3VA Technical comparison’.
If you get really stuck, ring Alps on 01484 887600 and ask to speak to one of our Educational Team who will be more than happy to discuss this with you.
Alps Senior Educational Consultant