3. What do you consider a reasonable period for a provider to review its practices and enact appropriate changes?
The timeframe is a matter for the sector, regulator/s and providers, but we would want to see sufficient time for meaningful consultation regarding implementation with professional bodies/PSRBs1, employers, current and prospective students in terms of safeguarding the value of degree classifications of alumni and future graduates.
4. How can the statement of intent be taken forward by the different national higher education systems of England, Wales, Scotland and Northern Ireland within their national quality and regulatory frameworks?
We encourage a consistent and unified approach across England and the devolved nations.
5. Are the evidence areas proposed at Table A for inclusion within a 'degree outcomes statement' appropriate for supporting an institution to identify potential 'grade inflation' risks and provide assurance to maintain public confidence?
In part.
We support academic autonomy and professional judgment. It is important to guard against the idea that universities have some sort of ‘national curriculum’.
There are a range of complex and inter-linked factors driving the range of degree outcomes awarded by a provider, and how those are benchmarked against peers or the sector more widely (Figure 10 on p24/25 of this UUK report). The Degree Outcomes Statement addresses only a few of these.
One notable absence is the operation of Examination boards / Assessment boards, in terms of the influence of local and subject-specific practices in determining student progression, and classifications of awards.
Although providers already publish a considerable amount of information covering many of these topics, it is positive to see that providers are being encouraged to articulate how these operate in an interdependent way towards the determination of degree outcomes, rather than considering them as policies in isolation.
We question how degree outcome statements would present and address the use of multiple approaches within a single institution and/or local variations or influences on algorithms within subjects, e.g. those required by PSRBs. Although our accreditation does not mandate approaches to assessment, student progression or award algorithms, some bodies do and we encourage greater transparency around this.
6. Do you consider there to be merit in gaining assurance from an 'external advisor on academic standards'?
No.
Please set out any other mechanisms for enhancing external assurance
We value and encourage greater reflection on student outcomes, especially subject-wide. However, this proposal is not the most practical way to address the challenge.
We value subject external examiners and support their autonomy and professional judgment.
The proposal does not describe sufficiently the relationship between the ‘external advisor on academic standards’ and external examiners; this is potentially problematic given that there is already known variation in the responsibilities and expectations of external examiners across the sector.
The proposal for a ‘super-examiner’ may add an additional burden on providers, which (as demonstrated by Advance HE research) already faces challenges in recruiting specialist examiners in some areas, and relatively low diversity overall.
We strongly support the strengthening of external examining practices within institutions, which may include taking a wider range of input from outside HEIs, e.g. practitioner and employer examiners across subject areas, with appropriate support to aid their understanding the application of regulations.
7. What are the a. opportunities and/or b. challenges associated with including the commitments to strengthening the external examiner system in the statement of intent?
a) Opportunities
We strongly support the continuing professional development of external examiners within subject specialisms.
We would welcome Advance HE’s work on professional development of external examiners being rolled out more widely, and more fully evaluated, in light of discussions around sector-wide or subject holistic calibration of degree classifications.
The Society’s involvement in pilot studies for the training and calibration of external examiners with Advance HE showed that a social moderation approach (critical discussion, using student work exemplars, about how subject-specific academic standards are defined, described in marking criteria and subsequently demonstrated by students) was of benefit both to academics in their role as external examiners in other providers and to their role as assessors in their home institution. The pilot workshops suggested that calibrating understanding of assessment tasks and academic standards before and after an assessment task had the opportunity to reduce the amount of variation in marks given, and therefore demonstrate a more closely aligned understanding of academic standards within and between institutions. Participants also valued the opportunity to share and discuss subject-specific assessment approaches. The proposals for the development of external examiners around comparability of standards could be considered beneficial for all academics engaged in assessment, not just external examiners.
It is highly unlikely that subject bodies can provide equal levels of support for the development of external examiners; not all subjects have a corresponding subject body with a remit of engaging with higher education. However, where appropriate subject associations exist, they should be integral to future work in this realm.
b) Challenges
UKPSF is, quite rightly, academic-facing. It does not, however, provide a clear framework for the induction, development and recognition of professional or dual-profession (academic/practitioner) examiners (which is more diverse and varied across sector than external examiners more generally).
UKPSF does not appear to have been universally adopted by the sector, especially in relation to internal training and development, such as that for inducting external examiners, which varies considerably by institution.
There is no explicit relationship between UKPSF and Subject Benchmark Statements (or other external articulations of subject-specific knowledge and skills in UK HE teaching) as a reference for assuring standards in a subject-specific way.
11. Does the proposed classification description in Annex A provide an appropriate reference point for degree classification practice?
No.
No relationship with Subject Benchmark Statements appears to have been considered in their preparation, nor with Degree Apprenticeship Standards, yet we note that these external reference points have the greatest input by employers and non-HEI stakeholders (whom the sector is seeking to reassure about standards).
The proposed classification description might be useful as a reference point for institutions and departments to calibrate their understanding of sector-wide standards, but as it stands, it is not fit for purpose as a sector-wide classification description as it is too vague to be prescriptive.
Institutional descriptions of degree classifications, where they exist, are generally more nuanced, appropriate to the institution’s teaching approach and mission. However, we note that this has resulted in varied approaches to descriptions of degree classifications, which has the potential to perpetuate uncertainty.
The proposed degree classification is subjective and internally inconsistent (especially the language used, e.g. in the top range: “sophisticated”, “thorough”, “accomplished”, “well beyond”, “excellent”, “mastery”; in the middling range: “adequate, “good”, “sound”, “proficient”). The choice of language also risks ambiguity with different subject interpretations. A high degree of professional judgment would be required to ensure comparability of standards across subjects.
12. Do you have any proposals for substantive changes to the classification criteria? Please explain your response.
We not recommend attempts to define/shape current classification system. Rather, we encourage approaches that consider creating new classification system more reflective of current assessment methods and practices, criterion-referenced assessment and sector/employer need.
External Examiners have, for some time, quite rightly encouraged providers and assessors to “use the full range of grades available” to recognise student achievement, and we question whether a classification description based on existing classification grade boundaries is still fit-for-purpose in a changing context.
13. Do you agree that the proposed classification description should be incorporated into national quality assurance and regulatory frameworks, as is appropriate for different national contexts? In England, this would mean the use of the proposed classification description as 'sector-recognised standards' as defined in section 13(3) of HERA.
The classification description lacks specificity in its current format, so is unlikely to be appropriate for underpinning regulatory frameworks.
14. How should the proposed classification description be incorporated into: a. institutional practice b. other relevant documents or frameworks?
If the proposed classification is adopted, it will be important for the QAA to give attention to the varying presentations of classifications and/or standards articulated in Subject Benchmark Statements.
In particular, where a sector-wide classification description varies from a subject-based classification description, advice from the QAA on which should take precedence will be important.
Further consultation with employers and PSRBs on the relationship between Benchmark Statements and sector-wide classification descriptors is strongly recommended.
15. What are the: a. benefits b. challenges, and/or c. national considerations of using a shared sector metric to inform institutional self-assessment of degree classifications over time?
We would like to see a unified and consistent approach across England and the devolved nations. It may not be prudent to implement a sector metric for England without consultation with devolved administrations on alignment.
The metric should be capable of taking into account a wide range of complex and interlinked factors driving degree outcomes, many of which are not currently captured by sector or university metrics (e.g. attitude and behaviour, learning gain). At very least, it would need to be accompanied by a provider narrative for context.
16. How should a sector metric for degree classifications over time be defined?
No answer.
17. How can sector reference points be better used, with more consistency, by external examiners to support institutions to protect the value of qualifications over time?
No answer.
18. Should the sector explore the steps that could be taken to remove, or reduce the impact of, the inclusion of upper degrees (1st and 2.1 awards) in algorithms used to rank university performance?
Yes.
In the event that the sector does take concrete steps towards addressing degree classifications, we would not wish to see league tables give a greater weighting to a ‘good’ degrees metric, given the myriad factors involved in degree classifications.
19. What should be the parameters and remit for a UK-wide task and finish group on the long-term sustainability of the UK's degree classification systems?
We support the suggestion in the consultation document that a Task and Finish group “work with employers and students to alter the perception of the value of 2:2 and 3rd classifications to avoid the perception that only a 2:1 or better is a ‘good’ degree”, and to work with PSRBs and employers more broadly on their understanding of academic standards as articulated by a degree classification.
However, we caution that there may be an unintended consequence of strengthening trust around ‘good’ degrees; seeing the value of 2:2s and 3rds further downgraded.
20. Which of the following options for reforming or enhancing the degree classification system should be considered in more detail? (Please indicate Yes/No)
Introduction of new upper award - for example, a starred first
No.
Introduction of a 'cohort ranking' - for example, providing additional information on graduates' position in the grade distribution
Yes, but we would recommend a percentage band ranking (e.g. top1%, top 5%, top 10%, …, which must include class size).
Resetting the classification boundaries - for example, moving up by 10 marks so 80 = 1st and so on
Yes, but not in the way that you suggest; a more comprehensive data-led review of grade boundaries based on analysis of marks, including subject-based variations.
More regular review of Subject Benchmark Statements to keep pace with improvements in teaching and learning
Yes, but caution against burden; also some subjects do not have SBS.
Universal HEAR format
Yes.
Other (please explain) comments relating to specific proposals as noted above
No reform required.