Since its introduction nearly a decade ago, impact assessment for universities has been adopted in numerous locations around the world. With two recent announcements from international governmental bodies around the role of impact in future exercises – and at least another expected – we can observe several trends emerging.
The Future of the Funding-Based Research Impact Model
On the 15th June 2023, UKRI (UK Research & Innovation), the body responsible for directing research and innovation funding, announced its early decisions for REF (Research Excellence Framework) 2028 – the UK’s national assessment that ultimately determines how much of £2 billion annual funding is allocated to each university. Meanwhile, a letter dated May and sent by the UGC (University Grants Committee) to Hong Kong universities outlines the proposed framework for the next iteration of the RAE (Research Assessment Exercise) in 2026, a similarly cyclical exercise based on the REF.
Both stand out as the early adopters of funding-tied research impact assessment; although, as early as 2012, a subset of Australian universities ran a trial for the assessment of impact, with the country later adopting the EIA in 2018.
The REF pioneered the impact component within a full research assessment in its 2014 iteration, where impact determined 20% of the total score; this then grew to 25% for 2021 (arguably counting for more than 29% when factoring in impact areas within the environment section) and is currently scheduled to remain at 25% in 2028 as part of an updated “impact and engagement” section.
The RAE followed the REF’s example to introduce impact in 2020, where it accounted for 15% of the total score, and is now set to grow to 20% by 2026.
The State of Global Research Impact Assessment
Moving further into the Pacific region, while Australia has cancelled its upcoming ERA and EI (Engagement & Impact) Assessments pending reviews, New Zealand is expected to introduce an impact element into its PBRF (Performance Based Research Funding), with the details set to be published in a report this month.
And elsewhere around the world, there is increasing movement towards some form of impact evaluation – such as in Poland, where the REF was again used as a model, in Japan, where the REF has been considered as a starting point for a national evaluation model for social impact, or in Peru, where the REF has been analysed and considered within a Peruvian context. Other different approaches include the Norwegian Humeval, which does not influence funding.
What can we learn from these developments?
-
The REF remains the archetypal model for impact evaluation
The REF has not escaped its criticisms, but it remains the most closely-replicated model for incorporating a systematic evaluation of various types of impact, supported by narrative, evidence and peer reviews, and benchmarking these against international and local standards. In several instances, this approach plays a direct role in influencing the amount of funding available to institutions, creating incentives to adopt best practice for capturing impact data, evidence and narratives.
However, other systems such as the Dutch SEP (Strategic Evaluation Protocol) 2021-2027 focus on a summative rather than a formative approach to impact evaluation. While the REF’s model has been deployed in various guises worldwide already, it is possible that other locations may be less keen to tie impact directly to funding, and may pursue a more formative route.
-
Impact evaluation models see increasing international adoption, but also continue to change
The REF has now entered its third iteration with impact as an evaluation and each cycle it is revised based on wide-ranging feedback, including from those involved in the submission process as well as an International Advisory Group (IAG). During this time it has seen the importance of impact grow from 20% to 25% – alongside many other tweaks. These include an elimination of the minimum 2* quality of underpinning research for impact case studies, which results from sector feedback that gathering evidence for this quality is burdensome amongst other issues. For our advice on best practice in gathering impact evidence more widely, download our handy PDF guide.
In the proposed framework for RAE 2026, UGC suggests upping the contribution of impact to each submission from 15% to 20%, while also considering to allow each university to submit a 3-minute video as a supplement describing the pathway and impact story as a result of feedback.
The definition of ‘reach’ is provided with a focus on more clarity around the diversity of beneficiaries of impact rather than purely geographic extent. With this clarity, academics will no longer need to worry about whether their research impact is too focused on local communities or regions, and be less concerned with making a conscious effort to explore global impact in order to score highly, ultimately reducing the burden on academics and research.
-
Impact Case Studies (ICS) are the international standard for evaluation
Impact can be difficult to capture and evaluate, but the predominant format for demonstration and assessment remains the Impact Case Study. Thousands of case studies are uploaded publicly on databases for locations such as Hong Kong, Poland and the UK.
In the recent announcements however, Hong Kong and the UK have taken slightly divergent approaches, with REF 2028 set to include a “structured explanatory statement” that could be worth up to 50% of the total impact submission, while RAE 2026 disposes of a supporting impact statement with the intention to instead cover this in the environment statement.
-
Impact is here to stay – whether or not it plays a role in funding
As we have seen, there are different models for impact assessment. As more and more countries introduce some form of evaluation for universities, we are likely to see further variations; some will be tied to funding and others not. Furthermore, impact rankings continue to grow, with a record 1,591 universities featured in the 2023 edition of the Times Higher Education (THE) Impact Rankings.
Regardless, setting universities up for impact capture and demonstration can be a lengthy process and require cultural change, so it is important to be prepared – and we know starting to capture information early can significantly enhance end-results, whether for benchmarking or funding purposes.
Where Do We Go From Here?
Impact assessment continues to grow in priority for many governments and therefore also for universities around the world.
Get in touch if you want to learn more about how you can set your institution up for impact success!