The Future of Research Evaluation: Navigating the REF and the Impact of AI

Author: Nick Hillman
Published: 7 September 2025

This blog has been kindly written for HEPI by Richard Watermeyer, Tom Crick, and Lawrie Phipps.

Rethinking Research Assessment: The Role of AI in REF 2029

Author: Nick Hillman
Published: 7 September 2025

This thought-provoking blog post comes from Richard Watermeyer, Tom Crick, and Lawrie Phipps, experts in the field of higher education. On September 5, HEPI and Cambridge University Press & Assessment will host the UK launch of the OECD’s Education at a Glance. Following that, on September 6, a webinar discussing students’ cost of living will take place, where interested individuals can register for free.

The landscape of research evaluation has evolved significantly, especially with the introduction of national assessments like REF (Research Excellence Framework) and RAE (Research Assessment Exercise). An essential pivot occurred with REF2014, marking the inclusion of “impact” as a measure of research excellence. As we look ahead to REF2029, there is a new focal point on “research culture.” However, its integration into the assessment criteria remains uncertain.

The Current State of REF

Recently, Sir Patrick Vallance, the UK Government’s Minister for Science, announced a temporary pause in the REF 2029 progress. This decision aims to ensure a credible assessment of research quality. The original proposed formula has come under criticism, suggesting it is not equipped to fulfill the government’s objectives regarding economic and social missions. Importantly, the concern arises that research culture may be sidelined or excluded from the REF entirely.

Some may view this potential rollback on research culture as a relief—an escape from an increasingly complex and costly accountability regime that aims to define what constitutes “excellent” research. Despite promises of simplification in each iteration of REF, reality has often seen the opposite.

Wider Implications of REF’s Hiatus

The implications extend beyond the potential omission of research culture. As financial difficulties plague the UK higher education sector, the substantial costs associated with the REF come under scrutiny. Our research indicates that the REF might not require just minor adjustments, but rather substantial revisions due to the rise of artificial intelligence in higher education.

With funding from Research England, we have consulted various research leaders across 17 UK higher education institutions. Responses varied on the efficacy of generative AI tools for REF, yet a consensus emerged: the inevitable integration of AI into REF processes. Applications for AI range from narrative generation and evidence reconnaissance to scoring research outputs and impact case studies, potentially revolutionizing how research quality is assessed. The vision includes real-time evaluations rather than adhering to rigid seven-year cycles.

The Challenges of Generative AI

However, the adoption of generative AI tools is fraught with challenges. Concerns surrounding bias, accuracy, and the potential for manipulation introduce significant risks. These tools are labeled "black boxes," capable of obscuring transparency and reproducibility, which are essential to responsible research evaluation as outlined by organizations like CoARA and COPE.

Despite these objections, generative AI is already being utilized trepidatiously by academics and support staff preparing for REF. As AI technology advances rapidly, it seems increasingly likely that REF panels will adopt these tools in the coming years. If the integrity of REF relies on aligning with government missions—especially regarding research and development for economic growth—current advancements in AI may play a crucial role.

Moving Forward

Arguments that the REF represents good value for public money must be re-evaluated in light of financial hardships within the sector and the growing allure of AI solutions. A recalibration of the REF’s components, processes, and responses is essential; it cannot be entirely outsourced to dominant tech vendors. The development of a guidebook for generative AI use in the REF is crucial, ensuring consistent practices are implemented.

The intersection of generative AI and the REF poses both opportunities and challenges. A critical question remains: Is three months sufficient to navigate these complexities? As we inch toward the definitive results of the REF-AI study, due in January 2026, the future of research evaluation hangs in a delicate balance.

Notes:
The REF-AI study is a collaborative effort between the universities of Bristol and Swansea and Jisc, with special thanks to Professor Huw Morris from UCL IoE for his input in earlier drafts of this article.