End of time-consuming Australian research assessment exercise causes both relief and concern
The “halt” in the Excellence in Research for Australia rankings has left researchers wondering how their work will be assessed.
The next round of Era, which ranks Australian university research and is carried out by the Australian Research Council, was cancelled after a directive last month from education minister Jason Clare.
Although some researchers and administrators expressed relief at the halt, work already put into the Era process has now been rendered redundant. The ARC had just completed a review of Era guidelines ahead of the 2023 round, with extensive industry input.
Catharine Coleborne, president of the Australasian Council of Deans of Arts, Social Sciences and Humanities, told Research Professional News that the group “welcomes the opportunity to scrutinise the efficacy of the Era model to assess research quality in Australian universities”.
She said that the deans’ “recommendation to the recent review of Era suggested that the round [should] trial a citation index alongside peer review to look at a finer calibration of [humanities and social sciences] research”.
“We note that reactions to minister Clare’s announcement in our community include frustration at the time already spent on the exercise by academic and professional staff inside universities,” she added, saying that the deans were looking forward “to being part of the discussions with the ARC and to representing our leadership body at the table as Era evolves”.
A statement from the Australian Council of Deans of Science said that the council “warmly welcomes the planned review of the ARC Era exercise”.
“Whilst this programme has been a valuable means for demonstrating the excellence of research across universities in Australia, the increasing focus on research translation and the availability of a broader range of research metrics calls for a fresh approach. We look forward to making positive contributions to the review that enable our excellence in both discovery and translational research to be recognised nationally and internationally, and for this to leverage a level of investment that aligns with other OECD nations.”
Ministerial expectations
In a letter of expectations sent to ARC chief Judi Zielke on 26 August, Clare said that “in light of the sector’s concerns about workload, I ask that you discontinue preparations for the 2023 Era round and commence work to develop a transition plan”. The transition plan, created in conjunction with the Department of Education and the research sector, should lead to a “a modern, data-driven approach informed by expert review”. Clare has asked to receive the plan by the end of 2022.
He said he was aware of the workload issues and that it was “important that the Era process continues to be valued as an important source of information, in the context of the existing available evidence of the quality of Australian research”.
Any new process should be ready to be implemented by 2024-25, Clare’s letter said.
Time-consuming process
The Group of Eight research-intensive universities welcomed the halt, as well as a review of the legislation underpinning the ARC. It is “timely to reassess the structure, functions and strategic objectives of Era and develop a modern, data-driven approach, informed by peer review”, a Go8 statement said.
Sue Bennett, a senior professor at the University of Wollongong and a current assessor for ARC grants, published an article on news website The Conversation saying that the halt provided “immediate relief” from administrative work, freeing Covid-exhausted researchers up for other research activity.
She told Research Professional News that the halt was a surprise to researchers, in addition to the already-flagged review of the ARC Act. Their reaction was mixed, with relief that the pressure was off but also concern about work already done and what would come next.
However, she said the decision recognised how much work Era had become. Researchers not only need to report their own work but also work in teams across universities to share understanding of the Era process, she said. “It takes a lot of people a lot of time, and a lot of systems time, and a lot of checking.”
The Era process typically takes more than a year, and her own university was already well advanced. They are now looking at what they have done to see what they can learn from it, and they hope to be able to use that in whatever replaces Era.
The sector is now focused on what that replacement might be, Bennett said, adding that researchers accepted that national measuring and benchmarking was important, not least to identify research “strengths and opportunities”.
Bennett believes the review of the ARC Act should also address performance on issues including gender equity and Indigenous research.
The research council’s strategic plan, which was made public at the end of August, included a goal of implementing “outcomes of the review of Era and Engagement and Impact by transitioning the assessments to a more modern, data-driven approach informed by expert review”, but it gave no details on how that would be achieved.
The Engagement and Impact assessment process is currently unaffected by the halt.
A version of this article also appeared in Research Europe