Supporting inclusive research assessment: the CoARA WG TIER perspectives

Current initiatives at the European and international level have emphasised the opportunity of shifting to more responsible research assessment, such as those related to the San Francisco Declaration on Research Assessment (DORA) since 2012 and the more recent efforts in the Coalition for Advancing Research Assessment  (CoARA) since 2022, to introduce improvements that address the diversity of research products and careers.

In the Coalition, the thematic Working Group Towards an Inclusive Evaluation of Research (TIER) aims at: advancing gender equality, intersectionality and diversity through better research assessment; ensuring inclusive and bias-mitigated processes in the evaluation of research quality; and developing training programs for institutions and evaluators. CoARA WG TIER is coordinated by the University of Milano-Bicocca, Italy; it counts approximately 80 members from around 35 EU and non-EU countries.  

In particular, the National Institute of Geophysics and Volcanology of Italy (INGV) joined CoARA to reinforce the value of the diversity of products, results, and practices that constitute the geoscientific research carried out in the Institute. By adhering to the principles of the Agreement of Reforming Research Assessment (ARRA), INGV supports a global reform of the scientific endeavour that moves from purely competitive to a collaborative and more inclusive one. INGV joined TIER as well, to benefit from shared data and best practices to mitigate gender bias in evaluation processes, and more generally, to improve the implementation of the principles of diversity and inclusion in its policies.

To give an example, in research assessment, systematic biases may occur when evaluation criteria do not account for the career paths of individuals.  For instance, maternity or health leaves are often not considered in the quantitative evaluation of scientific production; accidental confirmation biases may arise when evaluators lack sufficient information about the candidates or valorise stereotyped attitudes. The thematic working group TIER is committed to identifying the causes of such biases and to formulating actionable guidelines to counteract them. 

So far, TIER: has worked on five case-studies in Europe and UK to collect current statistics on gender distribution in science and more than 30 best practices already adopted both in Research Performing Organizations and Research Funding Organizations; collected a set of references for communication and training on gender bias, including videos and on line materials; realized a survey to individuals to test the self-perception of bias in evaluation, and is preparing another survey to institutions. A dissemination tour is in preparation in different countries, with the support of the CoARA National Chapters, in order to disseminate recommendations for new and innovative strategies to identify and overcome barriers in research assessment. INGV is going to implement TIER suggestions as foreseen in its Action Plan, and to support TIER dissemination tour via the co-chaired Italian National Chapter for the next two years.