Mind the Gap: From Policy to Practice at CNR. An Analysis of Career Advancement Criteria in Light of the Agreement on Reforming Research Assessment

This study examines the practical implementation of the Agreement on Reforming Research Assessment within the National Research Council of Italy by analysing the 2023 competitive procedures for the career progression of researchers and technologists. As the institution formally adopted the reform to shift towards a more responsible and qualitative evaluation framework, this research investigates the extent to which these high-level commitments were applied within the calls by the individual evaluation committees. The authors conducted a systematic analysis of 4,090 evaluation criteria established by 90 different committees to assess alignment with the Agreement’s four core commitments and the inclusion of Open Science principles. Results indicate a significant gap between institutional policy and actual practice: most committees do not fully recognise the diversity of research contributions, often undervaluing or excluding non-traditional outputs, such as teaching and evaluation activities, or public and policy engagement. Quantitative and mechanical metrics continue to be widely used in researcher evaluations, with journal-based indicators often used to assess research quality. Regarding Open Science practices, they appeared only sporadically in the assessed criteria. These findings suggest that traditional evaluation practices persist at the committee level, despite formal institutional reform. The study highlights that high-level policy adoption alone is insufficient without robust guidance to operationalise reform principles during the assessment process. This research provides insights into the challenges of transitioning from policy to practice in research assessment within a large, multidisciplinary Research Performing Organisation.