OPUS Deliverable 3.3 Intermediate Evaluation of Metrics Tested in the Pilots

The OPUS project is developing coordination and support measures for organisations to reform research assessment with a focus on recognising and rewarding researchers for Open Science. The main aim of the project is to develop a framework for researcher assessment, consisting of indicators/ metrics with a focus on Open Science and corresponding interventions for organisations to implement the framework, which will be tested in pilots and refined in consultation with the research community. This report is deliverable D3.3 of the OPUS project Intermediate Evaluation of Metrics Tested in the Pilots. The report builds on D3.1 on Indicators and Metrics to Test in the Pilots, which proposed a first draft of a researcher assessment framework (RAF) to assess researchers and recognise and reward Open Science in researcher assessment. The report also builds on D3.2 of the OPUS project on Baseline Audit of Metrics to Test in the Pilots which presented the indicators/metrics selected by each pilot organisation in the OPUS project and their corresponding baseline and targets for the pilots. This report is closely aligned with D2.3 Intermediate Evaluation of Interventions Tested in the Pilots which outlines progress towards meeting targets set on interventions which should be read in before this deliverable. That in turn builds on D2.1 on Interventions to Test in the Pilots which proposed a first draft series of interventions to support implementation of a researcher assessment framework (RAF) at research-performing organisations (RPOs) and research-funding organisations (RFOs) and D2.2 on Baseline Audit of Interventions to Test in the Pilots. The report has been developed in collaboration with the organisations running pilots in the OPUS project as listed in Table 1.1. The three RPOs (UNL, UCY, and UNIRI) and two RFOs (UEFISCDI and RCL) have selected and tailored indicators/metrics from D3.1 and corresponding interventions from D2.1 to test in each of their pilots. The pilot period in the OPUS project runs for 18 months (M) from January 2024 until June 2025. Pilot progress is being monitored and this report represents the interim evaluation covering M0-M9. A final evaluation will take place at M18 of the pilots. Pilot # Organisation Acronym Country Pilot #1 Nova University of Lisbon UNL Portugal Pilot #2 University of Cyprus UCY Cyprus Pilot #3 University of Rijeka UNIRI Croatia Pilot #4 Executive Agency for Higher Education, Research, Development, and Innovation Funding UEFISCDI Romania Pilot #5 Research Council of Lithuania RCL Lithuania Table 1.1: Pilot Organisations in the OPUS ProjectThe report structure is based on the pilots and begins in Section 2 with a description of the methodology used to carry out the co-monitoring with pilots to track progress towards meeting targets for interventions and indicators/metrics. The report next presents each pilot, together with their baseline and targets for their selected indicators/metrics at M9 and M18, and actual progress at M9, followed by a narrative progress update covering M0-9 (UNL in Section 3, UCY in Section 4, UNIRI in in Section 5, UEFISCDI in Section 6, and RCL in Section 7). Lastly, the report offers some conclusions and proposes next steps for the OPUS project in Section 8. It should be noted that the report does not go into detail on the implementation of selected interventions and indicators/metrics in the pilots as this will be presented in action plans describing the pilots by each of the pilot organisations. Full details can be found in the deliverable D4.1: Action Plans to Implement the Pilots - Initial and deliverable D4.3 Action Plans to Implement the Pilots – Final.