Peer review research assessment: are the reviewers really experts?

Abstract Peer review is a cornerstone of research evaluation, yet its credibility depends critically on the expertise of those conducting it, a condition too often assumed rather than verified. This study investigates the actual expertise of reviewers involved in Italy’s national research assessment exercise (VQR 2020(VQR-2024)). We analyze the disciplinary alignment between reviewers and submissions, as well as reviewers’ scientific performance, measured via a normalized ten-year citation impact indicator. Results reveal considerable variability in reviewer expertise and workload across fields, with some disciplinary sectors markedly underserved. Nearly half of the reviewers belong to the top 30% of national research performance, but a significant minority score below the median. Reviewers appointed by the evaluation agency (ANVUR) tend to outperform those selected by lottery, though both groups include cases of inadequate expertise. These findings challenge the assumption that peer reviewers in national evaluations are consistently qualified and call for more rigorous, transparent, and performance-informed selection processes to safeguard the integrity and credibility of peer review at scale.