Beyond rankings: a critical examination of bibliometric indicators in national research assessment

Abstract Since 2022, China has achieved the global top position in the number of papers published in Nature Index journals, which has sparked assertions that China is challenging the scientific dominance of the United States. This study critically examines 27 bibliometric indicators used for national research assessment by comparing China with 19 other most productive countries in Nature Index journals. The findings indicate that different metrics yield diverse results, with China’s rankings ranging from first to 20th across the 27 indicators. On one hand, this highlights the need for cautious use of rankings and the importance of looking beyond mere rankings to understand in which aspects China has surpassed the USA. On the other hand, it underscores the necessity for a pluralistic approach to research evaluation, incorporating a variety of indicators to minimize potential biases and achieve a more comprehensive understanding of China’s research performance. Notably, metrics designed to capture quality, such as Novelty and Disruption, show limited discriminative power at the national level, as articles with high values for these indicators appear to be randomly distributed across countries, showing no systematic pattern that aligns with other performance indicators.