Have university faculty themselves abolished the “Five-Only” evaluation criteria

In the reform framework of “Breaking the Five-Only”, peer review is highly expected to act as the “academic gatekeeper”. University faculty members are not only evaluatees but also evaluators; whether their evaluation practices can transcend the entrenched path dependence on the “Five-Only” paradigm is pivotal to the reform’s success. However, the academic community currently lacks quantitative empirical research on this issue. Based on textual materials from the letter review for faculty professional title promotion (2020-2024) at a 985 Project university, this study applied the BERT model for multi-label classification to explore the evolutionary trends of the overall characteristics of university faculty evaluation in the context of the “Breaking the Five-Only” reform. The results indicate that the evaluation practices in leading research universities exhibit a reform trajectory characterized by “research-led, multidimensional, and gradual progress”. Nevertheless, a structural transformation of the evaluation system has not yet materialized, which suggests that the “Breaking the Five-Only” reform still requires further deepening and advancement. This paper develops a big data analysis model based on review texts, which can better capture the authentic perspectives of review experts. It provides a methodological reference for dynamically monitoring the reform’s effectiveness and offers important implications for optimizing the design of the professional title review system and promoting the transformation of the evaluation paradigm in higher education.