Design flaws in the Polish science quality evaluation system: A technical analysis of latent properties

• In Poland, the science quality evaluation is susceptible to the optimization of research outcomes reporting. • The evaluation results are affected by institution size, disadvantaging larger ones. • The scientific category allocation is susceptible to manipulation by evaluators. • Allocation of scientific categories is linked to conflicts of interest. • We propose minor legal amendments to address these issues entirely. In 2018, Poland’s science and higher education underwent reforms introducing a new scientometrics system to periodically evaluate research quality in about 300 institutions. This system primarily relies on parametric evaluations, with limited expert input. Evaluation outcomes affect institutions’ government funding, as well as their authority to grant academic degrees and offer study fields. Initially applied in 2022 for the 2017–2021 period, the system remains largely unchanged for the 2026 evaluation of 2022–2025, sustained more by political instability than by trust in its efficacy. This paper examines the Polish science evaluation system through the lenses of statistics, decision-making, and operations research. We reveal four design flaws causing counterintuitive results, enabling institutions to boost scores by even 60% through creative reporting, discriminating based on size, permitting result manipulation by parties with conflicting interests, and deviating markedly from current decision science standards. Our theoretical insights are validated by data from the 2022 and 2026 evaluations. Recommendations for system improvements are also presented.