Enhancing Consistency in Peer Review: A Statistical Analysis of Discrepancies and Proposals for Improvement

ABSTRACT This paper investigates the inconsistencies present in peer review by analysing the evaluation patterns of reviewers involved in an educational award in the Arab Gulf Country States. A statistical approach was used to assess the degree of variation in scores assigned to 270 manuscripts reviewed by three different groups of reviewers. The study revealed significant differences in the evaluations, suggesting that at least two reviewers often showed discrepancies in their assessments despite using the standardised evaluation form. The observed discrepancies appear to reflect underlying complexities related to reviewer perspectives and evaluation standards, highlighting challenges in achieving uniformity across assessments. Additionally, it proposes a model to enhance peer review consistency, including methods for score adjustment and calibration to mitigate reviewer differences. The goal is to offer practical recommendations for improving the fairness, transparency and reliability of peer review systems, contributing to the ongoing development of academic publishing practices. For audiences beyond the academic community including publishers, editors and academic librarians, these findings show how practical statistical tools can strengthen peer review and build greater trust in academic publishing.