Unveiling the Reviewer Mindset: Evidence-Based Insights into Scholarly Peer Review

<p xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" class="first" dir="auto" id="d779380e79"> <b>Background and Aim:</b> <p xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" dir="auto" id="d779380e84">Peer review remains the cornerstone of scholarly publishing, yet a systematic understanding of reviewer priorities and patterns across disciplines remains limited. The aim of the study was to quantitatively reveal the primary concerns and decision-making criteria that shape peer review outcomes, moving beyond anecdotal perceptions to data-driven insights. <p xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" dir="auto" id="d779380e86"> <b>Methods:</b> <p xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" dir="auto" id="d779380e91">This study presents a novel large-scale meta-analysis of peer review comments from over 5,229 manuscript evaluations, addressing a critical gap in our understanding of scholarly assessment practices. Using advanced natural language processing techniques and large language models, we systematically extracted, categorized, and analysed reviewer comments to identify predominant concerns and evaluation patterns. Our methodology combined computational text analysis with structural classification to create a comprehensive classification of reviewer priorities. <p xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" dir="auto" id="d779380e93"> <b>Results:</b> <p xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" dir="auto" id="d779380e98">Methodological concerns (22.4%) consistently emerged as the primary focus of reviewer attention, with particular emphasis on research design justification, methodological rigor, and data validity, which have been grouped together. Presentation issues and literature integration (15.7%) formed secondary but substantial categories of reviewer feedback. Our analysis uncovered distinct patterns in how reviewers frame their evaluations, with notable differences between purely critical assessments and those offering constructive pathways for improvement. <p xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" dir="auto" id="d779380e100"> <b>Conclusion:</b> <p xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" dir="auto" id="d779380e105">This research makes a significant contribution to scholarly publishing by providing empirical evidence of review patterns that have previously been understood primarily through anecdotal experience. The findings offer valuable, evidence-based insights for authors preparing manuscripts, editors managing the review process, and the broader academic community working to enhance transparency in scholarly evaluation.