Artificial Intelligence in Peer Review: A Bibliometric-Guided Thematic Review and a Task-Contingent Legitimacy Framework

The rapid adoption of large language models (LLMs) has prompted extensive debate about their appropriate role in peer review, scholarly publishing’s primary quality-control mechanism. However, AI has not yet been formally approved as a peer-review tool by most academic journals. This study reviews the emerging AI-in-peer-review literature to identify research trends, synthesize empirical evidence across review tasks, and develop a conceptual framework for AI-assisted review. Using a PRISMA-guided Scopus search (176 records identified, 162 included), we combined three-layer content analysis (theme, editorial stance, and AI autonomy) with a synthesis of 18 empirical studies. The literature expanded from 6 records before 2023 to 45 records in the first half of 2026 and remains dominated by commentary and opinion (57%), with the remaining 43% comprising research studies, technical work, and reviews. Editorial perspectives are generally balanced, and authors overwhelmingly favor assistive, human-in-the-loop AI over human-only or full automation. Empirical evidence shows a task-contingent pattern: AI performs well on narrowly defined evaluative tasks (Pearson r > 0.9 in some settings) but less reliably when predicting editorial decisions (accuracy 40–67%; correlations as low as ρ = 0.00). AI legitimacy may depend more on task type than on any governance position, a pattern we formalize in a Task-Contingent Legitimacy framework offering a task-tiered policy approach and testable propositions.

Authors

Institutions

Publication Details

Journal
Publications
Published
2026-09-24
DOI
https://doi.org/10.3390/publications14040063
Primary Topic
scientometrics and bibliometrics research
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
article

Artificial Intelligence in Peer Review: A Bibliometric-Guided Thematic Review and a Task-Contingent Legitimacy Framework

Eungi Kim, Vaishali Singh
Publications
scientometrics and bibliometrics research
article

Artificial Intelligence in Peer Review: A Bibliometric-Guided Thematic Review and a Task-Contingent Legitimacy Framework

Eungi Kim, Vaishali Singh
article en

Abstract

The rapid adoption of large language models (LLMs) has prompted extensive debate about their appropriate role in peer review, scholarly publishing’s primary quality-control mechanism. However, AI has not yet been formally approved as a peer-review tool by most academic journals. This study reviews the emerging AI-in-peer-review literature to identify research trends, synthesize empirical evidence across review tasks, and develop a conceptual framework for AI-assisted review. Using a PRISMA-guided Scopus search (176 records identified, 162 included), we combined three-layer content analysis (theme, editorial stance, and AI autonomy) with a synthesis of 18 empirical studies. The literature expanded from 6 records before 2023 to 45 records in the first half of 2026 and remains dominated by commentary and opinion (57%), with the remaining 43% comprising research studies, technical work, and reviews. Editorial perspectives are generally balanced, and authors overwhelmingly favor assistive, human-in-the-loop AI over human-only or full automation. Empirical evidence shows a task-contingent pattern: AI performs well on narrowly defined evaluative tasks (Pearson r > 0.9 in some settings) but less reliably when predicting editorial decisions (accuracy 40–67%; correlations as low as ρ = 0.00). AI legitimacy may depend more on task type than on any governance position, a pattern we formalize in a Task-Contingent Legitimacy framework offering a task-tiered policy approach and testable propositions.

PublicationsVol. 14(4)
Keimyung University (KR)
Quality Education
Openalex Percentile: Top 9%
scientometrics and bibliometrics research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.