Assessing the quality of AI-assisted peer reviews: A comparative assessment of ChatGPT, Claude Sonnet and Google Gemini in reviewing scientific articles

Objectives: Peer review validates research findings and upholds the quality and credibility of scientific publications. This study aimed to assess the quality of peer-review reports generated by GPT-5.6, Claude Sonnet-4.6, and Google Gemini-3.1 Pro, while evaluating the same set of published scientific articles. Methodology: This comparative study was performed in the “Department of Physiology, College of Medicine, King Saud University, Riyadh, Saudi Arabia” from August 23, 2026, to September 10, 2026. In this study, 30 original published articles were identified, and each article was submitted to GPT-5.6 (n=30), Claude Sonnet-4.6 (n=30), and Google Gemini-3.1 Pro (n=30). Each AI model generated 30 individual review reports, for a total of 90 peer-review reports. A panel of five faculty members evaluated the peer-review reports, and each report was scored out of five marks. Each examiner evaluated peer-review reports generated by ChatGPT (30), Claude Sonnet (30), and Google Gemini (30). The frequencies, percentages, mean ± SEM and 95% CIs were calculated, and p-values < 0.05 were considered statistically significant.Results: AI models generated 90 peer-review reports, 30 each from ChatGPT, Claude Sonnet, and Google Gemini. Across the five examiners, ChatGPT received an overall grade of 634.1/750 (84.54%), Claude Sonnet received 690.4/750 (92.05%), and Google Gemini received 465.3/750 (62.04%). Claude’s Sonnet review reports received the highest grades, followed by ChatGPT and Google Gemini (p=0.001). Conclusions: The AI model, Claude Sonnet-4.6, produces significantly better review reports than GPT-5.6 and Google Gemini-3.1 Pro. These AI models summarize complex information and identify potential gaps, methodological concerns, and issues in the results and discussion sections of medical science articles. Integrating AI models within professional boundaries supports academic journals’ editorial workflows rather than replacing human peer review.

Authors

Institutions

Publication Details

Journal
Pakistan Journal of Medical Sciences
Published
2026-11-19
DOI
https://doi.org/10.12669/pjms.42.12.21007
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Assessing the quality of AI-assisted peer reviews: A comparative assessment of ChatGPT, Claude Sonnet and Google Gemini in reviewing scientific articles

Muhammad Zain Sultan Meo, Riham A. ElToukhy, Sultan Ayoub Meo, Yasir Waheed et al.
Pakistan Journal of Medical Sciences
Artificial Intelligence in Healthcare and Education
article

Assessing the quality of AI-assisted peer reviews: A comparative assessment of ChatGPT, Claude Sonnet and Google Gemini in reviewing scientific articles

Muhammad Zain Sultan Meo, Riham A. ElToukhy, Sultan Ayoub Meo, Yasir Waheed, David Charles Klonoff, Naseer Ahmed
article en

Abstract

Objectives: Peer review validates research findings and upholds the quality and credibility of scientific publications. This study aimed to assess the quality of peer-review reports generated by GPT-5.6, Claude Sonnet-4.6, and Google Gemini-3.1 Pro, while evaluating the same set of published scientific articles. Methodology: This comparative study was performed in the “Department of Physiology, College of Medicine, King Saud University, Riyadh, Saudi Arabia” from August 23, 2026, to September 10, 2026. In this study, 30 original published articles were identified, and each article was submitted to GPT-5.6 (n=30), Claude Sonnet-4.6 (n=30), and Google Gemini-3.1 Pro (n=30). Each AI model generated 30 individual review reports, for a total of 90 peer-review reports. A panel of five faculty members evaluated the peer-review reports, and each report was scored out of five marks. Each examiner evaluated peer-review reports generated by ChatGPT (30), Claude Sonnet (30), and Google Gemini (30). The frequencies, percentages, mean ± SEM and 95% CIs were calculated, and p-values < 0.05 were considered statistically significant.Results: AI models generated 90 peer-review reports, 30 each from ChatGPT, Claude Sonnet, and Google Gemini. Across the five examiners, ChatGPT received an overall grade of 634.1/750 (84.54%), Claude Sonnet received 690.4/750 (92.05%), and Google Gemini received 465.3/750 (62.04%). Claude’s Sonnet review reports received the highest grades, followed by ChatGPT and Google Gemini (p=0.001). Conclusions: The AI model, Claude Sonnet-4.6, produces significantly better review reports than GPT-5.6 and Google Gemini-3.1 Pro. These AI models summarize complex information and identify potential gaps, methodological concerns, and issues in the results and discussion sections of medical science articles. Integrating AI models within professional boundaries supports academic journals’ editorial workflows rather than replacing human peer review.

Pakistan Journal of Medical SciencesVol. 42(12)
Sutter Health (US), Alfaisal University (SA), University of California, San Francisco (US), King Saud University (SA), Mills Peninsula Health Services (US), Rehman Medical Institute (PK)
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon