A Chat About Peer Review

In February 2025, we received the peer reviewer comments for our invited review of uveitis therapeutics, a major (25-published page) article that had taken our team of seven clinicians almost a year to write. We were enthusiastic about the work and looked forward to tough constructive criticism to make the article even better. Reading through the comments, we initially wondered whether the wrong critique had been sent: ‘The methods section is meticulously detailed, allowing for reproducibility and transparency’ and ‘The results are presented systematically, with precise graphical representations’; yet, our work was a narrative review, and there were no methods and results sections, and further, the article had multiple large tables but no figures. In addition, none of the suggested revisions were specific to uveitis treatment. We concluded that the peer review was generated by artificial intelligence (AI), specifically by a large language model (LLM). Publicly available Quillbot AI Detector software [1] confirmed our suspicion: 100% of the review was ‘AI-generated’ and 0% was ‘human-written’ (accessed 26/2/2025). Artificial intelligence is embedded in our professional and personal lives, whether we are aware or not. Clinical and Experimental Ophthalmology has featured a number of current applications of AI in ophthalmology specifically: diagnosis and management of common and rare eye diseases [2-4], ophthalmic screening [5, 6] and administrative or regulatory processes [7, 8]. To imagine a future in which AI is not part of generating and disseminating knowledge is to bury one's head in the sand. On that basis, it is not unexpected that AI is being used in the preparation of manuscripts for publication. The peer-reviewed journals have rules around this: in the case of Clinical and Experimental Ophthalmology, ‘Manuscripts that have been prepared by or with the assistance of [AI] must transparently declare this and describe in detail both the extent of the assistance as well as the system specifications used in the description of the methodology’ [9]. In contrast, the use of LLM in peer review of manuscripts has received much less attention. The listed authors wrote this editorial, but in the process, we sought LLM input on the topic at hand. We asked 3 chatbots—ChatGPT [10], Google Gemini [11] and Microsoft Copilot [12] (accessed 18/8/2025)—the following question: ‘What are the advantages and disadvantages of using AI tools for scientific peer review?’ The three chatbots provided lists of advantages and disadvantages, which summarised some of the key issues. All noted the positives of speed, objectivity (reducing reviewer bias, long recognised to be an issue), integrity checking, and language support. Two chatbots also alluded to reviewer intellectual support, one of the most controversial aspects of using LLMs, as discussed below. In terms of negatives, all chatbots identified lack of specialist knowledge (the primary expectation of a peer reviewer), biases related to training, and the ‘black box’ issue of the unknowability of how an LLM derives its conclusion. One chatbot mentioned ‘hallucination’, better termed fabrication, and two noted important ethical and privacy concerns in sharing unpublished information with public software tools without author or journal permission. There are strong opinions in the sector regarding the use of LLM in peer review of manuscripts. In their Nature Career Column article, evolutionary biologist Carl Bergstrom and collective-behaviour scientist Joseph Bak-Coleman mount an impassioned counterargument, concerned about the erosion of human scientific activity and de-skilling of scientists. They write, ‘We risk replacing human proficiency with inferior automation and not recognizing what we are losing’. They fear LLMs ‘eroding’ ‘the depth and complexity in the human activity of science’ [13]. More temperate in his Nature editorial, biomedical data scientist James Zou refers to a tidal wave of LLM use in academic writing and peer review [14]. Work by his team generated a chatbot that provided scientific peer review, compared the reviews to those provided by humans, and showed that more than 50% of authors found the LLM-sourced reviews to have scientific value [15]. He hopes to see ‘community norms and resources … to ensure that LLMs benefit reviewers, editors, and authors without compromising the integrity of the scientific process’ [14]. The perception of LLMs having expertise in the peer review process is an illusion. Indeed, LLM output does not discriminate fact from fiction [16]. For clinician and non-clinician researchers in the ophthalmology space, we must acknowledge that we are not experts in the application of AI and proceed with caution. Wiley, the publisher of Clinical and Experimental Ophthalmology, provides best practice guidelines for using AI in peer review [9]. To quote abridged from this material: ‘AI technology can be used to improve the quality of the written feedback in a peer review report. This use must be transparently declared upon submission of the peer review report. Peer reviewers should not upload manuscripts into AI Technology’. As Wiley stresses, loading an author's manuscript into a public chatbot is a breach of confidentiality and privacy, and further may breach copyright, both of which have potential legal implications. At the least, it is likely that the journal will take punitive action against peer reviewers who are caught breaching this policy. From a philosophical perspective, reviewing provides the privilege to be first to read, and even contribute to, impactful research; employing an LLM to review a manuscript closes the door on this special opportunity. Furthermore, material submitted to a chatbot goes into training the LLM—if that material is fatally flawed, or otherwise in need of revision, there are worrisome implications for scientific knowledge. Clearly the peer reviewer faces diverse opinions on a range of issues; we suggest stepping into the author's shoes. From the author's perspective, an LLM-generated review that typically is generic and largely irrelevant, provides no constructive information to support subject-specific improvements of their work. Across medicine, including in ophthalmology, workloads continue to increase. Taking on the review of a manuscript adds to one's workload. Yet, it is part of the work of a researcher to critique; further it contributes to our continuing professional development. As the ophthalmic literature grows almost exponentially, review requests are increasing, not decreasing. So how can a reviewer use AI technology to their advantage, whilst also complying with the needs and policies of peer review? A Nature Career Column article by Dritjon Gruda, an organisational behavioural scientist, is helpful in this regard: Three AI-powered steps to faster, smarter peer review [17]. ‘Scan, dictate, refine’: While he reads the paper, he makes notes using speech-to-text software; he feeds the dictated notes into an offline LLM, to English-check and organise the review; and he polishes the text personally. Thus, the output content is his own, while the LLM is providing editorial support—but always under his supervision. When submitting your review to Clinical and Experimental Ophthalmology, indicate that you have used an LLM to correct language and organise your document. To quote Gruda: ‘By zeroing in on the most important parts of a paper and pairing efficient note-taking with the linguistic power of AI, we can give authors the feedback they need without burning ourselves out’. However, if you simply cannot find the time to complete a peer review, promptly decline the invitation and look forward to receiving another at a more convenient time. Peer-reviewed journals can take actions to guard against LLM-generated peer reviews. To educate their human peer reviewers on the issue, Clinical and Experimental Ophthalmology could place a link on the reviewer interface to the relevant section of Wiley's best practice guidelines. Another easy fix would be a check box on the review submission interface, asking the peer reviewer to indicate whether an LLM was used in the review, and if so, which and how. The journal uses sophisticated tools within Wiley's Research Exchange Screening platform that can screen new submissions for plagiarism and the use of generative AI [18]. The same tools could be implemented to screen peer reviews. Digital watermarks can be inserted to detect the use of an LLM [19], as text-based detection alone can prove challenging [20]. Editors need to take author concerns of AI-generated peer review seriously. When peer reviewers appear to have contravened the policies, an investigation is needed, with consequences that might include a reviewing ban and notification of the individual's academic institution. In any investigation, it is essential to recognise the reviewer's rights of response and due process as no AI-detection tool is perfect. Editors also must support authors whose work has been compromised under their watch. They should direct the authors to ignore an LLM-produced review, and might arrange a rapid re-review, perhaps by one of their editorial board members. It is unfair to subject authors to an extended peer review process. How an editorial office responds to such a situation may impact the attractiveness of their journal for new author submissions. A related recent concern for peer-reviewed journals is the possibility of authors inserting white text notes—requesting a positive review—into their manuscript; these ‘secret messages’ would not be seen by a human peer reviewer without highlighting, but would be recognised and likely influence an LLM [21]. Wiley's newly updated Research Exchange Screening platform would be expected to identify such nefarious suggestions in new manuscript submissions to Clinical and Experimental Ophthalmology. In reading our Editorial, some authors may become quite concerned around the possibility that their manuscripts could be peer-reviewed by an LLM. There are a few tell-tale signs that may help you identify an LLM-generated peer review. The language is often overdramatic or otherwise exaggerated. Even more obviously, the feedback is commonly generalised and not subject-specific, and in some cases, it may be quite irrelevant (such as references to methods and results in a narrative review). There are multiple software platforms, many publicly available, that can be used to help distinguish the work of LLMs from that of humans. We suggest you use several for a more certain assessment. If you have reason to suspect your manuscript has been reviewed by an LLM, you should share the evidence for your conclusion with the editorial team at Clinical and Experimental Ophthalmology, who certainly will take your concern seriously. They have several possible courses of action to progress the evaluation of your paper, but importantly, you should not be required to address an LLM-generated peer review. Artificial intelligence is a fact of life in 2025. Large language models are unreliable and inaccurate, and we are going to use them anyway [22-24]. There is a difference between a tool being widely available and easy to use, and one having the accuracy and reliability to be trusted without verification. An LLM-generated review is not peer review, because an LLM has no intrinsic understanding, let alone the expertise that is a prerequisite of peer review. We thank Clinical and Experimental Ophthalmology for caring enough about this issue to invite our Editorial. Finally, readers may be interested in the outcome for the article on uveitis therapeutics. After consideration of circumstances, we decided to withdraw the submission from the inviting peer-reviewed journal and target a new periodical; here we received two detailed field-specific critiques, clearly written by humans, which ultimately resulted in the publication of an excellent article. Concerningly, across this author group in 2025, several more of our articles have received AI-generated reviews. We look forward to initiatives at Clinical and Experimental Ophthalmology that address this growing problem. The authors wish to thank Ms. Janet Matthews for bibliographic support. The authors declare no conflicts of interest.