The Potential of Using ChatGPT-4 Vision for Detecting Image Manipulation in Academic Medicine Articles

To the Editor: Conducting medical research is one of the primary tasks in academic medicine. Ensuring the authenticity and reliability of scientific research is a crucial responsibility of medical researchers and educators. Image manipulation undermines scientific integrity, potentially directing research funds and efforts toward false leads. The number of articles retracted due to image manipulation issues is rising, with about 1 in 25 biomedical articles containing inappropriate duplicate images—images that are presented as original data but actually contain reused or duplicated elements from other images.1 Therefore, identifying and preventing image manipulation is an indispensable duty of every medical researcher and educator. Advances in artificial intelligence (AI) technology have provided potential solutions, such as the new large multimodal model ChatGPT-4 Vision (ChatGPT-4 V) with image recognition capabilities, which Yang and colleagues tested in object recognition and image understanding.2 To evaluate the potential of ChatGPT-4 V in detecting duplicate images, we collected 12 articles with possible image manipulation and selected images that could serve as representative examples of image manipulation. ChatGPT-4 V’s analysis of these images showed that it could effectively identify cases of simple duplication or splicing. However, its detection capabilities were limited for more complex image manipulation techniques, such as the rotation and flipping of an existing image to portray an original image. In a broader sense, integrating such advanced multimodal models into scientific scrutiny is promising, but it also comes with challenges. These include false positives (mistakenly identifying unaltered images as altered) and false negatives (missing manipulated images), especially in cases involving subtle manipulations. Such errors can result in wrongful accusations or overlooked manipulations. Additionally, the training data from multimodal models might not fully represent the diverse image types across different research fields. This may require collaboration between medical professionals and AI experts to ensure that the models are robust against various manipulations and adaptable to different scientific contexts. For now, researchers must judiciously combine the model’s recommendations with their own expertise to optimize its utility. We have preliminarily explored the potential feasibility of large multimodal models, like ChatGPT-4 V, in detecting image manipulation and their potential benefits for maintaining scientific integrity. As AI technology advances, we believe large multimodal models will play an increasingly important role in future medical research and education. We hope our comments will stimulate further discussion on the impact of AI on academic integrity and encourage more comprehensive research on this topic. Lingxuan Zhu, MDPhysician, Department of Oncology, Zhujiang Hospital, Southern Medical University, Guangzhou, ChinaHaoran Zhang, MDPhysician, Department of Oncology, Zhujiang Hospital, Southern Medical University, Guangzhou, ChinaPeng Luo, MDResearch assistant, Department of Oncology, Zhujiang Hospital, Southern Medical University, Guangzhou, China; email: [email protected]; ORCID: http://orcid.org/0000-0002-8215-2045