Comparative evaluation of GPT 5.2 Pro and Gemini 3 Pro large multimodal models for retinal optical coherence tomography interpretation

PURPOSE: Large multimodal models (LMMs) capable of interpreting medical images are increasingly accessible to patients, yet their reliability for retinal optical coherence tomography (OCT) interpretation remains uncertain. This study evaluated the performance of two state-of-the-art LMMs in detecting OCT features and generating diagnostic interpretations. METHODS: In this cross-sectional study, 330 publicly available retinal OCT B-scans spanning 12 diagnostic categories, including normal scans and common retinal diseases, were analyzed. Two commercially available LMMs (GPT 5.2 Pro and Gemini 3 Pro) were evaluated using zero-shot inference. Models were assessed separately for detection of nine predefined OCT features and for free-text diagnostic interpretation. Feature detection performance was measured using per-feature metrics and macro-averaged F1 scores, while diagnostic accuracy was assessed against expert-adjudicated reference standards. RESULTS: Gemini 3 Pro demonstrated higher feature detection performance than GPT 5.2 Pro (macro-averaged F1 score 0.558 vs. 0.472) and higher diagnostic accuracy (58.8% vs. 32.1%). Both models performed better for visually salient abnormalities but showed limited sensitivity for subtle structural features. Normal OCT scans were frequently misclassified as pathological, particularly by GPT 5.2 Pro, which failed to correctly identify any normal images. Recurrent, model-specific diagnostic biases and hallucinated diagnoses were observed. CONCLUSIONS: Although LMMs can extract clinically relevant information from OCT scans, their diagnostic reliability remains limited, with frequent overcalling of pathology and systematic biases. Given their increasing availability to patients, these limitations raise concerns about potential misinterpretation outside supervised clinical settings. Careful validation and clear usage boundaries are needed before such tools can be safely integrated into patient-facing workflows.

Authors

Institutions

Publication Details

Journal
Graefe s Archive for Clinical and Experimental Ophthalmology
Published
2026-11-16
DOI
https://doi.org/10.1007/s00417-026-07604-7
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Comparative evaluation of GPT 5.2 Pro and Gemini 3 Pro large multimodal models for retinal optical coherence tomography interpretation

Sumit Kumar Sharma, Victor C. F. Bellanda, Matthew J. Schulgit, David Shin et al.
7 links
Graefe s Archive for Clinical and Experimental Ophthalmology
Artificial Intelligence in Healthcare and Education
article

Comparative evaluation of GPT 5.2 Pro and Gemini 3 Pro large multimodal models for retinal optical coherence tomography interpretation

Sumit Kumar Sharma, Victor C. F. Bellanda, Matthew J. Schulgit, David Shin, Nitesh Mohan, Fares Antaki, Eliot N. Haddad, Sunil K. Srivastava, Yuka Onishi, Sandeep Singh, Cindy Chen
article en

Abstract

PURPOSE: Large multimodal models (LMMs) capable of interpreting medical images are increasingly accessible to patients, yet their reliability for retinal optical coherence tomography (OCT) interpretation remains uncertain. This study evaluated the performance of two state-of-the-art LMMs in detecting OCT features and generating diagnostic interpretations. METHODS: In this cross-sectional study, 330 publicly available retinal OCT B-scans spanning 12 diagnostic categories, including normal scans and common retinal diseases, were analyzed. Two commercially available LMMs (GPT 5.2 Pro and Gemini 3 Pro) were evaluated using zero-shot inference. Models were assessed separately for detection of nine predefined OCT features and for free-text diagnostic interpretation. Feature detection performance was measured using per-feature metrics and macro-averaged F1 scores, while diagnostic accuracy was assessed against expert-adjudicated reference standards. RESULTS: Gemini 3 Pro demonstrated higher feature detection performance than GPT 5.2 Pro (macro-averaged F1 score 0.558 vs. 0.472) and higher diagnostic accuracy (58.8% vs. 32.1%). Both models performed better for visually salient abnormalities but showed limited sensitivity for subtle structural features. Normal OCT scans were frequently misclassified as pathological, particularly by GPT 5.2 Pro, which failed to correctly identify any normal images. Recurrent, model-specific diagnostic biases and hallucinated diagnoses were observed. CONCLUSIONS: Although LMMs can extract clinically relevant information from OCT scans, their diagnostic reliability remains limited, with frequent overcalling of pathology and systematic biases. Given their increasing availability to patients, these limitations raise concerns about potential misinterpretation outside supervised clinical settings. Careful validation and clear usage boundaries are needed before such tools can be safely integrated into patient-facing workflows.

Graefe s Archive for Clinical and Experimental Ophthalmology
Cleveland Clinic (US), The Ohio State University (US)
Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon