Ophthalmic AI With Just Ten Examples: Comparing Multimodal Large Language Models, Vision Foundation Models, and Vision-Language Models

Purpose: To compare multimodal large language models (MLLMs), vision foundation models (VFMs), and contrastive vision-language models (VLMs) for ophthalmic image interpretation with no labeled examples (zero-shot) or as few as 10 labeled examples per class (10-shot). Design: Retrospective benchmarking study. Subjects: 4 public datasets (UCSD, IDRiD, REFUGE, ASOCT-MSDC) and 3 private datasets (OHTS, BPEI-DLS, WashU-OCT) of anterior- and posterior-segment ophthalmic imaging. Methods: Nine AI models were evaluated across 3 architectural families: MLLMs (Qwen3.8-27B, GLM-5.3-Flash, Gemma-4-31B-it), VFMs (DINOv3, RETFound, VisionFM), and VLMs (CLIP, EyeCLIP, SigLIP2). Models were assessed on disease classification, disease risk and severity grading, anatomical assessment, and lesion localization in fundus and OCT imaging. MLLMs were also compared with a 3D VFM (OCTCube-M) for whole-volume OCT classification. MLLMs were evaluated zero-shot and 10-shot, without additional training. Vision models (VFMs and VLMs) were adapted by linear probing and fine-tuning on the same 10-shot set and on the full training dataset (FD). Main Outcome Measures: Disease classification performance (macro-averaged F1, AUROC, quadratic weighted kappa [QWK]), anatomical localization accuracy, and cross-dataset generalization. Results: For retinal OCT disease classification, 10-shot MLLMs achieved a macro-F1 of 0.94, versus 0.59 (10-shot) and 0.96 (FD) for vision models. For diabetic retinopathy and macular edema grading, 10-shot MLLMs achieved QWKs of 0.69 and 0.73, versus 0.48 and 0.57 (10-shot) and 0.72 and 0.79 (FD) for vision models. For lesion localization, 10-shot MLLMs achieved an F1 of 0.75, versus 0.70 (10-shot) and 0.86 (FD) for vision models. For vertical cup-to-disc ratio estimation (OHTS), zero-shot MLLMs achieved a mean absolute error of 0.14, versus 0.21 (10-shot) for vision models. For whole-volume OCT classification (WashU-OCT), zero-shot MLLMs achieved a macro-F1 of 0.71, versus 0.50 (10-shot) and 0.82 (FD) for vision models. Conclusions: Open-weight MLLMs given 10 labeled examples per class outperformed VFMs and VLMs trained on the same examples in most tasks and approached the performance of fully trained VFMs and VLMs in selected disease classification and grading tasks. MLLMs also demonstrated promising zero-shot performance on non-public clinical datasets without task-specific training. By reducing the need for large labeled datasets and task-specific model adaptation, MLLMs could broaden who can develop and validate ophthalmic AI.