Specialized curricula for training vision language models in retinal image analysis
Clinicians spend significant time reviewing medical images and transcribing findings. By integrating visual and textual data, foundation models have the potential to reduce workloads and boost efficiency, yet their practical clinical value remains uncertain. In this study, we find that OpenAI's ChatGPT-4o and two medical vision-language models (VLMs) significantly underperform ophthalmologists in key tasks for age-related macular degeneration (AMD). To address this, we developed a dedicated training curriculum, designed by domain specialists, to optimize VLMs for tasks related to clinical decision making. The resulting model, RetinaVLM-Specialist, significantly outperforms foundation medical VLMs and ChatGPT-4o in AMD disease staging (F1: 0.63 vs. 0.33) and referral (0.67 vs. 0.50), achieving performance comparable to junior ophthalmologists. In a reader study, two senior ophthalmologists confirmed that RetinaVLM's reports were substantially more accurate than those written by ChatGPT-4o (64.3% vs. 14.3%). Overall, our curriculum-based approach offers a blueprint for adapting foundation models to real-world medical applications.
Authors
- Martin J. Menten (ORCID: https://orcid.org/0000-0001-8261-7810)
- Lars G. Fritsche (ORCID: https://orcid.org/0000-0002-2110-1690)
- Daniel Rueckert (ORCID: https://orcid.org/0000-0002-5683-5889)
- Maximilian Pfau (ORCID: https://orcid.org/0000-0001-9761-9640)
- Philip Müller (ORCID: https://orcid.org/0000-0001-8186-6479)
- Dimitra Mitsopoulou (ORCID: https://orcid.org/0000-0002-5002-1788)
- Sophie Riedl (ORCID: https://orcid.org/0000-0003-0003-0886)
- Thomas R. Taylor (ORCID: https://orcid.org/0000-0001-7906-790X)
- Ursula Schmidt‐Erfurth (ORCID: https://orcid.org/0000-0002-7788-7311)
- Julia Mai (ORCID: https://orcid.org/0000-0002-0528-9742)
- Hendrik P. N. Scholl (ORCID: https://orcid.org/0000-0002-7312-0316)
- Paul Hager (ORCID: https://orcid.org/0000-0003-2166-0262)
- Sobha Sivaprasad (ORCID: https://orcid.org/0000-0001-8952-0659)
- Kristina Pfau (ORCID: https://orcid.org/0000-0002-3642-3502)
- Andrew Lotery (ORCID: https://orcid.org/0000-0001-5541-4305)
- A Toby Prevost (ORCID: https://orcid.org/0000-0003-1723-0796)
- Maria Patsiamanidi
- Robbie Holland (ORCID: https://orcid.org/0000-0001-8373-8390)
- Johannes C. Paetzold (ORCID: https://orcid.org/0000-0002-4844-6955)
- Hrvoje Bogunović (ORCID: https://orcid.org/0000-0002-9168-0894)
- Christopher Holmes (ORCID: https://orcid.org/0000-0003-2018-3732)
Institutions
- Moorfields Eye Hospital NHS Foundation Trust (GB)
- University of Bonn (DE)
- Austrian Research Institute for Artificial Intelligence (AT)
- University of Basel (CH)
- University of Michigan (US)
- NIHR Imperial Biomedical Research Centre (GB)
- Moorfields Eye Hospital (GB)
- Institute of Molecular and Clinical Ophthalmology Basel (CH)
- University of Southampton (GB)
- Munich Center for Machine Learning
- University College London (GB)
- Imperial College London (GB)
- Technical University of Munich (DE)
- Medical University of Vienna (AT)
Publication Details
- Journal
- npj Digital Medicine
- Published
- 2025-08-19
- DOI
- https://doi.org/10.1038/s41746-025-01893-8
- Citations
- 3
- Primary Topic
- Retinal Imaging and Analysis
- Type
- article
- Field-Weighted Citation Impact
- 4.08