An integrated language-vision foundation model for conversational diagnostics and triaging in primary eye care

We present Meta-EyeFM, an integrated language-vision foundation model designed for conversational diagnostics and triaging in primary eye care. By combining a large language model (LLM) with eight task-specific vision foundation models (VFMs), Meta-EyeFM dynamically routes user queries and fundus photographs to the most appropriate VFMs (accuracy 96.8%). It demonstrates high performance in detecting ocular diseases (area under the receiver operating curve [AUC] ≥91.2%), differentiating disease severity (AUC ≥82%), identifying ocular signs (AUC ≥77.9%), and predicting systemic conditions like diabetes (AUC ≥79.8%). Meta-EyeFM is 11%-43% more accurate than Gemini-1.5-flash and GPT-4o LLM and generally outperforms junior ophthalmologist and optometrist graders in detecting different eye diseases. Its conversational interface and robust generalizability support its role as a diagnostic decision support tool in community settings. Through self-supervised learning and a user-friendly platform, Meta-EyeFM addresses the scarcity of skilled eye care professionals, offering scalable, explainable AI for enhancing vision screening and disease triage globally.

Authors

Institutions

Publication Details

Journal
Cell Reports Medicine
Published
2025-12-01
DOI
https://doi.org/10.1016/j.xcrm.2025.102476
Primary Topic
Retinal Imaging and Analysis
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

An integrated language-vision foundation model for conversational diagnostics and triaging in primary eye care

Sahil Thakur, Rick Siow Mong Goh, Charumathi Sabanayagam, Ming‐Chih Ho et al.
Cell Reports Medicine
Retinal Imaging and Analysis
article

An integrated language-vision foundation model for conversational diagnostics and triaging in primary eye care

Sahil Thakur, Rick Siow Mong Goh, Charumathi Sabanayagam, Ming‐Chih Ho, Ya Xing Wang, Zhi Da Soh, Songhomitra Panda‐Jonas, Zhi-Qiang Lee, Can Can Xue, Ching‐Yu Cheng, Raghavan Lavanya, Ecosse Luc Lamoureux, Quan V. Hoang, Rachel S. Chong, Wei‐Chi Wu, Jiangnan He, Xiaofeng Lei, Yang Zhou, Seang‐Mei Saw, Xinxing Xu, Tien Yin Wong, Kai Yu, Yih Chung Tham, Jie Xu, Yong Liu, Vinay Nangia, Preeti Gupta, Jost B. Jonas, Qingsheng Peng, Yang Bai, Lee Ching Linette Phang
article en

Abstract

We present Meta-EyeFM, an integrated language-vision foundation model designed for conversational diagnostics and triaging in primary eye care. By combining a large language model (LLM) with eight task-specific vision foundation models (VFMs), Meta-EyeFM dynamically routes user queries and fundus photographs to the most appropriate VFMs (accuracy 96.8%). It demonstrates high performance in detecting ocular diseases (area under the receiver operating curve [AUC] ≥91.2%), differentiating disease severity (AUC ≥82%), identifying ocular signs (AUC ≥77.9%), and predicting systemic conditions like diabetes (AUC ≥79.8%). Meta-EyeFM is 11%-43% more accurate than Gemini-1.5-flash and GPT-4o LLM and generally outperforms junior ophthalmologist and optometrist graders in detecting different eye diseases. Its conversational interface and robust generalizability support its role as a diagnostic decision support tool in community settings. Through self-supervised learning and a user-friendly platform, Meta-EyeFM addresses the scarcity of skilled eye care professionals, offering scalable, explainable AI for enhancing vision screening and disease triage globally.

Cell Reports MedicineVol. 6(12)
Beijing Tongren Hospital (CN), National University of Singapore (SG), Chang Gung University (TW), Heidelberg University (DE), Singapore National Eye Center (SG), University Hospital Heidelberg (DE), Institute of High Performance Computing (SG), Chang Gung Memorial Hospital (TW), Shanghai Eye Disease Prevention & Treatment Center (CN), Taoyuan Chang Gung Memorial Hospital (TW), Linkou Chang Gung Memorial Hospital (TW), Singapore Eye Research Institute (SG), Beijing Tsinghua Chang Gung Hospital (CN), Beijing Fengtai Hospital (CN), Suraj Eye Institute (IN), University of Pennsylvania (US)
Openalex Percentile: Top 47%
Retinal Imaging and Analysis
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.