Retinal foundation-model versus ImageNet pretraining for deep learning grading of diabetic retinopathy: an external validation study

Purpose To determine whether retinal foundation-model pretraining reduces the loss of diabetic retinopathy (DR) grading agreement on external fundus-photograph datasets compared with ImageNet-pretrained convolutional networks. Methods RETFound (linear probe and full fine-tuning), EfficientNet-B3 and ResNet-50 were developed on APTOS 2019 (3,662 images) and tested unchanged on Messidor-2 (1,744 images, adjudicated grades) and IDRiD (516 images). The prespecified primary outcome was the difference in internal-to-external quadratic weighted kappa (QWK) drop relative to EfficientNet-B3 for one training run per model; a post hoc analysis averaged three or four runs per model. Results Internal QWK was 0.879 (EfficientNet-B3) and 0.888 (RETFound). In the prespecified analysis, RETFound’s excess drop was + 0.020 (95% CI − 0.024 to + 0.064) on Messidor-2 and + 0.033 (− 0.023 to + 0.091) on IDRiD; averaged over runs, it was + 0.068 (+ 0.003 to + 0.134; Holm p = 0.074) and + 0.020 (− 0.041 to + 0.080). On Messidor-2, referable-DR sensitivity was 36–43% with specificity 90–98%, and EfficientNet-B3 graded 259 of 270 mild-DR images as no DR. Matching case mix and merging grades 0 and 1 reduced the Messidor-2 drop by 63–69%. Label smoothing reproduced RETFound’s lower external calibration error. Conclusion Retinal foundation-model pretraining did not reduce external performance loss. Similar internal agreement did not establish robustness across image sources and reference standards, and no model was suitable for autonomous screening at these operating points.

Authors

Publication Details

Journal
Research Square
Published
2026-11-20
DOI
https://doi.org/10.21203/rs.3.rs-11702966/v1
Primary Topic
Retinal Imaging and Analysis
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Retinal foundation-model versus ImageNet pretraining for deep learning grading of diabetic retinopathy: an external validation study

Pravinraj Ethiraj
1 links
Research Square
Retinal Imaging and Analysis
preprint

Retinal foundation-model versus ImageNet pretraining for deep learning grading of diabetic retinopathy: an external validation study

Pravinraj Ethiraj
preprint en

Abstract

Purpose To determine whether retinal foundation-model pretraining reduces the loss of diabetic retinopathy (DR) grading agreement on external fundus-photograph datasets compared with ImageNet-pretrained convolutional networks. Methods RETFound (linear probe and full fine-tuning), EfficientNet-B3 and ResNet-50 were developed on APTOS 2019 (3,662 images) and tested unchanged on Messidor-2 (1,744 images, adjudicated grades) and IDRiD (516 images). The prespecified primary outcome was the difference in internal-to-external quadratic weighted kappa (QWK) drop relative to EfficientNet-B3 for one training run per model; a post hoc analysis averaged three or four runs per model. Results Internal QWK was 0.879 (EfficientNet-B3) and 0.888 (RETFound). In the prespecified analysis, RETFound’s excess drop was + 0.020 (95% CI − 0.024 to + 0.064) on Messidor-2 and + 0.033 (− 0.023 to + 0.091) on IDRiD; averaged over runs, it was + 0.068 (+ 0.003 to + 0.134; Holm p = 0.074) and + 0.020 (− 0.041 to + 0.080). On Messidor-2, referable-DR sensitivity was 36–43% with specificity 90–98%, and EfficientNet-B3 graded 259 of 270 mild-DR images as no DR. Matching case mix and merging grades 0 and 1 reduced the Messidor-2 drop by 63–69%. Label smoothing reproduced RETFound’s lower external calibration error. Conclusion Retinal foundation-model pretraining did not reduce external performance loss. Similar internal agreement did not establish robustness across image sources and reference standards, and no model was suitable for autonomous screening at these operating points.

Research Square
Good health and well-being
Retinal Imaging and Analysis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon