Retinal foundation-model versus ImageNet pretraining for deep learning grading of diabetic retinopathy: an external validation study
Purpose To determine whether retinal foundation-model pretraining reduces the loss of diabetic retinopathy (DR) grading agreement on external fundus-photograph datasets compared with ImageNet-pretrained convolutional networks. Methods RETFound (linear probe and full fine-tuning), EfficientNet-B3 and ResNet-50 were developed on APTOS 2019 (3,662 images) and tested unchanged on Messidor-2 (1,744 images, adjudicated grades) and IDRiD (516 images). The prespecified primary outcome was the difference in internal-to-external quadratic weighted kappa (QWK) drop relative to EfficientNet-B3 for one training run per model; a post hoc analysis averaged three or four runs per model. Results Internal QWK was 0.879 (EfficientNet-B3) and 0.888 (RETFound). In the prespecified analysis, RETFound’s excess drop was + 0.020 (95% CI − 0.024 to + 0.064) on Messidor-2 and + 0.033 (− 0.023 to + 0.091) on IDRiD; averaged over runs, it was + 0.068 (+ 0.003 to + 0.134; Holm p = 0.074) and + 0.020 (− 0.041 to + 0.080). On Messidor-2, referable-DR sensitivity was 36–43% with specificity 90–98%, and EfficientNet-B3 graded 259 of 270 mild-DR images as no DR. Matching case mix and merging grades 0 and 1 reduced the Messidor-2 drop by 63–69%. Label smoothing reproduced RETFound’s lower external calibration error. Conclusion Retinal foundation-model pretraining did not reduce external performance loss. Similar internal agreement did not establish robustness across image sources and reference standards, and no model was suitable for autonomous screening at these operating points.
Authors
- Pravinraj Ethiraj (ORCID: https://orcid.org/0009-0000-2975-1631)
Publication Details
- Journal
- Research Square
- Published
- 2026-11-20
- DOI
- https://doi.org/10.21203/rs.3.rs-11702966/v1
- Primary Topic
- Retinal Imaging and Analysis
- Type
- preprint