Discrimination transfers but calibration does not across fundus cameras in diabetic retinopathy screening
Abstract Deep-learning screening for referable diabetic retinopathy (DR) increasingly runs on cameras it was not developed on. We asked whether a referable-DR classifier trained on tabletop fundus photographs (BRSET) keeps its calibration — the agreement between predicted probability and observed frequency — as well as its discrimination when applied without adaptation to handheld images (mBRSET) and an independent external dataset (APTOS 2019). Across two networks and five training runs, zero-shot discrimination stayed within 0.03–0.04 AUROC of networks trained on each target (AUROC 0.89–0.96). Calibration did not: on handheld images, expected calibration error (ECE) was 0.06–0.07 without recalibration and 0.10–0.11 with the source-fitted calibrator (0.009 in-domain), and a threshold giving 90% sensitivity on the source detected only 42–47% of referable handheld images. On the handheld target, prevalence matching attributed about four-fifths ( $$79\pm 10\%$$ ) of the calibration loss to its higher disease prevalence; on the external target, prevalence and other dataset differences partly offset each other. Prevalence correction without labels did not restore calibration, whereas Platt or isotonic recalibration with 100–200 labelled target images did (ECE 0.03–0.04). Half-precision models preserved discrimination and calibration; 8-bit integer quantization reduced zero-shot AUROC by 0.07–0.14. New cameras therefore require recalibration on local labels.
Authors
- Sairam Tabibu
- Mugdha Abhyankar
Publication Details
- Journal
- Scientific Reports
- Published
- 2026-11-14
- DOI
- https://doi.org/10.1038/s41598-026-78618-8
- Primary Topic
- Retinal Imaging and Analysis
- Type
- article
- Field-Weighted Citation Impact
- 0.00