Calibrated Confidence and Confidence-Stratified Triage for Diabetic Retinopathy Severity Grading: A Class-Imbalance-Aware Deep Learning Study on the DDR Dataset

Background: Deep learning systems for diabetic retinopathy (DR) grading are usually reported through accuracy and area under the curve, yet clinical use depends on whether a model’s confidence can be trusted. Problem: calibration and confidence-based triage are rarely evaluated on large, naturally imbalanced five-grade datasets. Objective: to quantify how temperature scaling and Monte Carlo (MC) Dropout change the reliability of an EfficientNet-B0 DR grader and to convert calibrated confidence into auto-clear/human-review operating points. Materials: A total of 12,522 DDR fundus photographs on the five ICDR grades, only 1.9% of them severe non-proliferative DR. Methods: Methods included five-fold cross-validation grouped by near-duplicate images, with inner calibration splits; overall and per-grade expected calibration error (ECE); clustered bootstrap; comparison with other calibration maps, loss functions and backbones; Grad-CAM and LIME. Results: Macro-F1 was 0.705 (95% CI 0.691–0.718). Temperature scaling reduced ECE from 0.186 to 0.078 without changing any prediction, but grade-specific maps were needed to bring per-grade ECE below 0.01. Mutual information from MC-Dropout flagged errors better than confidence (AUROC 0.748 vs. 0.650). At a no-DR threshold of 0.80, 30.1% of images were auto-cleared with a 1.46% referable-DR false-clear rate. Conclusions: One temperature corrected most top-label miscalibration but not the grade-level error caused by class weighting; the triage thresholds need external validation before clinical use.