AI Should Read Mammograms Only When Confident: A Hybrid Breast Cancer Screening Reading Strategy

Background Quantifying the uncertainty in artificial intelligence (AI)-based interpretations of mammograms could help AI integration in screening. Purpose To evaluate the reduction in radiologists' reading workload in mammographic screening while maintaining performance by incorporating an AI model that includes uncertainty quantification. Materials and Methods An AI model was introduced that outputs a probability of malignancy (PoM) and a measure of its uncertainty. Eight candidate uncertainty metrics, based on one or all suspicious regions, were tested. A hybrid reading approach was proposed, with recall decisions made by the model only when predictions were deemed confident; otherwise, radiologist double reading was applied. The approach was retrospectively optimized and tested with a previously unseen set of screening examinations from July 2003 to August 2018, split 50-50. Recall and cancer detection rates were compared with standard double reading. The model's area under the receiver operating characteristic curve (AUC) was compared between examinations with uncertain predictions and examinations with certain predictions. One-tailed <i>P</i> values were obtained with bootstrapping, and the significance threshold was .05. Results The dataset included 41 469 examinations from 15 522 women; the median age was 59 years (IQR, 54.0-66.0 years). With the best-performing uncertainty metric, the entropy of the mean PoM of one region, 61.9% of the examinations were read by radiologists. Hybrid reading resulted in a recall rate of 23.6‰ (95% CI: 21.6, 25.5) and a cancer detection rate of 6.6‰ (95% CI: 5.5, 7.7), similar to that of standard double reading (23.9‰ [95% CI: 21.9, 25.8; <i>P</i> = .27] and 6.6‰ [95% CI: 5.5, 7.7; <i>P</i> = .14], respectively). The model's AUC was lower for examinations with uncertain predictions (0.87 [95% CI: 0.82, 0.92]) than for examinations with certain predictions (0.96 [95% CI: 0.89, 0.99]) (<i>P</i> = .02). Conclusion Incorporating an AI mammography interpretation model that includes uncertainty quantifications in the reading of screening mammograms may reduce the radiologists' workload substantially without changing cancer detection and recall rates. © RSNA, 2025 <i>Supplemental material is available for this article.</i> See also the editorial by Baltzer in this issue.