Evaluating the performance of artificial intelligence and radiologists accuracy in breast cancer detection in screening mammography across breast densities

Artificial intelligence (AI) has the potential to improve breast cancer detection in mammography. This study aimed to compare the diagnostic accuracy of radiologists and AI system in detecting breast cancer using digital mammograms, considering the varying breast densities. In this retrospective cohort study, we analyzed 617 mammograms, including 104 biopsy-confirmed cancer cases in a cancer-enriched sample. These images were evaluated by the Lunit INSIGHT mammogram AI system, while radiologists classified lesions according to the BI-RADS system. Biopsy results served as the reference standard. Out of a total of 158 biopsies, radiologists detected 102/158 (64.5 %) of cancers, while AI identified 90/158 (57 %) with no statistically significant difference. Overall, radiologists were more sensitive (98 % vs. 87 %) but less specific (17 % vs. 44.4 %) with both differences being statistically significant (P = 0.005). AI had a higher positive predictive value (PPV: 74 % vs. 69 %), especially in non-dense breasts, where it showed improved specificity (58 % vs. 16 %) and PPV (82 % vs. 71 %); P = 0.02. Conversely, radiologists outperformed AI in detecting malignant tumors in dense breasts. In 12 discrepant cases (1.9 %), radiologists correctly identified all malignancies missed by AI, while AI did not rectify any radiologist errors. Although AI showed better performance in non-dense tissue, radiologists remained more accurate overall in dense breasts. Although both AI and radiologists showed comparable performance in overall cancer detection, radiologists demonstrated higher sensitivity, especially in dense breasts, while AI achieved higher specificity. • Radiologists were more sensitive than AI in detecting breast cancer, particularly in dense breasts. • AI demonstrated better specificity and positive predictive value, performing well in non-dense breasts. • When radiologists and AI disagreed, radiologists identified all missed malignancies, showing greater accuracy.