Assessing the robustness of AI lesion risk scores at different exposure settings using an anthropomorphic breast phantom

To assess the robustness of risk scores provided by an artificial intelligence (AI) system for digital mammography (DM), when varying the exposure conditions. An anthropomorphic breast phantom containing a lesion, was imaged with DM at different tube voltages (kV), tube loadings (mAs), and anode/filter combinations (W/Rh, Mo/Mo, and Mo/Rh). The organ doses were extracted from the DICOM header and used as a substitute for average glandular dose. The images were analyzed with an AI system, which provided a lesion risk score which translates to suspicion for malignancy. Correlations between the lesion risk score and the exposure conditions were investigated. In most imaging conditions, weak to moderately strong positive associations between lesion risk scores and kV and mAs, respectively, were reported (varying by anode/filter combinations). When organ dose increased the AI risk scores plateaued, and further increase did not increase the lesion risk score. For typical clinical settings (W/Rh, 27 kV and 71 mAs) the range of lesion risk scores was 33-56 (mean: 42, SD: 9). Greatest reported variability (range: 36-63, mean: 51, SD: 12) was found at 27 kV and 36 mAs (using W/Rh). Images of suboptimal quality may result in inaccurate AI system performance. The unexpectedly large intra-group variability of AI risk scores should be further investigated.