RADIFUSION: a multi-radiomics deep learning based breast cancer risk prediction model using sequential mammographic images with image attention and bilateral asymmetry refinement

<i>Objective.</i>Breast cancer is a significant public health concern, and early detection is critical for triaging high-risk patients. Sequential screening mammograms can provide important spatiotemporal information about changes in breast tissue over time, which may be useful for breast cancer risk prediction models.<i>Approach.</i>In this study, we propose a deep learning architecture called radiomics fused gated attention (RADIFUSION) that utilizes sequential mammograms and incorporates a linear image attention mechanism, radiomics features, a new gating mechanism to combine different mammographic views, and bilateral asymmetry-based finetuning for breast cancer risk assessment. We evaluate our model on a screening dataset called the Cohort of Screen-Aged Women, consisting of 8723 patients altogether.<i>Main results.</i>Based on results obtained on the independent testing set consisting of 1749 women, our approach achieved slightly better performance compared to other state-of-the-art models with area under the receiver operating characteristic curves (AUCs) of 0.905, 0.872 and 0.866 in the three respective metrics of 1 year AUC, 2 year AUC and 3 year AUC. Our study highlights the importance of incorporating various deep learning mechanisms, such as image attention, radiomics features, a gating mechanism, and bilateral asymmetry-based fine-tuning, to improve the accuracy of breast cancer risk assessment. We also demonstrate that our model's performance was enhanced by leveraging spatiotemporal information from sequential mammograms.<i>Significance.</i>Our findings suggest that RADIFUSION can provide clinicians with a useful tool for breast cancer risk assessment.