Multi-Modal Deep Learning in Breast Cancer Diagnosis: A Review of Recent Advances

Breast cancer detection often requires multiple imaging modalities to overcome the shortcomings of each modality. Deep learning advancements facilitated our incorporation of these multi-modalities into precise diagnostic systems. This paper reviews the recent applications of multi-modal deep learning in breast cancer detection and classification. This approach is further consolidated through recent advances such as multi-view vision transformers and different fusion strategies utilized in networks to leverage complementary information between modalities, improving both tumor detection and lesion characterization. These works represent a key advance toward developing AI systems that can replicate radiologists’ comprehensive reviews without additional burden on healthcare ecosystem. We also addressed clinical barriers in implementation, such as data heterogeneity and interoperability, and suggested pathways for future directions to overcome these challenges. This study describes the steps required for a safe and effective integration of multi-modal deep learning into healthcare facilities in near future.