Adaptive Context-Aware Hybrid Fusion (ACAHF) for Multi-Modality Data Integration in Breast Cancer Detection

Breast cancer detection and prognosis prediction is expected to improve significantly if different data modalities such as imaging, genomics, and clinical records are integrated. However, many of the existing methods are not adaptable to weight the modalities according to their relevance and quality; hence, they may not be efficient. In this study, we propose a new technique, Adaptive Context Aware Hybrid Fusion (ACAHF) to take care of this issue. ACAHF combines early, intermediate, and later advantage of fusion strategies along with attention mechanisms and contextual weight for the integrative optimization of imaging-genomic-clinical data sets. In our method, fusion strategies are dynamically adopted based on intermodal relations and input quality, which guarantees the most informative modalities to be prioritized. We illustrate our technique on breast cancer datasets, including TCGA (The Cancer Genome Atlas) and DDSM Digital Database for Screening Mammography, and show that it performs better than currently available models on the dataset.