Category-Aware Fusion Network for Multi-Modal Brain Tumor Segmentation

Automated segmentation of tumor from multi-modal MRI is crucial for clinical diagnosis and scientific research. Multi-modal MRI provides comprehensive tissue information, essential for accurate delineation of tumor boundaries and differentiation from healthy tissues. However, effectively fusing information from different modalities while preserving relevant features and minimizing redundancy poses significant challenges. The inherent heterogeneity and variability between modalities complicate the fusion process, requiring sophisticated algorithms for feature extraction, alignment, and integration. To address these challenges, this paper proposes a Category-Aware Fusion Network (CAFNet) designed to fuse multi-modal information efficiently while considering computational costs. The model employs an inter-layer fusion strategy to explore inter-modality relationships and aggregate category features at multiple scales. This approach enhances the effectiveness of multi-modal information fusion for brain tumor segmentation. The CAFNet includes a Category Feature Update (U) module to capture long-range dependencies in feature maps, ensuring accurate and stable feature extraction. And Category Feature Fusion (CF) module fuses multi-scale and multi-feature information while reducing computational costs and enhancing tumor feature extraction. Additionally, a Feature Alignment (FA) module is introduced to address inconsistent feature distributions between modalities due to domain differences, promoting effective multi-modal feature fusion. The proposed method demonstrates superior performance in brain tumor segmentation tasks on the BraTS2018 and BraTS2020 datasets, validating its effectiveness in enhancing the accuracy and stability of multi-modal MRI segmentation.