Fine-Grained Hierarchical Progressive Modal-Aware Network for Brain Tumor Segmentation

Brain tumors are highly lethal and debilitating pathological changes that require timely diagnosis and treatment. Magnetic resonance imaging (MRI), a non-invasive diagnostic tool, provides complementary multi-modal information crucial for accurate tumor detection and delineation. However, existing methods struggle to effectively fuse multi-modal information from MRI sequences and often fail to perform modality-specific feature extraction, which hinders accurate tumor segmentation. Furthermore, the inherent challenges posed by the blurred boundaries and complex morphological characteristics of tumor structures present additional substantial obstacles to achieving precise segmentation. To address these issues, we propose FiHam, a fine-grained hierarchical progressive modal-aware network that introduces a novel multi-modal fusion strategy and an advanced feature extraction mechanism. Specifically, FiHam employs a progressive fusion strategy that extracts modality-specific features at lower levels and integrates multi-modal features at higher levels to effectively leverage complementary information from tumor images. Additionally, we design a gated cross-attention modal-fusion module that adaptively selects and integrates dual-modal features using cross-attention mechanisms to enhance modality fusion. To further refine segmentation accuracy, we incorporate a tiny U-Net into the encoder to capture boundary features and complex tumor morphology. Extensive experiments on three large-scale, multi-modal brain tumor datasets demonstrate that FiHam achieves state-of-the-art performance, delivering significant improvements in segmentation accuracy and generalizability across diverse MRI modalities.