NSMA-Net: A Neighboring Slice and Modality-Aware Network for Multi-Modal Brain Tumor Segmentation

Multi-modal MR image (MRI) is commonly used for brain tumor research, as it provides complementary imaging information to guide the segmentation of different brain regions. However, due to the complex relationships between modalities, simple 3D networks struggle to capture the nonlinear information that is both similar and complementary across modalities. Clinically, radiologists compare information across modalities to identify the most indicative slices and emphasize comparisons between intra-slice and neighboring slices, while fusing distant slices is often redundant. To address these challenges, we propose a Neighboring Slice and Modality-Aware Network (NSMA-Net) for multi-modal brain tumor segmentation. Specifically, we group modalities and apply a Neighboring Slice Adaptive Attention (NSAA) module to each group, enabling the lateral fusion of pixel-level features from neighboring slices. Furthermore, we design a Multi-Dimensional Calibration and Interaction (MDCI) Module to extract modality-specific information while utilizing complementary information from other modalities calibrated through multi-dimensional attention. We conduct extensive experiments on the BraTS2020 and BraTS2021 challenge datasets. Experimental results demonstrate that compared to state-of-the-art models, NSMA-Net achieves superior segmentation performance on both benchmark datasets. The source code is available at https://github.com/qiefanwumi/NSMA-Net.