Masked and Noise-Masked Multimodal Brain Tumor Image Segmentation Using SegFormer and Shared Encoder Framework

Medical image segmentation is a critical task in clinical diagnosis and treatment, particularly for brain tumor analysis using imaging modalities such as Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) scans. However, segmentation is often hindered by noise introduced through scanner artifacts, motion blur, and inconsistent acquisition conditions. To address this limitation and the challenges of missing and noisy pixels in an MRI image, this work introduces the Masking and Noise-masking Multimodal SegFormer (MNMS) - a transformer-based framework designed for brain tumor segmentation, which collectively combines complementary information from multiple MRI modalities while integrating dual masking strategies, one to handle incomplete data and another to overcome noise artifacts. Being a multimodal framework, MNMS can effectively work with and provide valuable segmentation results for any single modality which it has been trained for, ensuring robustness in real-world clinical scenarios where multimodal data may not always be available. MNMS employs masking to overcome computational complexity while maintaining local and global features essential for accurate medical image segmentation. Additionally, noise-masking introduces a controlled Gaussian noise into the MRI images creating random variations in the pixel intensities and thereby encouraging the model to learn the invariant and essential patterns in MRI images. Evaluation on Brain Tumor Segmentation (BraTS2020) challenge dataset demonstrates that MNMS outperforms conventional convolutional neural network (CNN) based methods, achieving superior accuracy and Dice Similarity Coefficient (DSC) scores. Specifically, the proposed MNMS model achieved a DSC score of 0.9341, outperforming UNEt TRansformers (UNETR), which has a DSC of 0.8208 and 3D U-Net (0.8179). These results highlight its effectiveness in multimodal brain tumor segmentation, ultimately contributing to improved diagnostic accuracy and patient care.