A Multimodal Fusion Approach for Weakly Supervised Segmentation of MRI Medical Images

Due to the scarcity and high cost of acquiring labeled data in the field of image segmentation, weakly supervised semantic segmentation (WSSS) methods based on Class Activation Mapping (CAM) have been proposed. In the medical image segmentation domain, current research on weakly supervised learning primarily focuses on single-modal methods. However, multimodal imaging provides richer information. Magnetic Resonance Imaging (MRI) is commonly used for brain tumor segmentation. We propose a novel optimized CAM method tailored for MRI brain tumor segmentation, with the core idea being multimodal fusion. Initially, we generate CAM for different MRI modalities (in our experiments, we selected T2 and FIAIR modalities), employing weakly supervised methods. Subsequently, we fuse these CAM to obtain optimized CAM for segmentation. Our method is evaluated on the BraTS dataset, and experimental results demonstrate its superiority over existing techniques at the same level of supervision.