SFFR-Transformer: Spatial and Frequency Feature Recalibration Transformer for Incomplete Multimodal Medical Image Segmentation

Multimodal magnetic resonance imaging (MRI) is vital for the precise segmentation of brain tumors. However, missing or incomplete multimodal data is a frequent challenge in clinical practice, significantly affecting segmentation performance. Current advanced methods primarily focus on fusing multimodal images in the spatial domain, often neglecting the interplay between different modalities in the frequency domain. In this work, we propose a novel framework named SFFR-Transformer, which utilizes a frequency and spatial hybrid multi-head attention Transformer. This approach facilitates the complementary fusion of spatial and frequency domain information, enhancing the reconstruction of missing modalities. Moreover, most existing methods map fused modalities directly to all segmentation targets. Inspired by the correlation between single modalities and specific sub-targets, we introduce a modality-sub-target matching module. This module decouples the fusion modalities from the segmentation targets, enabling more accurate mapping between single modalities and their corresponding sub-targets. Comprehensive experiments on the publicly available BraTS2018 and BraTS2020 datasets demonstrate that our framework surpasses state-of-the-art methods, particularly in scenarios involving missing modalities.