Cross-Attention Mechanism and Vision Transformer-Enhanced Multimodal Medical Imaging Fusion for Nasopharyngeal Carcinoma Segmentation

Motivation: The segmentation of nasopharyngeal carcinoma (NPC) is vital for diagnostic and prognostic processes. NPC segmentation is challenging due to its intricate anatomy, variability, and closeness to essential structures. Goal(s): This study aims to improve NPC segmentation accuracy by leveraging multiple modalities, such as DCE-MRI and PET-CT. However, it is worth noting that previous research has not fully harnessed the potential of cross-modal features through cross-attention mechanisms. Approach: This paper introduces a new approach, integrating cross-attention with the Vision Transformer structure, enabling efficient interaction between features from various modalities. Results: The experiments show that the proposed model offers superior performance and state-of-the-art results. Impact: The proposed method aims to enhance NPC segmentation results by utilizing multimodal medical imaging fusion, such as DCE-MRI fusion and PET-CT fusion. This approach has the potential to benefit other segmentation tasks involving multimodal medical image data.