Enhancing Tumor Segmentation in PET-CT Scans: A Comparative Analysis of Multimodal Fusion Strategies

In the field of medical image segmentation, the efficient fusion of multimodal information presents significant challenges, especially in tumor segmentation using Positron Emission Tomography (PET) and Computed Tomography (CT) scans. This paper presents an insightful approach to these challenges by introducing three fusion strategies: early, late, and hyper-fusion. These strategies are implemented through an encoder-decoder architecture, utilizing a Swin Transformer encoder and an attention gating mechanism in the decoder. Experimental results using the Lung-PET-CT-Dx dataset demonstrate that these models, notably the hyper-fusion model, surpass their unimodal counterparts, especially in complex tumor scenarios. This research offers a unique comparative analysis of these strategies, providing a valuable reference for future studies in multimodal image segmentation.