A Multimodal Brain Tumor Segmentation Method: Parallel Fusion of Transformer and CNN

While convolutional neural network(CNN) have historically achieved significant success in 3D medical image segmentation, recent advancements in Vision Transformer (ViT) have led to increased popularity of Transformer architectures in this field. Leveraging the global information extraction capabilities of Transformers, we propose a novel parallel structure combining CNN and Transformer for multimodal brain tumor segmentation, named PTCNet. We introduce two key improvements to enhance multimodal segmentation:1) Introducing a novel channel attention module that replaces spatial self-attention in the Transformer to improve cross-modal information extraction.2) Enhancing the reversible feature extraction module based on residual structures to better detect small lesions. We evaluated our method on the BraTS 2023 brain metastasis dataset, achieving an average DSC of 61.42%, which is 1.12% higher than the second place. The innovative structure and improved modules of PTCNet significantly enhance performance in multimodal brain tumor segmentation tasks. Code and models are available at https://github.com/naiqizh/PTCNet.