GSFormer: A Gated Skip Connection and Feature Fusion Transformer-based Neural Network for 3D MRI Brain Tumor Segmentation

How can local and global features be comprehensively considered to optimize dense prediction tasks, like medical image segmentation? To overcome this problem, this study showed an encoder-decoder based architecture consisting of 3D convolutional neural network and a Transformer model specifically designed for brain tumor segmentation. The encoder initially processes the input image to extract local features. A multi-modal self-attention mechanism was proposed to learn long-range dependencies across modalities. To fully mine the relationship of features between shallow and deep neural layers, we design a gated fusion strategy using Gated Recurrent Units. Ablation experiments on 3D MRI scans demonstrated the method's effectiveness. The results of the BraTS 2021 benchmark for automatic brain tumor segmentation, where it achieved competitive, and in some cases superior, results compared to other advanced approaches. This confirms that the combination of Transformers, multi-modal attention mechanisms, and gated fusion strategies can significantly enhance the accuracy of segmentation in complex medical imaging tasks.