Hybrid feature pyramid attention and capsule network (FpaNet+CapsNet) for enhanced spatial and contextual feature extraction for improved brain tumor segmentation

Gliomas are among the most dangerous brain tumors, and their treatment depends on accurately outlining the tumor in magnetic resonance imaging (MRI). These outlines are hand-drawn, vary from expert to expert, and are time-consuming to create. Automatic techniques may help, but glioma boundaries are difficult to detect because of the wide variety of shapes, sizes, and contrast. While existing deep learning models address only part of this issue, attention-based networks can capture multi-scale information but may lose spatial relationships in parts of the tumor, and capsule networks can retain these relationships but lose broader context. This study therefore proposes a hybrid model, FpaNet+CapsNet, combining a Feature Pyramid Attention Network and a Capsule Network. The attention module will see the fine details and the context, and the capsule module will preserve the shape and structure of the tumor. Four MRI sequences (T1, T1Gd, T2, and FLAIR) were used for the BraTS2020 model. Patients were split into training (236), validation (59), and test (74) groups prior to pre-processing and augmentation. Both modules proved useful on the test set, achieving Dice scores of 99.05%, 98.05%, and 99.50% for enhancing tumor, tumor core, and whole tumor, respectively, with an overall accuracy of 99.6% and an average HD95 of 2.40 mm. Ablation and statistical tests confirmed the usefulness of both modules, and the model was successful with a Dice score of 0.962 on the unseen BraTS2021 set. These results indicate that the model can accurately and reliably segment gliomas, but requires further clinical validation.