Temporally consistent longitudinal brain tumor segmentation using a temporal spatial transformer network

Abstract Proper monitoring of tumor progression and evaluation of treatment responses highly depend on longitudinal brain tumor segmentation from MRI data. Current deep learning methodologies have mainly concentrated on analyzing single-time-point images, which restricts the ability to incorporate temporal dynamics during the segmentation process. The study proposes a new approach called Temporal-Spatial Transformer Network (TST-Net), which can be used for longitudinal brain tumor segmentation. The proposed methodology involves a temporal attention module to integrate the progression-aware information across temporally successive MRI images and a spatial attention module to improve tumor detection. Preprocessing of BraTS longitudinal MRI data and subsequent training of TST-Net on pre-aligned datasets were done in an end-to-end fashion. Evaluation was done by Dice similarity coefficients and compared with current methods. TST-Net proved to be more effective than previous solutions, obtaining 86.0%, 88.0%, and 91.0% Dice scores for improving tumor segmentation, tumor core, and overall tumor segmentation, respectively. This study reveals that incorporating the temporal dimension along with spatial attention leads to increased accuracy of brain tumor segmentation. Incorporating temporal information and applying spatial attention can effectively decrease inconsistencies between follow-up images and contribute to improved tumor area detection. TST-Net can serve as an important tool for clinical applications. TST-Net presents an efficient solution to brain tumor segmentation due to its ability to combine both spatial and temporal attention.