ExU-Trans: a self-explanatory transformer with U-Net based hybrid model for brain tumor segmentation using MR imaging

Brain tumor segmentation from MRI images is a critical task for accurate diagnosis and treatment planning. While traditional segmentation approaches leverage deep learning models, such as U-Net and its variants, they often struggle with balancing precision and interpretability, especially in handling noisy and complex MRI data. Earlier works have demonstrated remarkable performance improvements through hybrid models, yet limitations persist in effectively capturing both global and local contexts, as well as ensuring model transparency for clinical application. These shortcomings highlight the need for frameworks that not only excel in segmentation accuracy but also provide explainability for reliable medical decision-making. To address these challenges, we propose a novel Explainable U-Net Transformer (ExU-Trans) framework for brain tumor segmentation in MRI images. ExU-Trans integrates the interpretability strengths of Vision Transformers with the spatial precision of U-Net in a dual-encoder structure. The Vision Transformer encoder employs Self-Explanatory Multi-Head Attention (SE-MHA) to generate noise-resistant attention maps, while a modified U-Net encoder focuses on extracting multiscale spatial features critical for tumor delineation. To enhance interpretability, a Discriminative Attribute Explainer (DAE) is embedded within the transformer encoder, pinpointing tumor-specific attributes at a pixel level. Furthermore, a Contextual Self-Attention mechanism enriches feature representations by capturing both global and local contexts, refining tumor detection. The outputs from both encoders are combined using a bivariate fusion module that incorporates spatial and channel attention, producing a unified feature representation. Finally, an attribute-guided loss function enables self-supervised learning, resulting in highly accurate and inherently interpretable segmentation outcomes. Extensive evaluations demonstrate the superiority of ExU-Trans in terms of segmentation accuracy and explainability, offering a reliable tool for clinical brain tumor diagnosis and treatment planning.