Reimplementing TransUNet: A Ground-Up Approach to Medical Image Segmentation with Transformers and CNNs

Accurate segmentation of abdominal organs in computed tomography (CT) scans is a fundamental task in medical image analysis. Traditional convolutional neural networks (CNNs), such as U-Net, are effective in capturing local spatial features but struggle with modeling long-range dependencies, which are crucial for multi-organ segmentation. To address this, hybrid architectures like TransUNet combine convolutional encoders with Transformer-based global attention. In this work, we present a from-scratch implementation of TransUNet, built entirely without relying on pre-trained weights or external libraries beyond the core deep learning framework. Our implementation includes custom modules for patch embedding, Transformer encoding, and U-Net-style decoding with skip connections. We evaluate our model on the Synapse Multi-Organ Segmentation dataset, focusing on the segmentation of eight abdominal organs. Our approach achieves a mean Dice coefficient of 77.89%, mean IoU of 66.86%, and mean pixel accuracy of 98.37%. Notably, our model demonstrates high accuracy in segmenting the liver (Dice = 94.07%) and left kidney (Dice = 88.32%), while showing room for improvement on smaller and harder-to-segment organs like the pancreas and gallbladder. To enhance interpretability, we incorporate Gradient-weighted Class Activation Mapping (Grad-CAM) visualizations, highlighting the regions where the model focuses during segmentation. These results validate the effectiveness of our fully custom TransUNet pipeline and provide insights into both performance and model decision-making, making our approach well-suited for clinical applications.segmentation