Swef-UNet: Toward an Efficient Pure Transformer-Based Medical Image Segmentation

Transformers have significantly advanced the field of medical image segmentation. Despite the advancements, challenges remain: notably, the computational intensity of the self-attention mechanism and the nuanced learning of spatial representations within purely Transformer-based frameworks. In response, building on the strength of Transformers, this paper introduces a novel architecture by first, reformulating the self-attention mechanism to incorporate efficient attention, reducing the computational burden without compromising the richness of information captured. This adaptation maintains the core advantages of dot-product attention - capturing complex dependencies within the data - but with a fraction of the memory and computational cost. Second, by employing deep supervision at each decoder layer of our UNet-like network, designed to enhance segmentation accuracy by promoting the development of richer, more discriminative features throughout the network. Together, the adaptation of efficient attention and the implementation of deep supervision offer a balanced solution between computational efficiency and segmentation precision. Our findings reveal the model's capacity to outperform state-of-the-art results.