A Hybrid Vision Transformer Based Edge‐Guided Boundary Attention U‐Net for Skin Lesion Segmentation

ABSTRACT Accurate skin lesion segmentation plays a vital role in computer‐aided diagnosis for early detection and timely treatment of skin cancer. However, existing deep learning models often struggle to simultaneously capture long‐range contextual dependencies and preserve fine lesion boundary details in challenging dermoscopic images. To address these limitations, we propose Trans‐EGBAU‐Net, a hybrid Vision Transformer (ViT) based Edge‐Guided Boundary Attention (EGBA) U‐Net for accurate and boundary‐aware skin lesion segmentation. The proposed framework employs a ViT encoder to learn global contextual representation from image patches through multiple transformer layers. To compensate for the limited local spatial sensitivity of transformer features, a lightweight Edge Attention Module (EAM) is incorporated after the encoder to enhance edge‐aware representations around lesion boundaries. Furthermore, the proposed EGBA mechanism utilizes edge‐enhanced transformer features generated by the EAM to guide the Boundary‐Guided Attention Gate (BGAG) during skip‐feature refinement, while a standard Attention Gate (AG) selectively emphasizes semantically relevant lesion regions. The complementary integration of AG and EGBA enables simultaneous semantic feature selection and contour‐aware boundary refinement, resulting in improved delineation of irregular and low‐contrast lesion boundaries. Experimental evaluation on the HAM10000, ISIC 2018, and PH2 datasets demonstrates competitive segmentation performance, achieving accuracies of 0.9793, 0.9688, and 0.9679; Dice coefficients of 0.9446, 0.9165, and 0.9155; IoU scores of 0.9009, 0.8753, and 0.8596; Sensitivities of 0.9796, 0.9345, and 0.9680; and Specificities of 0.9849, 0.9772, and 0.9488, respectively. Ablation studies further validate the effectiveness of the proposed attention mechanisms, making Trans‐EGBAU‐Net a reliable framework for boundary‐aware skin lesion segmentation.