Lightweight Skin Lesion Images Segmentation Based on  <scp>CNN</scp> and Transformer

ABSTRACT Accurate segmentation of skin lesion images is essential for diagnosing and treating skin disorders. While current research primarily aims to enhance segmentation accuracy through the use of complex network models, the large size of these models restricts their practical application in clinical settings. To address this challenge, we propose SMedt (skin‐medical transformer), a high‐precision, parameter‐efficient model for skin lesion image segmentation. SMedt combines CNN and transformer architectures within a dual‐branch structure to extract both global and local features effectively. The model's global branch employs a dual‐attention mechanism that integrates channel and spatial attention, along with skip connection cross attention (SCCA) between the encoder and decoder layers, to enhance global feature decoding. The local branch incorporates an All‐aggregation decoder (all decoder) method, enabling the capture of multi‐scale features, while pyramid stacking improves the extraction of local features across different channel dimensions. We evaluated SMedt on the ISIC2016 and ISIC2018 datasets. On the ISIC2016 dataset, the model achieved a 0.17% improvement in the Dice coefficient over the second‐best model, FAT‐Net, while reducing model parameters by 91%. On the ISIC2018 dataset, SMedt improved the Dice coefficient by 1.1% over the state‐of‐the‐art Efficient UNet, while reducing model parameters by 73.26%. Compared with the latest lightweight model UCM‐Net, SMedt improves segmentation accuracy (Dice) by 1.92% on ISIC2018, achieving a better balance between accuracy and model size. By leveraging the strengths of both CNN and Transformer models, SMedt maintains high segmentation accuracy while keeping a low parameter count, thereby enhancing diagnostic accuracy and clinical efficiency.