A hybrid network combining convolutional neural networks and transformers for skin disease segmentation

This paper presents a new hybrid network combining convolutional neural networks and transformers for skin disease segmentation, named TBMAU-Net, specifically designed for high-precision segmentation of skin lesion areas. The model leverages Transformers to capture long-range dependencies and employs Convolutional Neural Networks (CNNs) to extract detailed local information. A novel hybrid module that integrates CNNs with Transformers is introduced to effectively capture both global and local feature information. To enhance the extracted features, we utilize a depthwise separable convolution attention module, which dynamically assigns attention weights during the encoding phase, thereby enriching the channel and spatial information. We also introduce a multi-scale aggregation module that fuses features from multiple scales, promoting robust feature interactions and compensating for variations in image resolutions. Extensive experiments conducted on the publicly available ISIC 2016 and PH2 datasets demonstrate that TBMAU-Net achieves an optimal balance between accuracy and efficiency. It outperforms many state-of-the-art methods in segmentation performance, showing remarkable capability in handling irregular shapes, significant color variations, ambiguous boundaries, and complex interference-prone lesion areas. These advanced functionalities significantly ease the challenge for medical professionals in accurately assessing complex skin lesions. By integrating the strengths of both Transformers and CNNs, TBMAU-Net represents a significant step forward in medical image segmentation, offering improved precision and reliability in diagnosing skin conditions.