Feature adaptation network for skin lesion segmentation in medical images

Early diagnosis and treatment of skin diseases (including skin cancer) are crucial for patient prognosis, and accurate medical image segmentation plays a key role in identifying lesion areas. This paper proposes a skin disease image segmentation method based on an improved SAM model, called SAM-FA. The method enhances the adaptability and performance of the SAM model in medical image segmentation tasks by introducing a lightweight Feature Adapter (FA) module. Specifically, SAM-FA freezes the pretrained image encoder and incorporates the FA module into the Transformer layer, significantly reducing computational complexity and improving the model’s segmentation capability for complex lesion areas and low-contrast images. The FA module optimizes feature representation through a downsampling and upsampling mechanism, projecting high-dimensional features into a lower-dimensional space and mapping them back to a higher-dimensional space. Additionally, the FA module combines sparse and dense prompting strategies to enhance feature interaction, improving the model’s segmentation ability for lesions with complex boundaries and irregular shapes. In experiments using the ISIC2017 and ISIC2018 datasets, the results show that SAM-FA significantly outperforms traditional models such as U-Net, MALUNet, SAM, and SAMed in key metrics such as Dice, Precision, and mIoU. This research provides an efficient and precise solution with significant potential for skin disease medical image segmentation tasks and offers valuable insights for future medical image processing technologies.