Hybrid approach for skin lesion analysis: integrating modified U-Net segmentation with vision transformers for multi-class skin cancer detection

Skin cancer is a major global health concern, where early and accurate detection is crucial for patient survival. Traditional CNN-based methods in skin lesion classification face challenges, particularly with the complexity of spatial and semantic features. To address these issues, we propose a unique hybrid deep learning approach integrating vision transformer (ViT) and a modified U-Net model. ViT processes images as tokens instead of pixels, enabling superior feature extraction and classification. Preprocessing techniques, including non-local means filtering for denoising and unsharp masking for contrast enhancement, are applied to enhance model robustness. Our hybrid approach integrates U-Net for precise segmentation, achieving metrics such as IOU of 92.46%, AUC of 97.64%, and dice coefficient of 95.96%. ViT for classification achieves exceptional accuracy, precision, recall, and F1 score, all at 99%. Using the HAM10000 dataset, our method surpasses existing techniques, demonstrating remarkable effectiveness in skin cancer detection and classification.