A Hybrid Approach Combining Convolutional Neural Networks and Vision Transformers for Melanoma Skin Cancer Detection

Melanoma is among the most aggressive and dangerous forms of skin cancer, arising from melanocytes, the cells that give skin its color. Early detection is vital, as identifying melanoma in its early stages greatly increases the chances of survival compared to later stages. Prompt diagnosis can help prevent the spread of the disease, simplify treatment, and enhance patient outcomes, highlighting the importance of advanced detection methods in dermatology. In this paper, a hybrid deep learning methodology is proposed that leverages the strengths of Convolutional Neural Networks (CNNs) for extracting local spatial features such as edges and textures, while Vision Transformers (ViTs) model global dependencies using self-attention mechanisms. This combination provides a holistic perception of skin lesion images. ResNet50, pre-trained on ImageNet, serves as the CNN backbone, while ViT processes images in patches to capture long-range dependencies. The outputs of both models are concatenated and passed through dense layers for improved classification reaching an accuracy of 94%.