A Hybrid Deep Learning Approach for Automatic Melanoma Diagnosis: Integrating Convolutional Neural Networks with Vision Transformers

The most lethal type of skin lesion is melanoma. The likelihood of survival for melanoma is significantly increased by early detection. Nevertheless, a number of characteristics, such as diminished contrast between the skin and abrasion and visual similarities between non-melanoma and melanoma, make it challenging to identify melanoma with accuracy. The novel melanoma recognition method described in this study consists of three primary phases: preprocessing, segmentation and classification. To maximize image quality and get the data ready for segmentation, the preprocessing step included denoising techniques, grayscale conversion, Gaussian thresholding, and Contrast Limited Adaptive Histogram Equalization (CLAHE). Three widely used edge detection algorithms-Canny, Sobel, and Prewitt-were applied to the segmentation process in order to identify lesion boundaries and extract features unique to melanoma. Performance criteria such as Root Mean Square Error (RMSE), Structural Similarity Index (SSIM), and Peak Signal-to-Noise Ratio (PSNR) were used to assess the accuracy of the segmentation. Based on the comparison analysis, Sobel was found to be the best algorithm for correctly identifying melanoma lesions. The outcomes showed how well the preprocessing methods worked to enhance image quality and how well the segmentation algorithms identified melanoma lesions. This research's subsequent phase will concentrate on classifying images using deep learning models. The segmented images are used to train deep learning model namely Convolutional Neural Networks with Vision Transformer. Superior performance is obtained in terms of accuracy, precision, recall, and F1 score when denoising, segmentation, and classification techniques are combined. This highlights the potential of deep learning and image processing fusion in improving skin disease recognition and performance of classification systems.