From local textures to global attention: a hybrid feature fusion network for skin lesion classification

Accurate detection of skin cancer detection using RGB images remains a challenge due to multiple factors including variability in lesion appearance, difference in skin types, and the clinical interpretability of the models. To address these challenges, we present a unified feature-fusion framework that integrates deep learning methods for accurate classification of dermoscopic skin lesions. Our proposed method consists of preprocessing in which we performed normalization, class aware selective augmentation, followed by feature-level fusion from three customized deep learning architectures i.e.; VGG-16 with adaptive layer configuration, ResNet-50 with dermatological feature enhancement, and a Vision Transformer (ViT) with dynamic patching. These extracted features are then fused to form a comprehensive feature vector and classification is done using an HDFNet which is a two-layer Deep Neural Network (DNN). We have trained and tested the proposed model on four publicly available datasets including ISIC 2019, ISIC 2020, PAD-UFES, and DermQuest DERMIS. We achieved classification accuracy of 94.5% and an AUC-ROC of 97%. Our proposed method outperforms the existing state-of-the-art models and also provide interpretable predictions supported by Grad-CAM-based visual explanations.