Transformer-enhanced multi-scale skin disease classification model using ResNet–MobileNet feature fusion and grad-CAM + + explainability

Skin diseases are common among more than 900 million people globally and are highly difficult to diagnose because of visual similarities between the diseases, as well as the variation of skin color. The purpose of this research is to create a multi-class skin disease classification model based on deep learning that provides interpretable predictions and includes fairness analysis across different skin tones. The proposed model that combines Residual Network-50 Layers (ResNet-50), to extract local features, and Mobile Network Version 2 (MobileNet-V2), to extract global features, to which the fusion of the features is conducted with the help of a Vision Transformer (ViT). Classification relies on a fully connected layer, whereas explainability depends on Gradient-weighted Class Activation Mapping++ (Grad-CAM++) to assist in it. The model is trained using a Kaggle dataset of 2000 dermoscopic images in five classes (Eczema, Basal cell carcinoma, Melanoma, Benign keratosis, Melanocytic nevi) that is curated and has stratified train, validation, and test splits. Also, independent datasets, such as HAM10000 and ISIC 2018, are externally validated to test generalization. Our model achieves a total accuracy of 95.3%, precision of 94%, recall of 95%, F1-score of 94%, Area Under the Receiver Operating Characteristic Curve (AUC-ROC) of 97, and Matthews Correlation Coefficient (MCC) of 90. The performance is constant with external validation, and it is slightly reduced in domain shift. The sensitivity was 97.8 in Melanoma detection. Accuracy depended on the skin tone, and 92.1% of the light, 90.5% medium, and 88.7% dark skin tones were accurate. Grad-CAM + + provides interpretable visual explanations, achieving a mean Intersection over Union (IoU) of 72.6% with expert masks and a clinical agreement of 89.3%, indicating good alignment with clinical annotations. The findings reveal that the proposed model offers competitive performance, better interpretability, and reasonable generalization across datasets. Nonetheless, more research is needed to decrease gaps in fairness and improve performance for various populations.