Improving Skin Lesion Detection with Transformer-Based Architectures

This article describes the methodology for adjusting and comparing eleven variants of Transformer architectures for the classification of skin lesions using images: five variants of Google’s Vision Transformer (ViT) and six variants of Microsoft’s Swin Transformer. We present the methodology used to achieve these results, which includes meticulous hyperparameter tuning and a robust data augmentation strategy to address the class imbalance problem. This approach allowed us to surpass the state of the art on the DermaMNIST dataset with respect to CNN-based models, and achieve very competitive results on the ISIC Challenge 2019 dataset with respect to Transformer-based models. In addition, we employed the CheferCAM method to provide visual explanations that identify the most influential image regions in the models’ predictions.