Leveraging Segmentation Maps to improve Skin Lesion Classification

We propose a novel approach for skin lesion classification that leverages a transformer architecture to integrate diverse clinical information (dermoscopic images, segmentation maps, and patient clinical information) for more accurate diagnosis.By incorporating binary semantic segmentation maps as input, we directly provide the model with border details critical for distinguishing between benign and malignant lesions.This integration improves classification performance compared to models that use only dermoscopic images or clinical data.To the best of our knowledge, this is the first application of segmentation maps to enhance skin lesion classification.Our experiments on the ISIC dataset yield promising results, highlighting the potential of combining advanced transformer models with multimodal data for improved dermatological diagnostics.