Reset: A Residual Set-Transformer Approach to Tackle the Ugly-Duckling Sign in Melanoma Detection

The dermatological concept of the Ugly-Duckling Sign (UDS) emphasizes the importance of comparing skin lesions within the same patient for enhanced diagnostic accuracy in melanoma detection, stating that atypical lesions are more likely to be cancers. However this concept is still underutilized in research, as most work on melanoma detection rely on classification ConvNets which lack the capacity to compare images together. Addressing this research gap, we introduce ReSeT (Residual Set-Transformer), a framework designed to compare skin lesions within patients during prediction. ReSeT comprises an encoder that takes individual images as input to generate embeddings, and a Set-Transformer with a residual prediction layer that compares these embeddings while predicting. We demonstrate that our architecture ReSeT significantly enhances performance compared to ConvNets and we highlighting the necessity of residual connections in the context of multi-output Transformers. We also observe that self-supervised encoders are able to generate embeddings of comparable quality to those of supervised models, showing their robustness and impact on image comparison tasks.