Beyond Aggregate Accuracy: Evaluating Fusion Strategies for Skin Lesion Classification Under Natural Class Imbalance

This preprint investigates multimodal skin lesion classification under the natural class imbalance of the HAM10000 dataset. It systematically compares three image–metadata fusion strategies: concatenation, Hadamard product, and self/cross-attention, using dermoscopic images alongside patient metadata such as age, sex, and anatomical localization. Across the evaluated configurations, ResNet-50 with Hadamard fusion achieves the highest accuracy and balanced accuracy, while EfficientNet-B4 with attention achieves the highest macro-AUC. The results demonstrate that fusion strategy substantially influences classification performance and that higher AUC does not necessarily correspond to stronger balanced performance under severe class imbalance. The study highlights the importance of evaluating multimodal medical image classifiers using complementary aggregate and class-sensitive metrics rather than relying on a single headline measure.