Evaluation Boundaries in Multimodal Skin Lesion Classification: Data Leakage, Patient Reuse, and Cross-Release Overlap

Multimodal skin-lesion classifiers combine dermoscopic images with clinical metadata, but reported gains may depend on how related images, lesions, patients, and dataset releases are separated between model development and testing. This study evaluates how these boundaries affect classification performance and the measured benefit of metadata fusion. HAM10000, PAD-UFES-20, and International Skin Imaging Collaboration (ISIC) releases are assessed using image-level, lesion-level, patient-level, content-based, and cross-release protocols. Image-only, metadata-only, late-fusion, metadata-gated, and cross-attention models are compared under matched settings with repeated seeds and confidence intervals. In HAM10000, with a fine-tuned ConvNeXt-Small encoder, random image-level splitting increased the macro-averaged F1 score (macro-F1) by 9.48 percentage points relative to lesion-disjoint evaluation, whereas cross-attention changed macro-F1 by only −0.34, −0.86, and +1.11 points across the tested protocols. On PAD-UFES-20, 32.22–40.46% of test patients under lesion-disjoint evaluation were also represented in training through different lesions, and cross-attention macro-F1 was 0.8085 under lesion-disjoint and 0.7124 under patient-disjoint evaluation. Retaining shared HAM10000 identities, and thereby their acquisition source, in the ISIC 2019 model-development pool was associated with an 18.2-point macro-F1 difference. For the datasets and models tested, evaluation boundaries produced larger macro-F1 changes than the fusion methods, highlighting the need to align the separation unit with the intended generalization target.

Authors

Institutions

Publication Details

Journal
Mathematics
Published
2026-11-19
DOI
https://doi.org/10.3390/math14224264
Primary Topic
Cutaneous Melanoma Detection and Management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Evaluation Boundaries in Multimodal Skin Lesion Classification: Data Leakage, Patient Reuse, and Cross-Release Overlap

H.A. Khan, Sitara Afzal, Jong‐Ha Lee
Mathematics
Cutaneous Melanoma Detection and Management
article

Evaluation Boundaries in Multimodal Skin Lesion Classification: Data Leakage, Patient Reuse, and Cross-Release Overlap

H.A. Khan, Sitara Afzal, Jong‐Ha Lee
article en

Abstract

Multimodal skin-lesion classifiers combine dermoscopic images with clinical metadata, but reported gains may depend on how related images, lesions, patients, and dataset releases are separated between model development and testing. This study evaluates how these boundaries affect classification performance and the measured benefit of metadata fusion. HAM10000, PAD-UFES-20, and International Skin Imaging Collaboration (ISIC) releases are assessed using image-level, lesion-level, patient-level, content-based, and cross-release protocols. Image-only, metadata-only, late-fusion, metadata-gated, and cross-attention models are compared under matched settings with repeated seeds and confidence intervals. In HAM10000, with a fine-tuned ConvNeXt-Small encoder, random image-level splitting increased the macro-averaged F1 score (macro-F1) by 9.48 percentage points relative to lesion-disjoint evaluation, whereas cross-attention changed macro-F1 by only −0.34, −0.86, and +1.11 points across the tested protocols. On PAD-UFES-20, 32.22–40.46% of test patients under lesion-disjoint evaluation were also represented in training through different lesions, and cross-attention macro-F1 was 0.8085 under lesion-disjoint and 0.7124 under patient-disjoint evaluation. Retaining shared HAM10000 identities, and thereby their acquisition source, in the ISIC 2019 model-development pool was associated with an 18.2-point macro-F1 difference. For the datasets and models tested, evaluation boundaries produced larger macro-F1 changes than the fusion methods, highlighting the need to align the separation unit with the intended generalization target.

MathematicsVol. 14(22)
Keimyung University (KR)
Cutaneous Melanoma Detection and Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon