Explainable multimodal AI for skin lesion risk prediction via 3D imaging and clinical data

Accurate diagnosis of skin lesions remains challenging due to their morphological variability and the limitations of conventional diagnostic methods. In this study, we developed an explainable artificial intelligence (AI) framework that integrates three-dimensional total body photography (3D TBP) images with structured clinical data to classify and assess the risk of six common skin-lesion types. Using the ISIC 2024 dataset comprising 1,075 patients, 41 clinical and lesion-specific features were extracted and analyzed. A multinomial logistic-regression model was implemented for decision support, and model interpretability was assessed using Shapley Additive Explanations (SHAP) and Class Activation Maps (CAM). The clinical-only XGBoost model achieved moderate accuracy (basal cell carcinoma 78.6%, nevus 72.6%), while CNNs trained on 3D TBP images achieved 87.1% accuracy for nevus. The multimodal fusion model substantially outperformed unimodal approaches, achieving recall and F1 scores above 95% and Area Under the Curve (AUC) values exceeding 0.95 (0.98 for nevus and actinic keratosis), and ranked among the top-performing entries in the ISIC 2024 challenge (partial false-positive rate = 0.1734). The integrated scoring system, visualized through nomograms, identified key predictors such as visual_classifier and tbp_lv_symm_2axis. This interpretable multimodal AI framework enhances diagnostic accuracy and risk stratification, offering a transparent and clinically actionable tool for precision dermatology and early detection of skin cancer.