A multimodal vision foundation model for clinical dermatology

Diagnosing and treating skin diseases require advanced visual skills across domains and the ability to synthesize information from multiple imaging modalities. While current deep learning models excel at specific tasks such as skin cancer diagnosis from dermoscopic images, they struggle to meet the complex, multimodal requirements of clinical practice. Here we introduce PanDerm, a multimodal dermatology foundation model pretrained through self-supervised learning on over 2 million real-world skin disease images from 11 clinical institutions across 4 imaging modalities. We evaluated PanDerm on 28 diverse benchmarks, including skin cancer screening, risk stratification, differential diagnosis of common and rare skin conditions, lesion segmentation, longitudinal monitoring, and metastasis prediction and prognosis. PanDerm achieved state-of-the-art performance across all evaluated tasks, often outperforming existing models when using only 10% of labeled data. We conducted three reader studies to assess PanDerm's potential clinical utility. PanDerm outperformed clinicians by 10.2% in early-stage melanoma detection through longitudinal analysis, improved clinicians' skin cancer diagnostic accuracy by 11% on dermoscopy images and enhanced nondermatologist healthcare providers' differential diagnosis by 16.5% across 128 skin conditions on clinical photographs. These results show PanDerm's potential to improve patient care across diverse clinical scenarios and serve as a model for developing multimodal foundation models in other medical specialties, potentially accelerating the integration of artificial intelligence support in healthcare.

Authors

Institutions

Publication Details

Journal
Nature Medicine
Published
2025-06-06
DOI
https://doi.org/10.1038/s41591-025-03747-y
Citations
46
Primary Topic
Cutaneous Melanoma Detection and Management
Type
article
Field-Weighted Citation Impact
48.09
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

A multimodal vision foundation model for clinical dermatology

Harald Kittler, Elisabetta Magnaterra, Zongyuan Ge, David Powell et al.
46 citations
Nature Medicine
Cutaneous Melanoma Detection and Management
48.09
article

A multimodal vision foundation model for clinical dermatology

Harald Kittler, Elisabetta Magnaterra, Zongyuan Ge, David Powell, Victoria Mar, Siyuan Yan, José Bañuls, Clare Primiero, Litao Yang, H. Peter Soyer, Philipp Tschandl, Monika Janda, Pascale Guitera, Peter M. Ferguson, Jennifer Nguyen, Simon Chong Wee See, Zhonghua Wang, C. Paul Bonnington, Cristina Vico‐Alonso, Aik Beng Ng, Lie Ju, Zhen Yu, G. Tan, Vincent Tang, Ming Hu
article en
46 citations

Abstract

Diagnosing and treating skin diseases require advanced visual skills across domains and the ability to synthesize information from multiple imaging modalities. While current deep learning models excel at specific tasks such as skin cancer diagnosis from dermoscopic images, they struggle to meet the complex, multimodal requirements of clinical practice. Here we introduce PanDerm, a multimodal dermatology foundation model pretrained through self-supervised learning on over 2 million real-world skin disease images from 11 clinical institutions across 4 imaging modalities. We evaluated PanDerm on 28 diverse benchmarks, including skin cancer screening, risk stratification, differential diagnosis of common and rare skin conditions, lesion segmentation, longitudinal monitoring, and metastasis prediction and prognosis. PanDerm achieved state-of-the-art performance across all evaluated tasks, often outperforming existing models when using only 10% of labeled data. We conducted three reader studies to assess PanDerm's potential clinical utility. PanDerm outperformed clinicians by 10.2% in early-stage melanoma detection through longitudinal analysis, improved clinicians' skin cancer diagnostic accuracy by 11% on dermoscopy images and enhanced nondermatologist healthcare providers' differential diagnosis by 16.5% across 128 skin conditions on clinical photographs. These results show PanDerm's potential to improve patient care across diverse clinical scenarios and serve as a model for developing multimodal foundation models in other medical specialties, potentially accelerating the integration of artificial intelligence support in healthcare.

Nature MedicineVol. 31(8)
Universitat de Miguel Hernández d'Elx (ES), The University of Queensland (AU), Royal Prince Alfred Hospital (AU), Princess Alexandra Hospital (AU), The Alfred Hospital (AU), Melanoma Institute Australia (AU), Monash Health (AU), Hospital General Universitario de Alicante Doctor Balmis (ES), University of Florence (IT), Monash University (AU), Medical University of Vienna (AT)
Openalex Percentile: Top 0%
Cutaneous Melanoma Detection and Management
48.09
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.