Generating dermatopathology reports from gigapixel whole slide images with HistoGPT
Histopathology is the reference standard for diagnosing the presence and nature of many diseases, including cancer. However, analyzing tissue samples under a microscope and summarizing the findings in a comprehensive pathology report is time-consuming, labor-intensive, and non-standardized. To address this problem, we present HistoGPT, a vision language model that generates pathology reports from a patient's multiple full-resolution histology images. It is trained on 15,129 whole slide images from 6705 dermatology patients with corresponding pathology reports. The generated reports match the quality of human-written reports for common and homogeneous malignancies, as confirmed by natural language processing metrics and domain expert analysis. We evaluate HistoGPT in an international, multi-center clinical study and show that it can accurately predict tumor subtypes, tumor thickness, and tumor margins in a zero-shot fashion. Our model demonstrates the potential of artificial intelligence to assist pathologists in evaluating, reporting, and understanding routine dermatopathology cases.
Authors
- Sophia J. Wagner (ORCID: https://orcid.org/0000-0003-3763-2282)
- Alexander Böhner (ORCID: https://orcid.org/0000-0003-1590-0805)
- Francesco Ciompi
- Nneka I. Comfere (ORCID: https://orcid.org/0000-0002-8001-2639)
- Dennis H. Murphree (ORCID: https://orcid.org/0000-0002-7800-455X)
- Brenna C. Novotny (ORCID: https://orcid.org/0000-0003-4732-1736)
- Valentin Koch (ORCID: https://orcid.org/0009-0003-9678-374X)
- Avital L. Amir
- Stephan A. Braun (ORCID: https://orcid.org/0000-0002-5600-0195)
- Annette Feuchtinger (ORCID: https://orcid.org/0000-0003-4324-2547)
- Daan Geijs (ORCID: https://orcid.org/0000-0003-3984-1766)
- Carsten Marr (ORCID: https://orcid.org/0000-0003-2154-4552)
- Valerio Lupperger (ORCID: https://orcid.org/0000-0002-0112-2283)
- Robert Kaczmarczyk (ORCID: https://orcid.org/0000-0002-8570-1601)
- Judith Lefkes (ORCID: https://orcid.org/0000-0003-0813-4049)
- Geert Litjens (ORCID: https://orcid.org/0000-0003-1554-1291)
- M. R. D'Amato (ORCID: https://orcid.org/0009-0002-2258-0715)
- Tingying Peng (ORCID: https://orcid.org/0000-0002-7881-1749)
- Paul Schmidle (ORCID: https://orcid.org/0000-0002-4257-1659)
- Tilo Biedermann (ORCID: https://orcid.org/0000-0002-5352-5105)
- Manuel Tran (ORCID: https://orcid.org/0009-0004-9431-754X)
- Kilian Eyerich (ORCID: https://orcid.org/0000-0003-0094-2674)
- Ruifeng Guo (ORCID: https://orcid.org/0000-0003-3345-0502)
- Chen Wang (ORCID: https://orcid.org/0000-0003-2638-3081)
- Antien L Mooyaart
- Heather D Hardway
Institutions
- Mayo Clinic (US)
- Radboud University Nijmegen (NL)
- University of Freiburg (DE)
- University Medical Center Freiburg (DE)
- Erasmus MC (NL)
- WinnMed (US)
- Radboud University Medical Center (NL)
- Helmholtz Zentrum München (DE)
- Munich Leukemia Laboratory (Germany) (DE)
- University Hospital Münster (DE)
- Oncode Institute (NL)
- Mayo Clinic in Arizona (US)
- Mayo Clinic in Florida (US)
- Heinrich Heine University Düsseldorf (DE)
- Technical University of Munich (DE)
Publication Details
- Journal
- Nature Communications
- Published
- 2025-05-27
- DOI
- https://doi.org/10.1038/s41467-025-60014-x
- Citations
- 11
- Primary Topic
- AI in cancer detection
- Type
- article
- Field-Weighted Citation Impact
- 27.29
Funders
- European Commission