Ecologically sustainable benchmarking of AI models for histopathology
Deep learning (DL) holds great promise to improve medical diagnostics, including pathology. Current DL research mainly focuses on performance. DL implementation potentially leads to environmental consequences but approaches for assessment of both performance and carbon footprint are missing. Here, we explored an approach for developing DL for pathology, which considers both diagnostic performance and carbon footprint, calculated as CO<sub>2</sub> or equivalent emissions (CO<sub>2</sub>eq). We evaluated various DL architectures used in computational pathology, including a large foundation model, across two diagnostic tasks of low and high complexity. We proposed a metric termed 'environmentally sustainable performance' (ESPer), which quantitatively integrates performance and operational CO<sub>2</sub>eq during training and inference. While some DL models showed comparable diagnostic performance, ESPer enabled prioritizing those with less carbon footprint. We also investigated how data reduction approaches can improve the ESPer of individual models. This study provides an approach facilitating the development of environmentally friendly, sustainable medical AI.
Authors
- Jesper Kers (ORCID: https://orcid.org/0000-0002-2418-5279)
- N. Schmitz (ORCID: https://orcid.org/0009-0006-2200-1495)
- Martin Strauch (ORCID: https://orcid.org/0000-0001-6754-211X)
- Yu-Chia Lan (ORCID: https://orcid.org/0000-0002-8327-4642)
- Peter Boor (ORCID: https://orcid.org/0000-0001-9921-4284)
- Alireza Vafaei Sadr (ORCID: https://orcid.org/0000-0002-5733-6678)
- Tri Q. Nguyen (ORCID: https://orcid.org/0000-0001-6475-0706)
- David L. Hölscher (ORCID: https://orcid.org/0000-0001-8595-0694)
- Roman David Bülow (ORCID: https://orcid.org/0000-0002-8527-7353)
- Pourya Pilva (ORCID: https://orcid.org/0009-0001-0301-8738)
- Huong Thi Thanh Nguyen
- Leon Niggemeier (ORCID: https://orcid.org/0009-0007-9664-4490)
Institutions
- Pennsylvania State University (US)
- Leiden University Medical Center (NL)
- University Medical Center Utrecht (NL)
- Universitätsklinikum Aachen (DE)
- Amsterdam University Medical Centers (NL)
- University of Amsterdam (NL)
- RWTH Aachen University (DE)
Publication Details
- Journal
- npj Digital Medicine
- Published
- 2024-12-24
- DOI
- https://doi.org/10.1038/s41746-024-01397-x
- Citations
- 11
- Primary Topic
- AI in cancer detection
- Type
- article
- Field-Weighted Citation Impact
- 3.82
Funders
- Deutsche Forschungsgemeinschaft
- Universiteit van Amsterdam
- Bundesministerium für Bildung und Forschung
- Nierstichting
- RWTH Aachen University
- Gemeinsame Bundesausschuss