Comparative Evaluation of a Medical Large Language Model in Answering Real-World Radiation Oncology Questions: Multicenter Observational Study
A state-of-the-art medical LLM can answer real-life questions from the clinical practice of radiation oncology similarly well as clinical experts regarding overall quality and potential harmfulness. Such LLMs can already be deployed within the local hospital environment at an affordable cost. While LLMs may not yet be ready for clinical implementation as general AI assistants, the technology continues to improve at a rapid pace. Evaluation studies based on real-life situations are important to better understand the weaknesses and limitations of LLMs in clinical practice. Such studies are also crucial to define when the technology is ready for clinical implementation. Furthermore, education for health care professionals on generative AI is needed to ensure responsible clinical implementation of this transforming technology.
Authors
- Paul Martin Putora (ORCID: https://orcid.org/0000-0002-4972-6115)
- Hathal Haddad (ORCID: https://orcid.org/0000-0002-5413-814X)
- Eleni Gkika (ORCID: https://orcid.org/0000-0001-5455-252X)
- Nikola Čihorić (ORCID: https://orcid.org/0000-0003-1534-8520)
- Florian Putz (ORCID: https://orcid.org/0000-0003-3966-2872)
- Fabio Dennstädt (ORCID: https://orcid.org/0000-0002-5374-8720)
- Susanne Rogers (ORCID: https://orcid.org/0000-0003-0094-1890)
- Lucas Mose (ORCID: https://orcid.org/0009-0006-0693-2716)
- Markus Glatzer (ORCID: https://orcid.org/0000-0003-2277-7577)
- Martin Maffei (ORCID: https://orcid.org/0009-0002-7559-4459)
- Lawrence Chiang
- Janna Hastings (ORCID: https://orcid.org/0000-0002-3469-4923)
- Sebastian M. Christ (ORCID: https://orcid.org/0000-0001-5330-3820)
- Nicolas Bachmann (ORCID: https://orcid.org/0000-0003-2576-822X)
- Oliver Riesterer (ORCID: https://orcid.org/0000-0002-9508-0546)
- Maiwand Ahmadsei (ORCID: https://orcid.org/0000-0002-0646-2762)
- Elena Riggenbach (ORCID: https://orcid.org/0000-0001-7278-923X)
- Daniel R. Zwahlen (ORCID: https://orcid.org/0000-0003-4359-7735)
- Daniel Hoefler (ORCID: https://orcid.org/0009-0001-3642-9592)
- Jan Caspar Peeken (ORCID: https://orcid.org/0000-0003-2679-9853)
- Dubravko Sinovcic
- Paul Windisch (ORCID: https://orcid.org/0000-0003-1040-4888)
- Daniel M. Aebersold (ORCID: https://orcid.org/0000-0002-9493-3834)
- Paul‐Henry Mackeprang (ORCID: https://orcid.org/0000-0002-1505-0137)
- Katarina Bryjova
- Roberto Gaio
- Max Schmerder (ORCID: https://orcid.org/0009-0009-3663-4412)
- Irina Filchenko (ORCID: https://orcid.org/0000-0002-9486-1017)
Institutions
- SIB Swiss Institute of Bioinformatics (CH)
- University of Bonn (DE)
- Friedrich-Alexander-Universität Erlangen-Nürnberg (DE)
- University of Zurich (CH)
- University of St.Gallen (CH)
- Heidelberg University (DE)
- University Hospital Bonn (DE)
- University Hospital of Bern (CH)
- University Hospital Heidelberg (DE)
- Centre Hospitalier Universitaire Vaudois (CH)
- TUM Klinikum (DE)
- Helmholtz Zentrum München (DE)
- Universitätsklinikum Erlangen (DE)
- Ospedale di Bolzano (IT)
- Kantonsspital Winterthur (CH)
- University Hospital of Zurich (CH)
- Heidelberger Institut für Radioonkologie (DE)
- University Children's Hospital Tübingen (DE)
- Comprehensive Cancer Center Erlangen (DE)
- Kantonsspital Aarau (CH)
- Deutsches Konsortium für Translationale Krebsforschung (DE)
- Technical University of Munich (DE)
- University of Lausanne (CH)
Publication Details
- Journal
- Journal of Medical Internet Research
- Published
- 2025-03-31
- DOI
- https://doi.org/10.2196/69752
- Citations
- 4
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 2.62