Development and validation of an autonomous artificial intelligence agent for clinical decision-making in oncology
Clinical decision-making in oncology is complex, requiring the integration of multimodal data and multidomain expertise. We developed and evaluated an autonomous clinical artificial intelligence (AI) agent leveraging GPT-4 with multimodal precision oncology tools to support personalized clinical decision-making. The system incorporates vision transformers for detecting microsatellite instability and KRAS and BRAF mutations from histopathology slides, MedSAM for radiological image segmentation and web-based search tools such as OncoKB, PubMed and Google. Evaluated on 20 realistic multimodal patient cases, the AI agent autonomously used appropriate tools with 87.5% accuracy, reached correct clinical conclusions in 91.0% of cases and accurately cited relevant oncology guidelines 75.5% of the time. Compared to GPT-4 alone, the integrated AI agent drastically improved decision-making accuracy from 30.3% to 87.2%. These findings demonstrate that integrating language models with precision oncology and search tools substantially enhances clinical accuracy, establishing a robust foundation for deploying AI-driven personalized oncology support systems.
Authors
- Marie-Elisabeth Leßmann (ORCID: https://orcid.org/0009-0005-7029-0028)
- Daniel Truhn (ORCID: https://orcid.org/0000-0002-9605-0728)
- Isabella C. Wiest (ORCID: https://orcid.org/0009-0004-4848-614X)
- Maximilian Tschochohei
- Jacqueline Lammert (ORCID: https://orcid.org/0000-0002-0380-574X)
- Nikolaus Schultz (ORCID: https://orcid.org/0000-0002-0131-4904)
- Jan Clusmann (ORCID: https://orcid.org/0000-0003-2925-8438)
- Manuel Salto‐Tellez (ORCID: https://orcid.org/0000-0001-8586-282X)
- Dyke Ferber (ORCID: https://orcid.org/0009-0006-6195-9276)
- Georg Wölflein (ORCID: https://orcid.org/0000-0002-0407-7617)
- Sebastian Foersch (ORCID: https://orcid.org/0000-0002-4740-6900)
- Jakob Nikolas Kather (ORCID: https://orcid.org/0000-0002-3730-5348)
- Omar S. M. El Nahhas (ORCID: https://orcid.org/0000-0002-2542-2117)
- Dirk Jaeger (ORCID: https://orcid.org/0000-0002-1276-7802)
Institutions
- European Organisation for Rare Diseases (FR)
- Memorial Sloan Kettering Cancer Center (US)
- University of St Andrews (GB)
- Johannes Gutenberg University Mainz (DE)
- Heidelberg University (DE)
- University Hospital Heidelberg (DE)
- TUM Klinikum (DE)
- University Medical Center of the Johannes Gutenberg University Mainz (DE)
- Fresenius (Germany) (DE)
- National Center for Tumor Diseases (DE)
- Universitätsklinikum Aachen (DE)
- Royal Marsden Hospital (GB)
- University Medical Centre Mannheim (DE)
- Deutsches Konsortium für Translationale Krebsforschung (DE)
- Technical University of Munich (DE)
- Technische Universität Dresden (DE)
- RWTH Aachen University (DE)
Publication Details
- Journal
- Nature Cancer
- Published
- 2025-06-06
- DOI
- https://doi.org/10.1038/s43018-025-00991-6
- Citations
- 76
- Primary Topic
- Radiomics and Machine Learning in Medical Imaging
- Type
- article
- Field-Weighted Citation Impact
- 103.37
Funders
- National Institute for Health and Care Research
- Department of Health and Social Care
- European Commission
- Deutscher Akademischer Austauschdienst
- Deutsche Forschungsgemeinschaft
- Bundesministerium für Bildung und Forschung
- Bundesministerium für Gesundheit
- Deutsche Krebshilfe
- Deutschen Konsortium für Translationale Krebsforschung
- Gemeinsame Bundesausschuss
- HORIZON EUROPE Framework Programme