An agentic AI system enhances clinical detection of immune-related adverse events in a multi-phase validation study
Abstract Immune-related adverse events (irAEs) affect up to 40% of patients receiving immune checkpoint inhibitors, yet identification requires laborious, inconsistent manual chart review. Here we show that agentic artificial intelligence coupled with human verification could enhance efficiency, performance and consistency for irAE assessment. We developed and evaluated an agentic large language model system to extract the presence, temporality, severity grade, attribution and certainty of six irAE types from clinical notes. Retrospectively, the system achieved a macro-averaged F 1 of 0.92 for current-event detection and 0.66 for multi-class severity grading; self-consistency improved F 1 by 0.14. The best-performing configuration cost approximately US$0.02 per note. In prospective silent deployment over 3 months, the binary F 1 was 0.72–0.79. In a randomized crossover study of clinical trial staff, agentic assistance reduced annotation time by 2.71 min ( P < 0.001), increased complete-match accuracy (odds ratio 1.45; 95% confidence interval 1.01–2.08; P = 0.045) and improved interannotator agreement (Krippendorff’s α from 0.22–0.51 to 0.82–0.85).
Authors
- Hugo J.W.L. Aerts (ORCID: https://orcid.org/0000-0002-2122-2003)
- Benjamin Harris Kann (ORCID: https://orcid.org/0000-0002-4313-2754)
- Katherine C. Kellogg (ORCID: https://orcid.org/0000-0003-4372-3498)
- Bingxue K. Zhai
- Elad Sharon (ORCID: https://orcid.org/0000-0002-0044-9719)
- Ümit Topaloĝlu (ORCID: https://orcid.org/0000-0002-3241-8773)
- Jonathan Daniel Schoenfeld (ORCID: https://orcid.org/0000-0002-5119-0401)
- Biagio Ricciuti (ORCID: https://orcid.org/0000-0002-0651-2678)
- Jack Gallifant (ORCID: https://orcid.org/0000-0003-1306-2334)
- Danielle S. Bitterman (ORCID: https://orcid.org/0000-0003-0345-2232)
- Jeremy Lyle Warner (ORCID: https://orcid.org/0000-0002-2851-7242)
- Alexander Gusev (ORCID: https://orcid.org/0000-0002-7980-4620)
- Andrew Warrington (ORCID: https://orcid.org/0009-0009-4830-5683)
- Raymond H. Mak (ORCID: https://orcid.org/0000-0002-8754-0565)
- Tanna L. Nelson
- Matthew James Hadfield (ORCID: https://orcid.org/0000-0002-8768-0082)
- David Kozono (ORCID: https://orcid.org/0000-0002-8916-396X)
- Harry Hochheiser (ORCID: https://orcid.org/0000-0001-8793-9982)
- Guergana Savova (ORCID: https://orcid.org/0000-0002-5887-200X)
- Paul Catalano
- David C. Christiani (ORCID: https://orcid.org/0000-0002-0301-0242)
- Shan Chen (ORCID: https://orcid.org/0000-0001-7613-3940)
- Kee-Young Shin
- Patrick F. Doyle
- Bingyang Ye (ORCID: https://orcid.org/0009-0001-1696-4248)
- Paul Nguyen
- Joyce Guo
Institutions
- Boston Children's Hospital (US)
- Harvard University (US)
- University of Pittsburgh (US)
- Brown University (US)
- Rhode Island Hospital (US)
- Maastricht University (NL)
- Dana-Farber Cancer Institute (US)
- Dana-Farber/Harvard Cancer Center (US)
- National Cancer Institute (US)
- Mass General Brigham (US)
- Massachusetts Institute of Technology (US)
Publication Details
- Journal
- Nature Cancer
- Published
- 2026-10-29
- DOI
- https://doi.org/10.1038/s43018-026-01264-6
- Primary Topic
- Cancer Immunotherapy and Biomarkers
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Massachusetts General Hospital
- National Institutes of Health
- National Cancer Institute