A Novel Playbook for Pragmatic Trial Operations to Monitor and Evaluate Ambient Artificial Intelligence in Clinical Practice
We provide a publicly available framework and protocols to help safely implement ambient AI in health care. Innovations include an embedded pragmatic trial design, human factors engineering, compliance-driven feedback loops, and real-time monitoring to support deployment, ensuring fidelity before initiation of the clinical trial. (Funded by the University of Wisconsin Hospital and Clinics and the National Institutes of Health Clinical and Translational Science Award; NIH/ NCATS UL1TR002737; ClinicalTrials.gov number, NCT06517082.).
Authors
- Anne Gravel Sullivan (ORCID: https://orcid.org/0009-0009-7972-6907)
- Peter Kleinschmidt (ORCID: https://orcid.org/0000-0003-1672-6845)
- Joel Gordon (ORCID: https://orcid.org/0000-0001-7619-9384)
- Brian W. Patterson (ORCID: https://orcid.org/0000-0002-4584-3808)
- David Kunstman (ORCID: https://orcid.org/0009-0003-2816-2175)
- Kayla K. Lemmon (ORCID: https://orcid.org/0009-0001-9965-5351)
- Mary Ryan (ORCID: https://orcid.org/0000-0002-9137-1549)
- Frank Liao (ORCID: https://orcid.org/0000-0002-7577-7530)
- Michael Oberst (ORCID: https://orcid.org/0000-0003-1720-6702)
- Graham Wills (ORCID: https://orcid.org/0009-0005-3284-873X)
- Felice Resnik (ORCID: https://orcid.org/0000-0003-1494-1357)
- Leigh A. Mrotek (ORCID: https://orcid.org/0000-0001-6559-934X)
- Majid Afshar (ORCID: https://orcid.org/0000-0002-6368-4652)
- Tom Brazelton (ORCID: https://orcid.org/0000-0003-2840-3285)
- Mariah A. Quinn (ORCID: https://orcid.org/0000-0001-7534-739X)
- Elizabeth S. Burnside (ORCID: https://orcid.org/0000-0002-6600-435X)
- Stacy Rasmussen (ORCID: https://orcid.org/0009-0009-2202-0911)
- Heidi Twedt (ORCID: https://orcid.org/0000-0002-7091-0152)
- Cherodeep Goswami (ORCID: https://orcid.org/0009-0008-9805-4694)
- John Long (ORCID: https://orcid.org/0009-0003-9384-5386)
- Taran Shah
- Kirsten Abramson (ORCID: https://orcid.org/0000-0001-8491-3138)
- Josie Hintzke (ORCID: https://orcid.org/0009-0001-6840-1340)
- Anthony Stordalen
- Jason Dambach (ORCID: https://orcid.org/0009-0001-2354-230X)
Institutions
- University of Wisconsin–Madison (US)
- UW Health University Hospital (US)
- Johnson University (US)
- University of Wisconsin Health (US)
- Wisconsin Division of Public Health (US)
- Philadelphia University (US)
Publication Details
- Journal
- NEJM AI
- Published
- 2025-08-28
- DOI
- https://doi.org/10.1056/aidbp2401267
- Citations
- 13
- Primary Topic
- Electronic Health Records Systems
- Type
- article
- Field-Weighted Citation Impact
- 38.19
Funders
- National Institutes of Health
- National Center for Advancing Translational Sciences