Evaluating human-in-the-loop strategies for artificial intelligence-enabled translation of patient discharge instructions: a multidisciplinary analysis
Machine translation supported by artificial intelligence (AI) may enhance linguistically-concordant care for patients speaking languages other than English. This assessment of free-text inpatient discharge instructions in Arabic, Armenian, Bengali, simplified Chinese, Somali, and Spanish compared linguist, clinician, and family caregiver evaluations of translations generated by (1) ChatGPT-4o, (2) professional linguists, and (3) human-in-the-loop (AI-generated, professional linguist post-edited). Likert scales (1-5; higher is better) evaluated linguistic and clinical characteristics of each translation. ChatGPT-4o exhibited variable performance relative to professional translations, with poorest ratings for digitally underrepresented languages (Armenian and Somali). Conversely, human-in-the-loop translations achieved comparable, often better, outcomes to professional translations for all languages, (e.g., Armenian mean overall quality: 3.9 [95% CI 3.7-4.2] vs. professional 3.6 [3.4-3.9], p = 0.01), were most frequently preferred (46.5% vs. 28.4%) and had shorter mean translation time (7.1 [5.4-8.8] vs. 16.8 [13.7-19.9] min, p < 0.001). Human-in-the-loop strategies may enable safe, efficient, equitable machine translation application in clinical practice.
Authors
- Ryan Brewster (ORCID: https://orcid.org/0000-0003-0051-2623)
- Changjun Su (ORCID: https://orcid.org/0000-0003-0086-9155)
- Mondira Ray (ORCID: https://orcid.org/0000-0003-1248-6610)
- Maymona E Nageye
- Mohammad M. Sallam (ORCID: https://orcid.org/0000-0002-3528-6040)
- Amitra Hoq
- Alisa Khan (ORCID: https://orcid.org/0000-0002-1641-1061)
- Raffi Tachdjian (ORCID: https://orcid.org/0000-0002-5387-0013)
- Md Abdullah Al Mamoon
- Mourad Hana
- Martin E. Newell
- Osama Ibrahim
- Qing Li (ORCID: https://orcid.org/0000-0002-6777-855X)
- Gary Man-Kit Tse (ORCID: https://orcid.org/0000-0002-9405-9665)
- Nate Shaar
- M. Hoq Chowdhury
- Jonathan D. Hron (ORCID: https://orcid.org/0000-0003-2198-6174)
- Crystal Chang (ORCID: https://orcid.org/0000-0002-0647-7040)
- Angela L. Fan
- Priscilla Gonzalez
- Hoda Hassan
- Gerardo I. Pérez Soto
- Marwa Elborki
- Adiba Geeti
- Lucine Keseyan
- Arthur Ohannessian
- Ilan Rozen Eisenberg
- Hannah Lev
- Nicholas Kuzma
Institutions
- Cedars-Sinai Medical Center (US)
- Boston Medical Center (US)
- Boston Children's Hospital (US)
- Beth Israel Deaconess Medical Center (US)
- State University of New York (US)
- Kaiser Permanente (US)
- Harvard University (US)
- University of California, Los Angeles (US)
- SUNY Downstate Health Sciences University (US)
- The Ohio State University Wexner Medical Center (US)
- University of Mississippi Medical Center (US)
- NYU Langone Health (US)
- St. John Medical Center (US)
- Avalon University School of Medicine (CW)
- Center for Pain and the Brain (US)
- Boston Children's Museum (US)
- Mercy Medical Center (US)
- St. Christopher's Hospital for Children (US)
- Children's Hospital of Richmond at VCU (US)
- New York University (US)
- Stanford University (US)
- Icahn School of Medicine at Mount Sinai (US)
Publication Details
- Journal
- npj Digital Medicine
- Published
- 2025-10-24
- DOI
- https://doi.org/10.1038/s41746-025-02055-6
- Citations
- 3
- Primary Topic
- Interpreting and Communication in Healthcare
- Type
- article
- Field-Weighted Citation Impact
- 8.45
Funders
- Patient-Centered Outcomes Research Institute