Standardized pre-consultation by a large language model agent vs ophthalmology residents: a randomized clinical trial
History-taking is central yet constrained by limited clinical resources. We designed a pre-consultation large language model (LLM) agent for history-taking using a dual-agent architecture with skills-engineered symptom-oriented logic trees, and conducted a randomized clinical trial at a tertiary ophthalmic hospital in Guangzhou, China. Between May 10 and June 10, 2025, 172 of 175 approached patients with non-emergency appointments were randomized 1:1 to this LLM agent or ophthalmology residents. Among randomized patients (median age 56 years; 55% female), the LLM agent achieved better pre-consultation quality, measured by MedHistory score (range 0–100; adjusted mean difference, 17.7 points; 95% CI, 14.1 to 21.3; P < 0.001). It also received higher patience and empathy ratings (5 vs 4 and 5 vs 3 points, respectively; both P < 0.001), and had longer interactions (median 11.2 vs 3.1 minutes; P < 0.001). Test recommendation performance did not differ significantly across stages. Exploratory analyses showed the agent achieved superior diagnostic accuracy from history alone (F1 score 0.85 vs 0.68; P < 0.001), whereas ocular signs produced greater improvement among residents (interaction effect -0.19; P = 0.003), significantly narrowing the gap. These findings suggest engineered LLM agents can standardize clinical data acquisition, supporting a hybrid workflow where clinicians prioritize their expertise for physical examination and diagnostic synthesis. Trial Registration: ClinicalTrials.gov, NCT06824389, 02/05/2025.
Authors
- Xiaohang Wu (ORCID: https://orcid.org/0000-0002-9398-4330)
- Meimei Dongye
- Jianyu Pang (ORCID: https://orcid.org/0009-0007-2483-6717)
- Shaowei Bi (ORCID: https://orcid.org/0009-0002-9372-6021)
- Zhenzhen Liu (ORCID: https://orcid.org/0000-0002-4045-7481)
- Jingrui Wang (ORCID: https://orcid.org/0000-0002-2806-7261)
- Wei Qiang Wang (ORCID: https://orcid.org/0000-0002-1430-1360)
- Mingjie Luo (ORCID: https://orcid.org/0000-0001-7482-4975)
- Haotian Lin (ORCID: https://orcid.org/0009-0003-1173-4202)
- Yunxi Lai
- Yue Wu
- Jingjing Chen
- Zhenzhe Lin
- Ling Jin
Institutions
- Sun Yat-sen University (CN)
- Hainan Eye Hospital (CN)
- Zhongshan Ophthalmic Center, Sun Yat-sen University
Publication Details
- Journal
- npj Digital Medicine
- Published
- 2026-10-05
- DOI
- https://doi.org/10.1038/s41746-026-03232-x
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00