Quality and Accountability of ChatGPT in Health Care in Low- and Middle-Income Countries: Simulated Patient Study
Using simulated patients to mimic 9 established noncommunicable and infectious diseases, we assessed ChatGPT's performance in treatment recommendations for common diseases in low- and middle-income countries. ChatGPT had a high level of accuracy in both correct diagnoses (20/27, 74%) and medication prescriptions (22/27, 82%) but a concerning level of unnecessary or harmful medications (23/27, 85%) even with correct diagnoses. ChatGPT performed better in managing noncommunicable diseases than infectious ones. These results highlight the need for cautious AI integration in health care systems to ensure quality and safety.
Authors
- Ruopeng An (ORCID: https://orcid.org/0000-0001-9632-0209)
- Yuyi Yang (ORCID: https://orcid.org/0000-0002-7625-810X)
- Xiaojing Fan (ORCID: https://orcid.org/0000-0002-6704-0678)
- Yafei Si (ORCID: https://orcid.org/0000-0001-9334-8230)
- Xi Chen (ORCID: https://orcid.org/0000-0002-2058-0351)
- Xi Wang (ORCID: https://orcid.org/0000-0002-3087-2876)
- Sen Gong
- Jiaqi Zu
Institutions
- Zhejiang International Studies University (CN)
- Washington University in St. Louis (US)
- UNSW Sydney (AU)
- Yale University (US)
- Yale New Haven Health System (US)
- Duke Kunshan University (CN)
- New York University (US)
- Xi'an Jiaotong University (CN)
Publication Details
- Journal
- Journal of Medical Internet Research
- Published
- 2024-09-09
- DOI
- https://doi.org/10.2196/56121
- Citations
- 7
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.84