Evaluating the Accuracy of Responses by Large Language Models for Information on Disease Epidemiology
ChatGPT-4 outperformed in retrieving information on disease epidemiology compared to Bard and ChatGPT-3.5. However, all three LLMs presented inaccurate responses, including irrelevant, incomplete, or fabricated references. Such limitations preclude the utility of the current forms of LLMs in obtaining accurate disease epidemiology by researchers in the pharmaceutical industry, in academia, or in the regulatory setting.
Authors
- Monika Esser (ORCID: https://orcid.org/0000-0002-6835-859X)
- Sarah‐Jo Sinnott (ORCID: https://orcid.org/0000-0001-7586-686X)
- Juhaeri Juhaeri (ORCID: https://orcid.org/0000-0001-8235-8388)
- Corinne Jouquelet‐Royer
- Jiajie Zhang (ORCID: https://orcid.org/0000-0003-3928-8171)
- Kexin Zhu (ORCID: https://orcid.org/0000-0001-7173-6881)
- William A. Blumentals
- Anton Klishin
Institutions
- Babraham Institute (GB)
- Sanofi (France) (FR)
- Wyższa Szkoła Hotelarstwa i Gastronomii (PL)
- Sanofi (United States) (US)
- Sanofi (Germany) (DE)
- Sanofi (United Kingdom) (GB)
Publication Details
- Journal
- Pharmacoepidemiology and Drug Safety
- Published
- 2025-02-01
- DOI
- https://doi.org/10.1002/pds.70111
- Citations
- 5
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 3.27
Funders
- Sanofi