Human-Comparable Sensitivity of Large Language Models in Identifying Eligible Studies Through Title and Abstract Screening: 3-Layer Strategy Using GPT-3.5 and GPT-4 for Systematic Reviews
Our 3-layer screening method with GPT-4 demonstrated acceptable level of sensitivity and specificity that supports its practical application in systematic review screenings. Future research should aim to generalize this approach and explore its effectiveness in diverse settings, both medical and nonmedical, to fully establish its use and operational feasibility.
Authors
- Masahiro Takeshima (ORCID: https://orcid.org/0000-0003-0614-7524)
- Kentaro Matsui (ORCID: https://orcid.org/0000-0003-4538-5381)
- Tomohiro Utsumi (ORCID: https://orcid.org/0000-0002-4673-7230)
- Yumi Aoki (ORCID: https://orcid.org/0000-0002-2674-0707)
- Yoshikazu Takaesu (ORCID: https://orcid.org/0000-0002-9169-3249)
- Taku Maruki
Institutions
- Jikei University School of Medicine (JP)
- Kyorin University (JP)
- Akita University (JP)
- University of the Ryukyus (JP)
- St. Luke's International University (JP)
- National Center of Neurology and Psychiatry (JP)
- National Institute of Mental Health (JP)
Publication Details
- Journal
- Journal of Medical Internet Research
- Published
- 2024-08-16
- DOI
- https://doi.org/10.2196/52758
- Citations
- 34
- Primary Topic
- Machine Learning in Healthcare
- Type
- article
- Field-Weighted Citation Impact
- 11.81