Benchmarking Human–AI collaboration for common evidence appraisal tools
Current LLMs alone appraised evidence worse than humans. Human-AI collaboration may reduce workload for the second human rater for the assessment of reporting (PRISMA) and methodological rigor (AMSTAR) but not for complex tasks such as PRECIS-2.
Authors
- Julian Hirt (ORCID: https://orcid.org/0000-0001-6589-3936)
- Perrine Janiaud (ORCID: https://orcid.org/0000-0001-7684-8014)
- Ludwig Kappos (ORCID: https://orcid.org/0000-0003-4175-5509)
- John P. A. Ioannidis (ORCID: https://orcid.org/0000-0003-3118-6859)
- Lars G. Hemkens (ORCID: https://orcid.org/0000-0002-3444-1432)
- Tim Woelfle (ORCID: https://orcid.org/0000-0001-6279-4158)
Institutions
- University Hospital of Basel (CH)
- Laboratory for Biomedical Neurosciences (CH)
- Hospital Base (CL)
- Stanford Health Care (US)
- Stanford Medicine (US)
- Stanford University (US)
Publication Details
- Journal
- Journal of Clinical Epidemiology
- Published
- 2024-09-12
- DOI
- https://doi.org/10.1016/j.jclinepi.2024.111533
- Citations
- 24
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 2.89