Competency of Large Language Models in Evaluating Appropriate Responses to Suicidal Ideation: Comparative Study
Current versions of 3 major LLMs demonstrated an upward bias in their evaluations of appropriate responses to suicidal ideation; however, 2 of the 3 models performed equivalent to or exceeded the performance of mental health professionals.
Authors
- Joshua Breslau (ORCID: https://orcid.org/0000-0002-1194-4643)
- Alyssa Halbisen
- Aaron Kofner (ORCID: https://orcid.org/0000-0001-6980-1218)
- Bradley D. Stein (ORCID: https://orcid.org/0000-0003-1544-458X)
- Fang Zhang (ORCID: https://orcid.org/0000-0002-8282-8738)
- Jonathan Cantor (ORCID: https://orcid.org/0000-0003-4468-833X)
- Olesya Baker (ORCID: https://orcid.org/0000-0001-9125-2761)
- Li Ang Zhang (ORCID: https://orcid.org/0000-0001-9468-2513)
- Ateev Mehrotra (ORCID: https://orcid.org/0000-0003-2223-1582)
- Ryan K. McBain (ORCID: https://orcid.org/0000-0003-0073-0348)
- Hao Yu (ORCID: https://orcid.org/0000-0001-6169-4243)
Institutions
- Brigham and Women's Hospital (US)
- RAND Corporation (US)
- Harvard University (US)
- Brown University (US)
- Harvard Pilgrim Health Care (US)
Publication Details
- Journal
- Journal of Medical Internet Research
- Published
- 2025-03-05
- DOI
- https://doi.org/10.2196/67891
- Citations
- 18
- Primary Topic
- Suicide and Self-Harm Studies
- Type
- article
- Field-Weighted Citation Impact
- 41.52