Evaluation of artificial intelligence (AI) chatbots for providing sexual health information: a consensus study using real-world clinical queries
Prompt-tuned AI chatbots demonstrated superior performance in providing sexual health information compared to base ChatGPT, with high safety scores particularly noteworthy. However, all AI chatbots showed susceptibility to generating incorrect information. These findings suggest the potential for AI chatbots as adjuncts to human healthcare providers for providing sexual health information while highlighting the need for continued refinement and human oversight. Future research should focus on larger-scale evaluations and real-world implementations.
Authors
- Eric P. F. Chow (ORCID: https://orcid.org/0000-0003-1766-0657)
- Jason J. Ong (ORCID: https://orcid.org/0000-0001-5784-7403)
- Nyi Nyi Soe (ORCID: https://orcid.org/0009-0001-4554-3549)
- Rashidur Rahman (ORCID: https://orcid.org/0009-0001-9902-4066)
- Phyu Mon Latt (ORCID: https://orcid.org/0000-0001-5880-5731)
- Ria Fortune
- Claire Randall
- Christopher Kincaid Fairley (ORCID: https://orcid.org/0000-0001-9081-1664)
- Ei Thu Aung (ORCID: https://orcid.org/0000-0002-2560-3233)
- Alicia J. King (ORCID: https://orcid.org/0000-0002-9430-9804)
- Kay Htaik (ORCID: https://orcid.org/0000-0003-1754-4920)
- Catriona S. Bradshaw (ORCID: https://orcid.org/0000-0002-6643-5678)
- Sheranne Dobinson
- David Lee (ORCID: https://orcid.org/0009-0006-5423-4525)
- Lei Zhang (ORCID: https://orcid.org/0000-0003-2343-084X)
- Matthew Deneen
Institutions
- The University of Melbourne (AU)
- London School of Hygiene & Tropical Medicine (GB)
- Melbourne Sexual Health Centre (AU)
- Alfred Health (AU)
- Monash University (AU)
Publication Details
- Journal
- BMC Public Health
- Published
- 2025-05-15
- DOI
- https://doi.org/10.1186/s12889-025-22933-8
- Citations
- 10
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 6.54
Funders
- Monash University
- Medical Research Council
- National Health and Medical Research Council