Evaluating the Reliability and Accuracy of an AI-Powered Search Engine in Providing Responses on Dietary Supplements: Quantitative and Qualitative Evaluation
This is the first study to assess Copilot's performance on dietary supplement information. Despite its authoritative appearance, Copilot frequently cited noncredible sources and provided ambiguous or inaccurate information. Its tendency to avoid definitive stances and align with perceived user expectations poses potential risks for health misinformation. These findings highlight the need for integrating health communication principles-such as transparency, audience empowerment, and informed choice-into the development and regulation of AI search engines to ensure safe public use.
Authors
- Yuriko Nishiie
- Hiroko Okada (ORCID: https://orcid.org/0000-0001-7877-9753)
- Tsuyoshi Okuhara (ORCID: https://orcid.org/0000-0002-6251-3587)
- Takahiro Kiuchi (ORCID: https://orcid.org/0000-0001-5934-0681)
- Mingxin Liu (ORCID: https://orcid.org/0000-0001-6320-544X)
- Yinghan Xu (ORCID: https://orcid.org/0000-0002-7811-453X)
- Ritsuko Shirabe (ORCID: https://orcid.org/0000-0001-6427-210X)
Institutions
- Waseda University (JP)
- Tokyo Medical University Hospital (JP)
- University of Tokyo Health Sciences (JP)
- Department of Health (IE)
- University hospital Medical Information Network (JP)
- The University of Tokyo (JP)
Publication Details
- Journal
- JMIR AI
- Published
- 2025-10-29
- DOI
- https://doi.org/10.2196/78436
- Citations
- 1
- Primary Topic
- Health Literacy and Information Accessibility
- Type
- article
- Field-Weighted Citation Impact
- 2.82