Chatbots and Social Media-Embedded AI Assistants Provide Inconsistent Information on the Use of Unregulated Peptides Marketed for Musculoskeletal Recovery

Background: Unregulated peptides like BPC-157 are heavily marketed for musculoskeletal recovery despite the lack of U.S. Food and Drug Administration approval and minimal evidence from human trials. With peptide content widely promoted on social media, patients increasingly consult artificial intelligence (AI) tools to evaluate claims regarding the efficacy of these products. Purpose: We sought to compare the accuracy, completeness, and readability of AI-generated information on unregulated peptides for musculoskeletal recovery across dedicated AI tools and social-media-embedded assistants. Methods: We performed a cross-sectional content analysis study of AI outputs in response to 4 standardized questions about unapproved peptide use. Seven dedicated AI tools (ChatGPT, Gemini, Claude, OpenEvidence, Elicit, Undermind, and Consensus.ai) and 4 social-media assistants (Instagram, Facebook, TikTok, and X) were queried in May 2026. Readability was assessed using a composite of 4 grade-level formulas. Two evidence-anchored questions were scored by 3 blinded reviewers (2 orthopedic surgeons and 1 immunopathologist) using the validated Quality Assessment of Medical Artificial Intelligence (QAMAI) instrument (6-30) for the 7 dedicated AI tools but not for the 4 social media assistants. Results: Mean QAMAI score across the dedicated tools was 21.6/30 (good), ranging from 17.8 (Undermind, fair) to 25.3 (Claude, very good). Among dedicated tools, pooled QAMAI domain scores were highest for accuracy (4.0/5) and lowest for completeness (3.3/5). Among social-media assistants, X and Facebook were accurate, TikTok was mixed, and Instagram (Meta AI) spread misinformation, claiming BPC-157 accelerates recovery and endorsing post-surgical use. All 11 tools produced outputs at the college level (mean grade: 13.9), above the sixth-to-eighth-grade target for patient materials, with no correlation between quality and readability. Conclusion: Our analysis suggests that AI tools cannot substitute for clinical counseling on unregulated peptides. The accuracy subscore was generally strong, but completeness and readability remained poor. Medical societies should partner with AI developers to standardize evidence-anchored responses and prevent patient harm. Level of Evidence: Level IV: observational content analysis study.

Authors

Institutions

Publication Details

Journal
HSS Journal® The Musculoskeletal Journal of Hospital for Special Surgery
Published
2026-11-19
DOI
https://doi.org/10.1177/15563316261503920
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Chatbots and Social Media-Embedded AI Assistants Provide Inconsistent Information on the Use of Unregulated Peptides Marketed for Musculoskeletal Recovery

Tracey L. Bonfield, Jacob G. Calcei, John M. Apostolakos, Dipankar Yettapu et al.
HSS Journal® The Musculoskeletal Journal of Hospital for Special Surgery
Artificial Intelligence in Healthcare and Education
article

Chatbots and Social Media-Embedded AI Assistants Provide Inconsistent Information on the Use of Unregulated Peptides Marketed for Musculoskeletal Recovery

Tracey L. Bonfield, Jacob G. Calcei, John M. Apostolakos, Dipankar Yettapu, Mahan Miryala, Varshal Patel
article en

Abstract

Background: Unregulated peptides like BPC-157 are heavily marketed for musculoskeletal recovery despite the lack of U.S. Food and Drug Administration approval and minimal evidence from human trials. With peptide content widely promoted on social media, patients increasingly consult artificial intelligence (AI) tools to evaluate claims regarding the efficacy of these products. Purpose: We sought to compare the accuracy, completeness, and readability of AI-generated information on unregulated peptides for musculoskeletal recovery across dedicated AI tools and social-media-embedded assistants. Methods: We performed a cross-sectional content analysis study of AI outputs in response to 4 standardized questions about unapproved peptide use. Seven dedicated AI tools (ChatGPT, Gemini, Claude, OpenEvidence, Elicit, Undermind, and Consensus.ai) and 4 social-media assistants (Instagram, Facebook, TikTok, and X) were queried in May 2026. Readability was assessed using a composite of 4 grade-level formulas. Two evidence-anchored questions were scored by 3 blinded reviewers (2 orthopedic surgeons and 1 immunopathologist) using the validated Quality Assessment of Medical Artificial Intelligence (QAMAI) instrument (6-30) for the 7 dedicated AI tools but not for the 4 social media assistants. Results: Mean QAMAI score across the dedicated tools was 21.6/30 (good), ranging from 17.8 (Undermind, fair) to 25.3 (Claude, very good). Among dedicated tools, pooled QAMAI domain scores were highest for accuracy (4.0/5) and lowest for completeness (3.3/5). Among social-media assistants, X and Facebook were accurate, TikTok was mixed, and Instagram (Meta AI) spread misinformation, claiming BPC-157 accelerates recovery and endorsing post-surgical use. All 11 tools produced outputs at the college level (mean grade: 13.9), above the sixth-to-eighth-grade target for patient materials, with no correlation between quality and readability. Conclusion: Our analysis suggests that AI tools cannot substitute for clinical counseling on unregulated peptides. The accuracy subscore was generally strong, but completeness and readability remained poor. Medical societies should partner with AI developers to standardize evidence-anchored responses and prevent patient harm. Level of Evidence: Level IV: observational content analysis study.

HSS Journal® The Musculoskeletal Journal of Hospital for Special Surgery
University Hospitals Cleveland Medical Center (US), Case Western Reserve University (US)
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon