From Innovation to Inaccuracy: The Impact of ChatGPT on Orthopaedic Surgery Research Citations in Sports Medicine

Purpose Chat Generative Pre-Trained Transformer (ChatGPT) has continued to become widely utilized in orthopaedic surgery due to its efficiency and ability to produce easily digestible information. Researchers have used ChatGPT to produce topic specific outlines and assist in research endeavors. The purpose of this study was to evaluate the validity of ChatGPT as a resource to orthopaedic researchers in conducting literature reviews for presentations or research papers. Methods The following prompt was input into ChatGPT: “Write an outline for an orthopaedic presentation about anterior cruciate ligament (ACL) tears, include ten sources.” The same prompt was then utilized for three of the most common sports medicine pathologies in the shoulder, hip, and knee, 9 total prompts. Additionally, prompts were input into both ChatGPT versions 3.5 and 4.0 (total citations, n=180) to evaluate if there was a difference in citation accuracy. References were categorized as follows: Does not exist, improperly cited, or properly cited. Results Of the 180 references provided by ChatGPT 4.0, 58/180 (32.3%) references did not exist, 36/180 (20%) were improperly cited, and 86/180 (47.8%) were properly cited. For the ChatGPT 3.5 searches, 29/90 (32.2%) did not exist, 18/90 (20%) were improperly cited, and 43/90 (47.8%) were properly cited. The ChatGPT 4.0 searches had the exact same breakdown of properly, improperly cited, and did not exist as ChatGPT 3.5. The most referenced journals of the properly cited sources included the American Journal of Sports Medicine (ASM), Journal of Bone and Joint Surgery (JBJS), Arthroscopy, and Clinical Orthopaedics and Related Research (CORR). Conclusion The increased utilization of ChatGPT should be used with caution especially when citing orthopaedic surgery literature. Approximately half of the sources were either improperly cited or did not exist, questioning the credibility of ChatGPT as an aid in generating orthopaedic literature citations.