Utility of Generative Artificial Intelligence in Orthopaedic Surgery Patient Education Materials: A Scoping Review

Large language models (LLMs) are increasingly used to generate patient education materials (PEMs) and answer frequently asked questions (FAQs), though their accuracy and quality remain uncertain.This scoping review describes the accuracy, readability and quality of responses generated by LLMs to orthopaedic FAQs and PEMs, characterising the tools and prompting strategies used to assess them.A Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist-aligned scoping review of PubMed (NLM), Ovid MEDLINE® ALL and Embase databases (up to January 22, 2025) was conducted.Eligible studies used validated tools to assess LLMgenerated or modified PEMs or FAQ responses.Accuracy was assessed using qualitative ordinal scales; readability using validated indices including the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease Score (FRES); and quality using the DISCERN tool.Studies available only as abstracts, non-English publications, and studies not evaluating LLM outputs were excluded.Results were charted as reported, without pooling or formal risk-of-bias assessment.A total of 32 of 95 studies met inclusion criteria.Accuracy, readability and quality were reported in 16, 16 and 12 studies, respectively.Inaccuracies, omissions or potentially harmful content were reported in some studies; hallucination was explicitly assessed in 2/32 studies.Prompting was classifiable in 28/32 studies: 13/28 used zero-shot prompting alone, and 15/28 used additional prompting techniques.With zero-shot prompting alone, the mean FKGL was 12 to 15.A custom ChatGPT-4.0chat using zero-shot prompting, constraint prompting and reinforcement learning from human feedback had a lower FKGL (8.20 vs 13.99, P<0.001), higher FRES (60 vs 32, P<0.001) and improved DISCERN score (45.95 vs 38.41, P<0.001) than ChatGPT-4.0prompted by zero-shot prompting alone.Reported accuracy varied, and inaccuracies, omissions and potentially harmful content were reported.Readability exceeded recommended levels with zero-shot prompting alone.Hallucination was minimally assessed.Prompting was often basic or incompletely reported.Expert oversight remains essential; future research should use standardised evaluation methods and report prompts in full.