Large Language Models in Shoulder Dislocation: Performance Across Medical and Rehabilitation Domains

Background/Objectives: Large language models (LLMs) are increasingly used to obtain health information about musculoskeletal conditions. However, their performance across different types of clinical questions remains insufficiently understood. This study evaluated the accuracy, quality, information reliability, and readability of responses generated by three widely used LLMs (ChatGPT 5.2, Gemini 3, and DeepSeek V3.2) to questions about first-time traumatic anterior shoulder dislocation, with separate analyses of medical and rehabilitation questions. Methods: Sixty-five clinically relevant questions were categorized into Medical and Safety (n = 28) and Rehabilitation (n = 37) domains. Responses were independently evaluated by domain experts using a 5-point accuracy scale, the Global Quality Scale (GQS), modified DISCERN (mDISCERN), and Flesch Reading Ease (FRE). Model differences were analyzed using Friedman tests with post hoc pairwise comparisons, and inter-rater agreement was assessed using intraclass correlation coefficients. Results: In the Medical and Safety Domain, the models differed significantly in accuracy (p < 0.001), quality (p < 0.001), information reliability (p = 0.004), and readability (p = 0.011). ChatGPT 5.2 and Gemini 3 had higher accuracy and GQS scores than DeepSeek V3.2, while DeepSeek V3.2 showed significantly higher mDISCERN scores than ChatGPT 5.2. Gemini 3 demonstrated significantly higher FRE scores than DeepSeek V3.2. In the Rehabilitation Domain, no significant differences were found in accuracy, quality, or information reliability (all p > 0.05), whereas readability differed significantly (p < 0.001), with ChatGPT 5.2 and Gemini 3 showing higher FRE scores than DeepSeek V3.2. Median accuracy and GQS scores were 5 for all three models in the Rehabilitation Domain, suggesting a potential ceiling effect. Readability was relatively difficult across the models, with no statistically significant correlations between FRE scores and accuracy, GQS, or mDISCERN scores. Inter-rater agreement ranged from moderate to excellent. Conclusions: LLMs provided generally accurate and high-quality information about first-time traumatic anterior shoulder dislocation. In separate domain-specific analyses, significant differences among models were observed in accuracy, quality, information reliability, and readability for medical and safety questions. For rehabilitation questions, statistically significant differences were detected only in readability. These findings do not establish a statistically significant difference in model performance between the two clinical domains. LLM-generated information should be interpreted cautiously, particularly when addressing medical decisions and patient safety.