Assessing Chat Generative Pre-training Transformer’s Proficiency in Identifying, Diagnosing, and Managing Orthopedic Fractures

Abstract Background: The exploration of Chat generative pre-training transformer’s (GPT’s) applications in medicine is gaining momentum, with artificial intelligence, particularly ChatGPT, showing promise in enhancing orthopedic care. ChatGPT can assist clinicians by providing relevant information based on patient symptoms, medical history, and radiological findings, aiding in differential diagnosis, and suggesting appropriate imaging modalities. This study focuses on evaluating the effectiveness of ChatGPT-4 in diagnosing and managing common orthopedic fractures. Methods: This study involved inputting a diverse set of fracture images into the ChatGPT-4 model. The process commenced by prompting ChatGPT with four questions: “What does the radiograph show?” The second prompt instructed ChatGPT with the actual diagnosis, followed by inquiries on how a clinician should manage the condition and potential complications. All generated responses underwent grading by two authors (musculoskeletal radiologist and orthopedic trainee), utilizing a 5-point Likert scale. Intraclass correlation coefficient (ICC) analysis measurements were performed to assess inter-rater reliability. Descriptive statistical analysis was then employed to provide a comprehensive summary of the study’s findings. Results: ChatGPT demonstrated limitations in identifying diagnoses based on inputted X-ray images. However, it excelled in providing comprehensive information about fracture management and potential complications. There was excellent interobserver reliability with a kappa of 0.9. Conclusion: Our study underscores the utility of ChatGPT as a valuable tool for aiding in the management of common fractures, offering a comprehensive overview of diagnosis, management, and potential complications. The findings highlight its potential role as a supplementary resource in orthopedic practice.