Breaking Bones, Breaking Barriers: ChatGPT, DeepSeek, and Gemini in Hand Fracture Management
<b>Background</b>: Hand fracture management requires precise diagnostic accuracy and complex decision-making. Advances in artificial intelligence (AI) suggest that large language models (LLMs) may assist or even rival traditional clinical approaches. This study evaluates the effectiveness of ChatGPT-4o, DeepSeek-V3, and Gemini 1.5 in diagnosing and recommending treatment strategies for hand fractures compared to experienced surgeons. <b>Methods</b>: A retrospective analysis of 58 anonymized hand fracture cases was conducted. Clinical details, including fracture site, displacement, and soft-tissue involvement, were provided to the AI models, which generated management plans. Their recommendations were compared to actual surgeon decisions, assessing accuracy, precision, recall, and F1 score. <b>Results</b>: ChatGPT-4o demonstrated the highest accuracy (98.28%) and recall (91.74%), effectively identifying most correct interventions but occasionally proposing extraneous options (precision 58.48%). DeepSeek-V3 showed moderate accuracy (63.79%), with balanced precision (61.17%) and recall (57.89%), sometimes omitting correct treatments. Gemini 1.5 performed poorly (accuracy 18.97%), with low precision and recall, indicating substantial limitations in clinical decision support. <b>Conclusions</b>: AI models can enhance clinical workflows, particularly in radiographic interpretation and triage, but their limitations highlight the irreplaceable role of human expertise in complex hand trauma management. ChatGPT-4o demonstrated promising accuracy but requires refinement. Ethical concerns regarding AI-driven medical decisions, including bias and transparency, must be addressed before widespread clinical implementation.
Authors
- Mirco Pozzi (ORCID: https://orcid.org/0000-0001-6888-2191)
- Warren M. Rozen (ORCID: https://orcid.org/0000-0002-4092-182X)
- Yi Xie (ORCID: https://orcid.org/0000-0002-1416-5557)
- Ishith Seth (ORCID: https://orcid.org/0000-0001-5444-8925)
- Gianluca Marcaccini (ORCID: https://orcid.org/0000-0002-8621-8216)
- Roberto Cuomo (ORCID: https://orcid.org/0000-0002-8396-095X)
- Pietro Susini (ORCID: https://orcid.org/0009-0000-6168-0853)
Institutions
- University of Siena (IT)
- Peninsula Health (AU)
Publication Details
- Journal
- Journal of Clinical Medicine
- Published
- 2025-03-14
- DOI
- https://doi.org/10.3390/jcm14061983
- Citations
- 33
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 21.59