Can ChatGPT Effectively Answer Patient Questions on Achilles Tendon Ruptures

Background ChatGPT uses artificial intelligence (AI) to provide educational and conversational responses to prompts input by the user. With its increasing popularity, patients now have easy and convenient access to information about common orthopedic injuries and procedures. Our goal was to determine if ChatGPT could appropriately answer frequently asked questions regarding an Achilles tendon (AT) rupture. Methods Ten well-known healthcare institution pages were used to identify ten of the most common and clinically relevant questions regarding AT rupture. These questions were input to ChatGPT 3.5 as individual prompts, and the responses were graded for accuracy and readability using the DISCERN score, the Journal of the American Medical Association (JAMA) Benchmark criteria, and the Flesch-Kincaid grade level score. Results Of the ten responses, the average DISCERN score was 39 (range: 26-46). One response was rated as “Very Poor,” four were “Poor,” and five were “Fair.” The JAMA Benchmark Criteria was zero for all responses. Lastly, the average Flesch-Kincaid grade level was 13.4 (range: 11.1-15.0), which correlates to a college graduate reading level. Conclusion ChatGPT delivered the information in a concise and accessible format. However, many responses required clarification to avoid misleading the reader. Furthermore, the lack of reliable citations hinders the chatbot’s reliability. The complex verbiage in the responses also hinder its use and inclusivity of all patient populations. Thus, patients should seek medical education on AT ruptures from a board-certified orthopedic surgeon.