Comparative Analysis of Readability Levels of Hand Surgery Patient Information on the ASSH Website and Artificial Intelligence

Background: The American Medical Association (AMA) recommends patient education materials (PEMs) be written at a sixth-grade level to ensure accessibility. This study aims to compare hand surgery topics from the American Society for Surgery of the Hand (ASSH) website and artificial intelligence (AI)-generated PEMs to evaluate their alignment with the AMA’s recommendations. Methods: Ninety-eight hand surgery topics from the ASSH Hand Care “The Upper Extremity Expert” website were analyzed. Artificial intelligence (AI) platforms (Google Gemini 2.0 Flash and ChatGPT-4) were asked to: (1) create original PEMs based on ASSH website topics; (2) rewrite the ASSH topics at a sixth-grade reading level; and (3) create original PEMs over the ASSH topics at a sixth-grade reading level. Readability was assessed using the Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL) scores. Results: The ASSH website and AI-generated PEMs initially exceeded the sixth-grade readability level. Original ASSH PEMs ranged from a readability of 10th to 12th-grade (FRE 57.5 [6.6]; FKGL 9.4 [1.2]). Google Gemini and ChatGPT’s initial PEMs were at the college level (Google Gemini: FRE 49.7 [7.5]; FKGL 9.6 [1.1] vs ChatGPT: FRE 47.4 [9.1]; FKGL 10.4 [1.4]). Artificial intelligence–rewritten ASSH content had improved readability, and AI-generated PEMs at a sixth-grade readability had lower word counts, higher FRE, and lower FKGL than original ASSH PEMs (all P < .001). Conclusions: Artificial intelligence platforms can be used as a tool to create hand surgery PEMs at the recommended sixth-grade reading level. Future efforts are needed to ensure the accuracy of AI-generated PEMs and accessibility to patients of varying health literacy for informed decision-making.