Abstract 4144833: Large Language Models Improve Readability in Primary Responses to Coronary Artery Bypass Graft Questions for Patient Education

Introduction: The advent of Large Language Models (LLMs) such as ChatGPT and Google Gemini yields a new paradigm for patient education. Coronary artery bypass grafts (CABGs) are open-heart surgeries that naturally gives to many questions in potential patients. Prior research suggested that patient education material should be written at a 6 th grade reading level maximally to be comprehended by the majority of the US adult population. However, most education material is written at a much higher literacy level. Now with the increasing use of LLMs, the potential for informing patients at an appropriate readability level exists. Research Question: Can ChatGPT and Google Gemini improve the readability of their primary responses to patient education questions about Coronary Artery Bypass Grafts to a 6 th grade reading level? Methods: We obtained 80 questions from 4 independent participants regarding information they would want to know prior to undergoing a CABG procedure. Questions that could not be answered using public information, required clinician input, and repeats were filtered out leaving 20 questions. These questions were asked to ChatGPT 3.5 and Google Gemini. Then both LLMs were asked to answer the question at a 6 th grade level within the same conversation. A new conversation was created each of the 20 questions. All responses were then evaluated using a Flesch-Kincaid calculator where the Flesch-Kincaid reading ease score (FKRE) and the Flesch-Kincaid grade level (FKGL) were obtained. Data analysis was performed using Microsoft Excel and Graphpad Prism. Results: For the ChatGPT responses, the average FKRE was 31.865 and FKGL was 12.545. Following the 6 th grade level request for ChatGPT, the average FKRE was 74.57 (p < 0.0001) and FKGL was 5.93 (p < 0.0001). For the Google Gemini responses, the average FKRE was 46.395 and FKGL was 10.545. Following the 6 th grade level request for Google Gemini, the average FKRE was 72.945 (p < 0.0001) and FKGL was 6.76 (p < 0.0001). Conclusion: Both LLMs significantly increased the readability of their responses and did so to a 6 th grade level. However, ChatGPT was closer than Google Gemini in reaching the desired FKGL. Nonetheless, our work showed that both LLMs could successfully work with CABG information and yield responses at a readability level befitting the US population. Future studies need to assess the accuracy of this information for LLMs to have practical use in cardiac surgery patient education.