Comparative Analysis of Education Material Generated for Caregivers by Popular AI Chatbots on Palliative Care in Intensive Care Unit

INTRODUCTION: Over the years, palliative care in the Intensive Care Unit (ICU) has evolved from end-of-life care to a critical component of the ICU's management. The goal of palliative care in the ICU is to maximize the physical, psychological, and spiritual well-being of patients and their families facing life-limiting illnesses. Communication with caregivers, setting care goals, ensuring dignity in death, and sharing decisions are all part of this management. The caregivers of the patients with advanced chronic disease are the ones who are the most affected, sometimes even more than the patient himself. Caregivers often struggle with stress and anxiety while caring for their loved ones. In this era of the digital revolution, they take the support of technology to find answers to their queries to get some knowledge about the condition of their patients, and ask appropriate questions from care providers in the presence of certain time constraints. In this study, we generated educational material for the caregiver of a patient with advanced chronic disease admitted to an intensive care unit about palliative care in the intensive care unit. This study aims to evaluate the quality of text generated by the two most accessible and popular generative artificial Intelligence (AI) chatbots, ChatGPT and Google Gemini. The comparison will be done for the quality of the text generated across parameters such as readability, sentiment, accuracy, completeness, and suitability of the text. OBJECTIVES: The objective of our study was to compare the quality of the generated educational text by ChatGPT and Google Gemini. MATERIALS AND METHODS: The educational material was generated using ChatGPT, and Google Gemini in response to the query, “Generate educational material about palliative care in an intensive care unit for caregiver of a patient with advanced chronic disease admitted in an intensive care unit.” Readability tests and sentiment analysis were carried out using online tools validated and freely accessible. The expert panel of seven members will determine the accuracy and completeness of the information using a 5-point Likert scale. PEMAT analysis for printed text will also be carried out to determine understandability and actionability. Descriptive Statistics will be represented by the mean, median, standard deviation (SD), and range for scores across each parameter for both ChatGPT and Google Gemini. Comparative Analysis will be carried out by Paired t-test or Wilcoxon Signed-Rank Test, Paired t-test if data is normally distributed or Wilcoxon Signed-Rank Test if data violates normality. RESULTS: ChatGPT and Google Gemini generated text has shown similarity in average readability scores of 12.96, and 13.21, respectively, across the eight most popular readability formulas. The result implies the requirement of the same grade levels (College Level Entry) and age range (18–20 years) to understand the text information. Sentiment analysis revealed Google Gemini has negative sentiments (-39.1) compared to ChatGPT sentiment score (+7.8) which provides neutral sentiments. Accuracy & Completeness, actionability & understandability of the text are being conducted with the help of an expert panel of seven individual experts in given subjects and PEMAT analysis and results are awaited. DISCUSSIONS/CONCLUSIONS: To be completed after obtaining all the observations and results.