AI assistants for cancer pain management: A comparative evaluation of ChatGPT and Gemini responses in terms of response quality and readability

BackgroundCancer pain is common and distressing in oncology patients, who often seek management information via online and AI-based tools.ObjectiveThis study aimed to evaluate the quality and readability of responses provided by ChatGPT-4 and Gemini-2 to frequently asked questions regarding cancer painMethodsOn April 15, 2025, responses were collected from each artificial intelligence (AI) model using a set of frequently asked questions about cancer pain. These questions were selected based on expert input from oncology and pain management specialists. A total of ten questions were asked, and the responses were evaluated by eleven independent experts using a four-point Likert scale assessing accuracy, completeness, relevance, and clinical usefulness. Readability levels were analyzed using the Flesch-Kincaid Grade Level via WordCalc software.ResultsAccording to statistical analyses, significant differences were found in questions 2 (z = -2.583, p = 0.010), 3 (z = -2.927, p = 0.003), 5 (z = -2.583, p = 0.010), 7 (z -2.693, p = 0.007), 8 (z = -2.820, p = 0.005) and 9 (z = -2.529, p = 0.011). On the other hand, no statistically significant difference was found in questions 1, 4, 6 and 10, which shows that the models produced answers with similar quality levels for some questions.ConclusionChatGPT-4 produces content across a more consistent range of reading levels, whereas Gemini-2 shows a wider variation in reading levels and may be more sensitive to different types of prompts. The use of AI models in responding to cancer pain queries may contribute to better patient education and potentially support clinical decision-making in pain management.