Comparative Evaluation of the Information Quality, Reliability, and Comprehensibility of ChatGPT 4.0 and Gemini Advanced in Patients with Lumbar Disc Herniation

ABS TRACT Objective:The aim of this study was to evaluate the validity, reliability, and understandability of the responses provided by 2 publicly accessible large language model-based chatbots, ChatGPT 4.0 and Gemini Advanced, to the 26 most frequently searched questions about lumbar disc herniation (LDH).Material and Methods: The 26 most frequently searched queries related to LDH were retrieved from Google Trends.These queries were entered into the ChatGPT 4.0 and Gemini Advanced language models.The quality of information and reliability of the responses generated by ChatGPT 4.0 and Gemini Advanced were evaluated using the Quality Criteria for Consumer Health Information (DISCERN) and Global Quality Scale (GQS) measurement tools.All texts were also assessed for readability using the Flesch-Kincaid Grade Level (FKGL) and Flesch-Kincaid Reading Ease (FKRE) formulas.Results: When the responses about LDH generated by ChatGPT 4.0 and Gemini Advanced were evaluated using DIS-CERN analysis and GQS, no significant difference was found between the 2 groups.The FKGL was found to be significantly higher in the Gemini Advanced group than in the ChatGPT 4.0 group.However, the FKRE scores did not show a statistically significant difference between the 2 groups.The data produced by both language models were found to be poor in terms of information quality, reliability, and comprehensibility.Conclusion: Artificial intelligence-powered language models are still insufficient in providing valid and reliable information.Overreliance on AI models for obtaining medical information may lead to undesirable outcomes.Large language models may not fully replace the comprehensive medical evaluations and explanations conducted by healthcare professionals.