Evaluation of ChatGPT-4o and Gemini for gout management: a comparative analysis based on EULAR guidelines

Gout is the most common form of inflammatory arthritis, with a growing global prevalence and significant clinical burden. Although effective treatments are available, real-world implementation of guideline-based care remains suboptimal. This study aimed to comparatively evaluate the performance of two large language models (LLMs), ChatGPT-4o and Gemini 2.0 Flash, in responding to structured clinical questions derived from the 2020 and 2016 EULAR guidelines for the diagnosis and management of gout. Both models demonstrated moderate reliability and high response quality. Regarding guideline alignment, ChatGPT-4o provided fully aligned, complete, and accurate responses to 76.0% of the questions, whereas Gemini achieved this in 48.0%. Additionally, 8.0% of Gemini’s responses were entirely contradictory to the guidelines, while ChatGPT-4o had none. ChatGPT-4o significantly outperformed Gemini across reliability, quality, and alignment metrics. Readability assessments indicated that both models generated content requiring college-level comprehension. Both LLMs showed potential in supporting clinical decision-making for gout management. However, ChatGPT-4o consistently produced more accurate and guideline-concordant responses. While these tools may enhance clinical care and education, caution is warranted due to limitations in transparency and consistency.