Assessing Large Language Models in Neuroradiology: A Performance and Readability Comparison of ChatGPT and Gemini

Objective:To compare the performance, accuracy, readability, and clinical reliability of two large language models—ChatGPT (OpenAI) and Gemini (Google DeepMind)—in responding to neuroradiology-related questions. Methods:A cross-sectional model-comparison study was conducted using a standardized set of neuroradiology questions covering neuroanatomy, imaging physics, disease recognition, interpretation, and management. Both models received identical prompts, and responses were evaluated by fellowship-trained neuroradiologists using a structured scoring system assessing accuracy, completeness, clarity, and clinical safety. Error types were categorized, and statistical analyses were performed to compare performance. Additionally, readability metrics were applied to evaluate text complexity across multiple established indices. Results:Both models demonstrated strong baseline capability in neuroradiology knowledge. ChatGPT generally produced more detailed and stepwise reasoning, whereas Gemini provided concise responses with slightly more variability in accuracy. Readability analysis revealed that Gemini-generated texts were consistently easier to read and required a lower educational level for comprehension (p < .01 across most indices), while ChatGPT responses were more complex and required higher reading proficiency. No significant differences were found in SMOG or Linsear Write indices. Error patterns differed, with ChatGPT producing fewer hallucinations but more nonessential detail, while Gemini exhibited more omissions. Conclusions:ChatGPT and Gemini show promising utility as supportive tools in neuroradiology education and workflow tasks, but both exhibit limitations related to factual completeness and clinical safety. Gemini offers superior readability, while ChatGPT provides more structured reasoning. Neither model is sufficiently reliable for autonomous diagnostic or management decision-making. Continued evaluation using domain-specific benchmarks and multimodal datasets is recommended to guide safe clinical integration.