Comparative Analysis of Six Large Language Models' Generative Ability for Scientific Technical Writing in Ophthalmologic Subspecialties

PurposeAs the adoption of large language models (LLMs) accelerates in healthcare, the evaluation of their accuracy must accelerate in tandem.This study compares six LLMs' ability to generate accurate, thorough, and highquality scientific technical writing in seven areas compared to a human-written control. MethodsFour prompts were created in ophthalmic subspecialties and presented to ChatGPT (4.0), Gemini (2.5 Pro), DeepSeek (DeepSeek-Prover-V2-671B), Grok (3.0), Microsoft Copilot, and OpenEvidence (2.0).Each model's performance was analyzed based on seven factors: medical accuracy, thoroughness and comprehensiveness, organization, literature support, speech efficiency, citation classification, and readability.Data analysis was conducted in Excel, graphed, and compared to a control. ResultsThe control texts outperformed each of the artificial intelligence (AI) models in medical accuracy, thoroughness and comprehensiveness, and organization.OpenEvidence uniquely matched the control in literature support, with 24 (100%) complete and accurate citations.Grok was the most speech-efficient; Gemini was the most verbose.The readability for each LLM was, on average, lower quality than required for scientific technical writing. ConclusionThis study shows that each AI model does not compare to a human expert's ability to generate scientific technical writing in a subspecialty of ophthalmology.Only OpenEvidence demonstrated citation accuracy meeting review article standards; however, this platform did not meet standards in other areas.The study concludes that current unpaid AI platforms cannot replace human expertise in creating high-level scientific literature in ophthalmology.