The OB-GYN Take on GPT: Objective Assessment of Artificial Intelligence Models in Patient Education

Large language models (LLMs), a subset of generative artificial intelligence, have garnered significant attention in healthcare. While existing literature has examined the role of LLMs in medical education, primarily focusing on their ability to answer examination questions, few studies have investigated their potential in patient education and assessed the role of LLMs in medical education, few studies evaluate their role in patient education, and none have specifically evaluated their use in OB-GYN. Using a standardized rubric, we assessed the responses of three LLMs and the American College of Obstetricians and Gynecologists (ACOG) on 10 clinical questions that reflect common patient concerns in OB-GYN, which broadly represent common patient queries in OB-GYN. CoPilot by Microsoft demonstrated higher scores by expert reviewers; however, evaluators were more likely to recommend ACOG frequently asked questions (FAQs) or ChatGPT to patients when providing clinical counseling. This study supports the assertion that LLMs have a role in patient education and counseling in OB-GYN. .