Extending synthetic medical education: generating OSCE cases with LLMs

Objective Structured Clinical Examinations (OSCEs) are central to competency-based medical education, yet high-quality case development is resource-intensive. Large language models (LLMs) offer a scalable solution, but comprehensive OSCE case generation, including components such as candidate instructions, standardized patient scripts, and examiner marking schemes, is underexplored. We developed an LLM-based tool to generate complete OSCE stations. Three board-certified internists evaluated 24 generated cases across eight internal medicine diagnoses and three learner levels using a 12-item Likert instrument. Ratings were high across all domains; communication behaviors (4.64/5), EPA alignment (4.62/5), and overall value (4.61/5) performed best; guideline concordance scored lowest (4.24/5) with the greatest rater disagreement. As a feasibility study, these findings establish that whole, schema-validated OSCE stations can be generated from a single structured call at a quality experts rate as acceptable, while also identifying guideline concordance and difficult calibration as the domains most requiring faculty oversight -- providing reproducible infrastructure for, but not yet evidence of, effective EPA-aligned OSCE development.