Feasibility of an AI-simulated patient encounter delivered in group and individual learning formats for first-year medical students: a two-cohort pilot study
Generative artificial intelligence (AI) and large language model (LLM)–based simulated patients are being evaluated for problem-based learning and case-based learning in undergraduate medical education, but delivery formats are often under-specified and outcome measurement is inconsistently reported. We assessed the feasibility and acceptability of a single LLM-driven, AI-simulated patient encounter delivered in two formats—a facilitated whole-group session and a self-paced individual session—to first-year medical students (MS1), and we describe the development and provisional status of a learner-perception survey instrument. Two non-overlapping, non-randomized MS1 cohorts at one United States medical school completed the same case (Group, n = 7; Individual, n = 12) and the Behavioral Safety Language Scale–Provisional (BSLS-P) before and after the encounter. Pre-to-post change was tested with Wilcoxon signed-rank exact tests, with Benjamini–Hochberg false discovery rate (FDR) correction across subscale comparisons. Feasibility outcomes (recruitment, encounter completion, survey completion, technical failures, and encounter duration) and acceptability outcomes (post-encounter engagement ratings, delivery-format preference, and open-ended comments) are reported descriptively. Of 72 students invited, 19 (26%) enrolled; all completed the encounter and paired pre/post surveys (100%), with no technical failures (no encounter was terminated and no participant was excluded), although learners reported intermittent technical problems that interrupted but did not end encounters; the group session lasted approximately 35 min and individual encounters a median of 31 min (range 17–83). Pre-to-post shifts were small and imprecise; in the Group cohort, median engagement changed by + 0.33 points (Wilcoxon r = 0.13, 95% CI 0.00 to 0.90). The largest effect—a psychological-safety gain in the Individual cohort (Wilcoxon r = 0.62)—was significant before correction ( P =0.026) but not after (adjusted P =0.20). No between-cohort comparison reached significance. A single motivation item declined by one median point in both cohorts. The modal post-encounter preference was a hybrid “solo first, then group” model (7/12, 58%). On the feasibility and acceptability indicators examined, the encounter was delivered as planned in both formats and was rated favorably by learners; these data do not address educational effectiveness. Observed pre-to-post shifts are exploratory and require confirmation in an adequately powered, randomized sample. The provisional instrument is described to support that work.
Authors
- Samantha Lynch
- Shelley A. Tischkau (ORCID: https://orcid.org/0000-0002-8016-6398)
- Richard H. Selinfreund (ORCID: https://orcid.org/0000-0002-3778-5134)
- Sean McGinity
- James Lynch
- Elijah Meldrim
- Andrew Primous
Institutions
- Southern Illinois University Carbondale (US)
- Southern Illinois University School of Medicine (US)
Publication Details
- Journal
- BMC Medical Education
- Published
- 2026-11-13
- DOI
- https://doi.org/10.1186/s12909-026-10812-8
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- School of Medicine, Southern Illinois University