Feasibility of an AI-simulated patient encounter delivered in group and individual learning formats for first-year medical students: a two-cohort pilot study

Generative artificial intelligence (AI) and large language model (LLM)–based simulated patients are being evaluated for problem-based learning and case-based learning in undergraduate medical education, but delivery formats are often under-specified and outcome measurement is inconsistently reported. We assessed the feasibility and acceptability of a single LLM-driven, AI-simulated patient encounter delivered in two formats—a facilitated whole-group session and a self-paced individual session—to first-year medical students (MS1), and we describe the development and provisional status of a learner-perception survey instrument. Two non-overlapping, non-randomized MS1 cohorts at one United States medical school completed the same case (Group, n = 7; Individual, n = 12) and the Behavioral Safety Language Scale–Provisional (BSLS-P) before and after the encounter. Pre-to-post change was tested with Wilcoxon signed-rank exact tests, with Benjamini–Hochberg false discovery rate (FDR) correction across subscale comparisons. Feasibility outcomes (recruitment, encounter completion, survey completion, technical failures, and encounter duration) and acceptability outcomes (post-encounter engagement ratings, delivery-format preference, and open-ended comments) are reported descriptively. Of 72 students invited, 19 (26%) enrolled; all completed the encounter and paired pre/post surveys (100%), with no technical failures (no encounter was terminated and no participant was excluded), although learners reported intermittent technical problems that interrupted but did not end encounters; the group session lasted approximately 35 min and individual encounters a median of 31 min (range 17–83). Pre-to-post shifts were small and imprecise; in the Group cohort, median engagement changed by + 0.33 points (Wilcoxon r = 0.13, 95% CI 0.00 to 0.90). The largest effect—a psychological-safety gain in the Individual cohort (Wilcoxon r = 0.62)—was significant before correction ( P =0.026) but not after (adjusted P =0.20). No between-cohort comparison reached significance. A single motivation item declined by one median point in both cohorts. The modal post-encounter preference was a hybrid “solo first, then group” model (7/12, 58%). On the feasibility and acceptability indicators examined, the encounter was delivered as planned in both formats and was rated favorably by learners; these data do not address educational effectiveness. Observed pre-to-post shifts are exploratory and require confirmation in an adequately powered, randomized sample. The provisional instrument is described to support that work.

Authors

Institutions

Publication Details

Journal
BMC Medical Education
Published
2026-11-13
DOI
https://doi.org/10.1186/s12909-026-10812-8
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Feasibility of an AI-simulated patient encounter delivered in group and individual learning formats for first-year medical students: a two-cohort pilot study

Samantha Lynch, Shelley A. Tischkau, Richard H. Selinfreund, Sean McGinity et al.
4 links
BMC Medical Education
Artificial Intelligence in Healthcare and Education
article

Feasibility of an AI-simulated patient encounter delivered in group and individual learning formats for first-year medical students: a two-cohort pilot study

Samantha Lynch, Shelley A. Tischkau, Richard H. Selinfreund, Sean McGinity, James Lynch, Elijah Meldrim, Andrew Primous
article en

Abstract

Generative artificial intelligence (AI) and large language model (LLM)–based simulated patients are being evaluated for problem-based learning and case-based learning in undergraduate medical education, but delivery formats are often under-specified and outcome measurement is inconsistently reported. We assessed the feasibility and acceptability of a single LLM-driven, AI-simulated patient encounter delivered in two formats—a facilitated whole-group session and a self-paced individual session—to first-year medical students (MS1), and we describe the development and provisional status of a learner-perception survey instrument. Two non-overlapping, non-randomized MS1 cohorts at one United States medical school completed the same case (Group, n = 7; Individual, n = 12) and the Behavioral Safety Language Scale–Provisional (BSLS-P) before and after the encounter. Pre-to-post change was tested with Wilcoxon signed-rank exact tests, with Benjamini–Hochberg false discovery rate (FDR) correction across subscale comparisons. Feasibility outcomes (recruitment, encounter completion, survey completion, technical failures, and encounter duration) and acceptability outcomes (post-encounter engagement ratings, delivery-format preference, and open-ended comments) are reported descriptively. Of 72 students invited, 19 (26%) enrolled; all completed the encounter and paired pre/post surveys (100%), with no technical failures (no encounter was terminated and no participant was excluded), although learners reported intermittent technical problems that interrupted but did not end encounters; the group session lasted approximately 35 min and individual encounters a median of 31 min (range 17–83). Pre-to-post shifts were small and imprecise; in the Group cohort, median engagement changed by + 0.33 points (Wilcoxon r = 0.13, 95% CI 0.00 to 0.90). The largest effect—a psychological-safety gain in the Individual cohort (Wilcoxon r = 0.62)—was significant before correction ( P =0.026) but not after (adjusted P =0.20). No between-cohort comparison reached significance. A single motivation item declined by one median point in both cohorts. The modal post-encounter preference was a hybrid “solo first, then group” model (7/12, 58%). On the feasibility and acceptability indicators examined, the encounter was delivered as planned in both formats and was rated favorably by learners; these data do not address educational effectiveness. Observed pre-to-post shifts are exploratory and require confirmation in an adequately powered, randomized sample. The provisional instrument is described to support that work.

BMC Medical Education
Southern Illinois University Carbondale (US), Southern Illinois University School of Medicine (US)
School of Medicine, Southern Illinois University
Quality education, Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon