Toward multimodal physiological assessment of clinical performance in simulation-based medical education: a machine learning approach

BACKGROUND: Assessment of learner performance in simulation-based medical education commonly relies on faculty observation. Learners' physiological responses may provide complementary information about responses to clinical task demands, but their relationship with observed performance is complex and context dependent. Whether multimodal physiological patterns can reliably distinguish faculty-assigned performance categories across simulation events and learner populations remains unclear. METHODS: This prospective observational study developed machine-learning models using physiological data alone to classify learner performance according to faculty-assigned novice versus competent categories during high-fidelity simulation. Data were collected from 152 learners (74 Emergency Medicine residents, 70 Anesthesiology residents, 8 Emergency Medical Services students) across 470 simulation scenarios. A multimodal assessment platform synchronized physiological signals (electrodermal activity, heart rate, skin temperature) from Empatica E4 wristbands with faculty evaluations. A genetic algorithm was employed for feature selection, and neural network models were evaluated using multiple leave-N-out strategies to assess generalizability across learners. RESULTS: The neural network achieved 84-85% balanced accuracy across thresholds 0.45-0.70, with sensitivity 93.3-95.4% and specificity 72.9-76.2%. Despite class imbalance (80.6% competent, 19.4% novice), performance remained robust, with Matthews correlation coefficients of 0.687-0.706 and precision-recall area-under-the-curve (PR-AUC) values of 0.969-0.970 across thresholds. CONCLUSIONS: Multimodal physiological features enabled classification of faculty-assigned performance categories with strong discrimination in internal validation. These findings support the potential of temporally resolved, physiology-informed analytics to complement faculty assessment and facilitate targeted review of simulation events. Further work is needed to strengthen rating-label validity and temporal alignment and to evaluate model transparency, fairness, and generalizability across institutions, simulation contexts, and learner populations.