Artificial intelligence analysis of facial movements and satisfaction in online medical lectures: a cross-sectional study

Detecting learners’ real-time reactions during learning is important. One potential approach is to utilize learners’ nonverbal information. Among nonverbal cues, facial expressions have been shown to influence educational outcomes. Recent technological advances have made quantitative facial expression analysis feasible. In this study, a machine learning tool, a core component of artificial intelligence, was used to quantify subtle facial movements of medical students and examine their relationship with lecture satisfaction. Fifth-year medical students attended synchronous and asynchronous online lectures while being recorded. After the sessions, they completed a lecture satisfaction questionnaire. Learners’ facial expressions were analyzed using OpenFace 2.0, a machine learning tool capable of detecting “action units (AUs).” AUs, proposed by Ekman et al., represent subtle facial movements (e.g., AU1 “Inner brow raiser”). In this study, we focused on seven AUs, calculated the total minutes each AU was detected, and analyzed their association with the satisfaction levels reported in the questionnaire. Regression analysis revealed that overall satisfaction was significantly higher when AU45 (blink) frequency increased during synchronous lectures. A similar analysis using delivery satisfaction as the dependent variable showed that higher AU45 blink rates were significantly associated with greater satisfaction with lecture delivery in synchronous lectures. Eye blinks, which were more frequent in synchronous lectures, were the only facial cue significantly associated with satisfaction, whereas other facial expressions showed no significant relationship. Further research on learners’ blinking is warranted to better understand real-time responses and to improve the quality of online lectures.