Confusion Intensity Estimation using Graph Convolutional Networks with Facial Action Unit Clues

Confusion is a common emotion during the learning process, and its recognition is critical for evaluating knowledge acquisition. Therefore, evaluating learners’ confusion states is crucial. Facial expressions, as direct indicators of emotional states, are widely used in emotion analysis. However, most existing research focuses on basic emotions, with limited attention given to learning emotions. Estimating confusion intensity presents two key challenges: (1) Confusion often occurs in long sequences; and (2) Confusion exhibits significant individual variability. To address these challenges, we propose three key innovations: First, we introduce an unsupervised method for extracting confusion frames based on its unique characteristics. Second, we employ a Masked Autoencoder (MAE) to capture implicit features and retain relevant external cues of confusion. Third, we design an Action Unit-assisted Graph Convolutional Network (GCN) to leverage subtle facial muscle movements, establishing relationships between samples and mitigating individual differences. Experiments on the public dataset show that our method effectively estimates the intensity of the learners’ confusion.