Multimodal perspectives on affective dynamics in an intelligent tutoring system

Research has highlighted the relevance of affective processes for learning but often relies on self-reports without accounting for inter- and intraindividual dynamics. Multimodal research allows for a more comprehensive assessment of these dynamics. Most multimodal studies, however, rely on lab contexts, reducing ecological validity of results. The present study aims to address this gap. We examine situated inter- and intraindividual variability in concurrent activity emotions and affective activation and their interrelations with learning behaviour (prompt compliance) and domain knowledge after learning in an intelligent tutoring system. The sample consisted of 83 students ( M age = 15.52, SD = 1.90; girls = 56.6 %) from four secondary schools. We combined logfile data (prompt compliance), with electrodermal activity assessment (affective activation), standardized tests data (domain knowledge), and two types of self-report data: experience-sampling to capture concurrent activity emotions over time and topic-related emotions. Two-level dynamic structural equation models were applied. Results revealed concurrent emotions and prompt compliance to be self-predictive over time. Topic-related boredom and confusion were linked to concurrent boredom and confusion. Person-level affective activation negatively predicted person-level domain knowledge after learning, suggesting high activation may deplete cognitive resources. Topic-related boredom and confusion positively predicted increases in aggregated affective activation, suggesting these emotions drive activation. The study shows that once an emotion is established, it can persist for the learning session and that especially initial boredom can be detrimental for following learning processes and that particularly affective activation can deplete learning performance (domain knowledge after learning). • Assessed multimodal data while learning with ITS in ecological classrooms. • Examined affective dynamics, behavior, and domain knowledge after learning. • Concurrent emotions and prompt compliance were self-predictive. • Initial boredom and confusion shaped affective activation. • Affective activation related to lower domain knowledge after learning.