A Systematic Literature Review on Emotion Recognition and Classification in Online Learning

ABSTRACT Emotion recognition is essential in online learning as it helps understand and respond to learners' emotions, improving the learning experience. For this reason, this study explores how multimodal data (such as text, images, and speech) can be used for emotion recognition and classification in online learning. While existing research primarily focuses on single‐modal emotion recognition, a systematic review of the integration of multiple data modalities is lacking. To fill this gap, this paper conducts a systematic literature review to systematically explore the application of emotion recognition technologies in online learning from 2015 to 2024. Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta‐Analyses) guideline, we identified 1712 publications, of which 62 were finally included in this study after careful screening and full‐text analysis. This paper is developed on four research questions: (1) identify the main methods and techniques in emotion recognition, pointing out that deep learning (DL) and machine learning (ML) are the mainstream methods; (2) analyze the commonly used datasets, and find that image and speech mostly relies on publicly datasets, while textual data is mostly based on self‐constructed datasets; (3) summarize the primary categories or dimensions of emotions, and find that binary and ternary classification are most common; and (4) sort out the main steps of emotion recognition, including data preprocessing, feature extraction, and classification. By focusing on the technical aspect of emotion recognition in online learning, this study provides a theoretical foundation and practical guidance for building intelligent, emotion‐aware learning systems in the future.