Artificial intelligence in multimodal learning analytics: A systematic literature review

The proliferation of educational technologies has generated unprecedented volumes of diverse, multimodal learner data, offering rich insights into learning processes and outcomes. However, leveraging this complex, multimodal data requires advanced analytical methods. While Multimodal Learning Analytics (MMLA) offers promise for exploring this data, the potential of Artificial Intelligence (AI) to enhance MMLA remains largely unexplored. This paper bridges these two evolving domains by conducting the first systematic literature review at the intersection of AI and MMLA, analyzing 43 peer-reviewed papers from 11 reputable databases published between 2019 and 2024. The findings indicate a growing trend in AI-enhanced MMLA studies published predominantly in high-quality venues, led by education researchers with a predominant focus on tertiary education targeting diverse stakeholders. Guided by a novel conceptual framework, our analysis highlights the transformative role of AI across the MMLA process, particularly in model learning and feature engineering. However, it also uncovers significant gaps, including limited AI implementation in components requiring deep integration with learning theories, insufficient application of advanced AI techniques, and lack of large-scale studies in authentic learning environments. The review identifies key benefits, such as enhanced personalization and real-time feedback, while also addressing challenges related to ethical considerations, data integration, and scalability. Our study contributes by offering comprehensive recommendations for future research, emphasizing international collaboration, multi-level studies, and ethical AI implementation. These findings advance the theoretical understanding of AI's role in education, providing a foundation for developing sophisticated, interpretable, and scalable AI-enhanced MMLA approaches, potentially revolutionizing personalized learning across diverse educational settings. • The paper reviews 43 studies on AI and Multimodal Learning Analytics (MMLA), focusing on current trends, study contexts, AI applications, experimental designs, and main challenges and benefits. • Current State of Research: The number of AI-enhanced MMLA studies has shown an overall growth trend from 2019 to 2024, leading to an extensive geographical spread across 24 countries on six continents. • Application Contexts: AI-enhanced MMLA studies significantly focused on tertiary education, with significant attention also given to secondary and primary levels, targeting learners, instructors, researchers, and technology developers. • AI Implementation: AI techniques have increasingly been integrated into various phases of MMLA studies, predominantly in the model learning, feature engineering, and transformation sub-components, but significant opportunities remain to apply advanced AI methods, such as deep learning and generative models, to improve data processing, model interpretability, and overall educational impact, highlighting the need for further innovation in this evolving field. • Experimental Designs: The studies employ a variety of experimental designs, with a focus on diverse sample sizes and settings, and consider ethical implications such as privacy and data security, ensuring the reliability and generalizability of their findings. • Benefits and challenges. The reported benefits of AI-enhanced MMLA include improved personalization of learning, real-time feedback, and enhanced student engagement, while challenges involve ethical concerns, data integration complexities, and the need for interdisciplinary collaboration.