Applications of Machine Learning for at‐Risk Student Prediction in Online Education: A 10‐Year Systematic Review of Literature

ABSTRACT Background The growth of online education has provided flexibility and access to a wide range of courses. However, the self‐paced and often isolated nature of these courses has been associated with increased dropout and failure rates. Researchers employed machine learning approaches to identify at‐risk students, but multiple issues have not been addressed concerning the definition of at‐risk students, as well as the strengths and limitations of different machine learning models to predict at‐risk students. Objectives This systematic review aims to provide a comprehensive overview of the past 10‐year research focusing on applying machine learning techniques for predicting at‐risk students (i.e., failure, dropouts) in online learning environments. Methods Studies were extracted from the ACM Digital Library, IEEE Xplore Digital Library, Web of Science, ERIC, ProQuest, and EBSCO. A total of 161 studies published from 2014 to 2024 were included in the review. Results and Conclusions Findings revealed (1) four primary at‐risk definitions outlined in the reviewed studies, each focusing on specific stages of student engagement and performance in a course; (2) most studies relied on student behavioural engagement and academic factors as at‐risk predictors; (3) the adoption of deep learning and ensemble deep learning networks has significantly increased in the past 5 years, often outperforming classical machine learning models. While studies in which classical machine learning excelled often relied on the ensemble methodology and smaller sample sizes; (4) current machine learning practice evaluated by a list of criteria showed concerns regarding reproducibility, generalisability, and interpretability.