A Concept Drift Adaptive Machine Learning Model for Predicting Student Academic Performance Using Learning Management System

Learning Management Systems (LMSs) produce behavioral data which can be used to identify at-risk students. History-based training of predictive models assumes that relationships between variables are constant over time. Changes in student behavior, instructional practices and learning environments can in practice affect the predictiveness and cause concept drift. The aim of this study was to explore the concept drift problem and propose an adaptive machine-learning model that is aware of a concept drift in order to predict students' academic performance from LMS behavioral data. The quantitative, positivist, quasi-experimental and comparative design was used for 32,593 OULAD records and 15,024 UKZN records. The behavioral predictors were arranged in chronological order to maintain the temporal order. The Kolmogorov–Smirnov test, Population Stability Index, ADWIN, Drift Detection Method and Page-Hinkley were used for concept drift evaluation. An adaptive SGD (online) classification model based on incremental learning and prequential evaluation and retraining based on drift detection was contrasted with static models of logistic regression, random forest, and gradient boosting. The two datasets both exhibited temporal non-stationarity, albeit of varying significance and direction in the two contexts. Widespread statistically detectable and mostly low magnitude distributional change was observed for OULAD, while UKZN had more distinct distributional changes. Additionally, sequential monitoring revealed that the changes in prediction error were not systematically correlated with changes in behavior. The model was somewhat portable between the two institutional settings when compared across institutions. The results show that predictions performed using LMS are not static and are specific to particular institutions, and that prediction drift does not automatically lead to better predictive results. The proposed framework provides for the continuous monitoring of drift, chronological validation and model recalibration according to the context.