A Comparison of Real-Time User Classification Methods using Interaction Data for Open-ended Learning

When tutoring students it is useful to be able to predict whether they are succeeding as early as possible. This paper compares multiple methods for predicting from sequential interaction data whether a student is on a successful path. Predicting students' future performance and intervening has shown promise in improving learner outcomes and alleviating learner difficulty during open-ended learning. However, the literature lacks a systematic comparison of different classifiers across different open-learning datasets. This paper compares four real-time binary classifiers of learner types (on track to succeed or not) - an association rule-based classifier, a hidden Markov model-based classifier, a long short-term memory neural network classifier, and a stratified baseline classifier. Classifiers are trained and evaluated on three datasets representing different avenues of learning - an interactive simulation, massive open online courseware, and collaborative learning. A statistical evaluation of the real-time predictive performance of classifiers is conducted. This work also provides insights into model interpretability using explainable AI tools and discusses the tradeoff between accuracy and inherent interpretability of classifiers.