Deep Learning Based Knowledge Tracing: A Review, a Tool and Empirical Studies

Knowledge tracing (KT) involves utilizing historical data from students’ learning interactions to model their mastery of knowledge over time, with the aim of predicting their future performance in interactions. Recently, significant advancements have been achieved through the application of various deep learning methodologies to address the KT challenge. However, a considerable proportion of deep learning-based knowledge tracing (DLKT) approaches exhibit striking similarities in their methodologies, and model designs, and even the outcomes demonstrate minimal divergence. In addition, the evaluation procedures employed in current DLKT studies are not standardized, resulting in substantial inconsistencies in the reported area under the curve (AUC) outcomes, despite analyzing the same model on identical datasets. To address the two aforementioned problems, this paper proposes a generalized DLKT framework and represents the existing DLKT models with five components, i.e., multimodal data encoder, student knowledge memory, auxiliary knowledge base, learning outcome objective, and computational efficiency and scalability. Furthermore, we develop and open source a standardized DLKT benchmark platform named <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pyKT</small>,<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> that consists of a standardized set of integrated data preprocessing procedures on 9 popular datasets across different domains, and 21 frequently compared DLKT model implementations. With <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pyKT</small>, we conduct empirical and reproducible research to assess the performance of prevalent DLKT algorithms in an unbiased and clear setting over multiple data sources. Finally, we discuss the applications of KT techniques in the educational sector and their future development directions.