Evaluation of traditional machine learning algorithms for featuring educational exercises

Abstract Artificial intelligence (AI) algorithms are important in educational environments, and the use of machine learning algorithms to evaluate and improve the quality of education. Previous studies have individually analyzed algorithms to estimate item characteristics, such as grade, number of attempts, and time from student interactions. By contrast, this study integrated all three characteristics to discern the relationships between attempts, time, and performance in educational exercises. We analyzed 15 educational assessments using different machine learning algorithms, specifically 12 for regression and eight for classification, with different hyperparameters. This study used real student interaction data from Zenodo.org, encompassing over 150 interactions per exercise, to predict grades and to improve our understanding of student performance. The results show that, in regression, the Bayesian ridge regression and random forest regression algorithms obtained the best results, and for the classification algorithms, Random Forest and Nearest Neighbors stood out. Most exercises in both scenarios involved more than 150 student interactions. Furthermore, the absence of a pattern in the variables contributes to suboptimal outcomes in some exercises. The information provided makes it more efficient to enhance the design of educational exercises.