Who’s at risk? Characterizing low achievers in a large-scale assessment during the COVID-19 pandemic using educational data mining

Abstract Background: In response to the COVID-19 pandemic, schools experienced repeated closures from spring 2020 onwards. Subsequent analysis of educational monitoring data has revealed significant learning losses for students. For example, the IQB Trends in Student Achievement Study 2021 (IQB-Bildungstrend 2021) found unfavorable developments for 4th-graders at the end of the primary school, both in terms of the mean proficiency levels achieved and the proportion of students who failed to meet the minimum standards. However, there is a lack of empirical studies that address aspects related to distance learning in conjunction with situational variables, offering a comprehensive investigation into the comparative importance of a wide range of variables in characterizing low-achieving students during the pandemic. Methods: We applied educational data mining techniques to characterize low achievers using data from the IQB Trends in Student Achievement Study 2021 with $$n_r =$$ 24,500 cases in reading and $$n_m =$$ 24,511 in math, utilizing 103 predictor variables. Two separate models were developed using Prediction Rule Ensembles, an innovative machine learning method combining rules from a random-forest-like approach with linear predictors in a lasso-regularized regression. Results: We identified associations of failing to meet the minimum standards with socio-economic, motivational, and pandemic-related factors and ranked them by importance. Socio-economic and cultural background held high influences. Anxiety variables showed importances comparable to those of social disparity indicators and exceeded many other predictors in the models. Although COVID-19-related variables ranked below established risk factors, they consistently appeared among the top predictors, indicating a meaningful contribution to the characterization of low achievers. In addition, we identified and discussed interactions between well-known factors and novel COVID-19-related aspects. Conclusion: While the main predictor variables include well-established risk factors, there are associations regarding aspects of distance learning, often in interaction with these established factors. Further research is necessary to confirm these findings in a causal manner; nevertheless, they establish a foundation upon which future investigations can explore the potential contributions of these variables in the characterization of low-achieving students.