Exploring Item-Level Heterogeneous Treatment Effects in Educational Interventions Through Machine Learning Techniques and Item Response Models

Educational interventions are typically evaluated using traditional statistical methods focusing on overall test scores. Such evaluations may miss item-level effects. This work explores the use of machine learning (ML) to analyze item-level heterogeneity in treatment effects (HTE). Such approaches may allow for more personalized educational assistance. We compared robust ML approaches such as Random Forest, Gradient Boosting, Extreme Gradient Boosting, and K-Nearest Neighbors to a traditional explanatory item response model (EIRM) approach and found that ML approaches yield higher accuracy. Our results suggest that flexible models accounting for item-specific treatment effects have important predictive utility in educational interventions.