Arithmetic development: a longitudinal observational study into the use of adaptive technology

Background AI-driven Adaptive learning technologies (ALTs) are widely used in primary education. These technologies have shown promising short-term results in arithmetic development. However, their long-term impact remains unclear. Aims This longitudinal observational study with a staggered introduction of the ALT investigated how the arithmetic ability of primary school students developed from grade 1 to grade 4 and how this development differed between students who used an ALT for arithmetic practice and those who did not. We also explored how individual and school characteristics moderated any developmental differences. Sample Data were collected over three years and included 7885 children, of whom 4603 (58.4%) used an ALT and 3282 (41.6%) did not. Method Multigroup SEM growth curve analyses compared arithmetic development across the ALT and control groups. Hierarchical models (linear and nonlinear) were tested, including time-invariant covariates (gender, socioeconomic status, prior arithmetic ability, and school size). Results Students who used the ALT showed a trend towards greater growth than those who did not, with small-to-medium effect sizes. Gender and individual SES differences were small in both conditions. Schools with a higher proportion of lower-SES students showed greater growth when using the ALT. The ALT also appeared to mitigate negative associations between school size and arithmetic development. Conclusions Use of the ALT was associated with numerically greater, though not statistically significant, arithmetic development in the long-term. Larger schools and schools with a higher percentage of lower SES students may consider using an ALT to support teachers in delivering tailored instruction.