AIED unplugged, teacher workload, and numeracy learning: a clustered quasi-experimental mixed-methods study

Abstract Artificial Intelligence in Education (AIED) systems typically rely on student-device interaction, which limits their applicability in settings where such infrastructure is unavailable. The AIED Unplugged (AIED-U) paradigm addresses this challenge by positioning the teacher as a proxy between learners and the AIED system. However, empirical evidence comparing AIED-U with traditional practice remains scarce. This study evaluates how using an AIED-U system affects student learning and teacher workload in 19 authentic classrooms. We conducted a clustered, quasi-experimental mixed-methods study in Brazilian elementary school mathematics classrooms, where classes were randomly assigned to a control condition, a teacher-training condition, or an experimental condition combining training with the use of an AIED-U system. We measured learning gains through pre- and post-tests, for 221 of the 320 participating students who completed both tests, teachers’ perceived workload through the raw NASA Task Load Index, and teacher perceptions through semi-structured interviews. Students taught by trained teachers outperformed those in the control condition, both in the experimental (estimated difference of 12.31, 95% CI [0.34, 21.86]) and in the training (15.15, 95% CI [0.84, 31.10]) conditions, whereas these two conditions did not differ from each other (−4.33, 95% CI [−17.29, 7.23]). Perceived workload was descriptively lower in the experimental condition, but did not differ significantly across conditions on any subscale. However, teacher workload was negatively associated with learning gains and, notably, using the AIED-U system yielded a positive indirect effect on learning gains through a reduction in the effort teachers reported (3.65, 95% CI [1.10, 6.40]). These findings provide preliminary evidence that teacher-mediated AI may offer a practical strategy for improving educational opportunities in resource-constrained settings, calling for replications to establish whether this potential holds in longer interventions and other educational settings.