Intelligent Tutoring Systems Need Teachers

ABSTRACT Background The development and distribution of digital learning software, such as intelligent tutoring systems (ITSs), has evolved into a billion‐dollar industry, impacting a vast number of students worldwide. A large number of studies on ITSs have focused on their effects on learning outcomes. However, less is known about student engagement and dropout when ITSs are used in real classroom settings over extended periods, particularly with regard to how these patterns may be linked to different assignment scenarios within ITSs. Objective The present study aimed to explore whether student engagement and dropout varied based on whether mathematics problems within the ITS were assigned by teachers or self‐assigned by students. Methods We evaluated rich data from an ITS for learning mathematics used in Germany and the Netherlands (~139,000,000 problems; n ~ 194,000 students) between 2016 and 2023. To examine whether students' engagement and dropout, both within and between students, varied based on the two assignment scenarios (teacher assigned vs. self‐assigned problems), we employed regression and survival analyses. Results and Conclusions Our results revealed that, in both Germany and the Netherlands, students with teacher‐assigned problems consistently (i) dropped out later, (ii) were active for significantly more weeks and (iii) worked through more mathematics problems each week than those who self‐assigned problems. The results were robust across academic school years in the examined period. Overall, our study raises questions about the use of digital learning software as a stand‐alone solution and suggests embedding such software in real‐life learning scenarios involving teachers.