Should there be a teacher in-the-loop? A study of generative AI-personalized tasks middle school

Abstract Large language models can be used to design instructional tasks to align to students’ needs. These generative AI models can create tasks that correspond to students’ out-of-school interests in areas like sports or music, which may enhance students’ interest in learning. However, little is known about teachers’ processes for creating or modifying tasks with generative AI, despite commercial AI companies’ claims that they will save teachers time. In the present study, 7 middle school mathematics teachers used ChatGPT to create personalized versions of problems in their curriculum to correspond to their students’ interests. The prompting moves teachers made and their students’ open-ended feedback on personalized problems were examined through thematic analysis. Teachers’ efficiency when creating problems and the ratings of their 7th grade students who received personalized assignments were examined. Results from this sample suggest that having a teacher-in-the-loop may result in generative AI-enhanced personalization being enacted at a broad grain size, whereas students often expressed preference for a smaller grain-size where they receive specific popular culture references that interest them. Teachers used prompting to adjust popular culture references and address issues with the depth, ethics, or realism of the problems generated, in some cases showing epistemic agency. The article closes with an exploratory, design-focused look at scalability trade-offs with teacher-facing personalization, patterns of teacher prompting practices, and opportunities for teachers to exercise epistemic and ethical monitoring of personalization outputs.