Communicative AI Agents in Mathematical Task Design: A Qualitative Study of GPT Network Acting as a Multi-professional Team

Abstract This study explores the application of communicative AI agents, specifically a network of customized generative pretrained transformer agents, in designing mathematical tasks. It focuses on how these AI agents, functioning as a multi-professional team, can perform mathematical task design (concerning a collection of task activities and not curriculum materials/textbooks) through collaborative and context-aware communication. Concentrating on four perspectives—mathematical depth, language sensitivity, natural differentiation, and competence orientation—four different AI agents were instructed to evaluate and modify six mathematical tasks based on individual research knowledge bases. In a consensus-seeking process, the AI agents were connected via a chat chain, prompting multiple iterations to modify the tasks. The output (six AI-modified tasks) was then evaluated by six in-service teachers as human experts by making them choose blindly between the original and the AI-modified tasks and by then analyzing the additional comments to their decisions in qualitative content analysis. Furthermore, the AI-modified tasks were rated on a multidimensional Likert scale. The results indicate that for the AI-modified tasks, achieving a balance between substantial text generation and precise task formulation is crucial and was not always found in the GPT network output. At the same time, the combination of the four AI agents was able to enrich the tasks with potential solution approaches and specific calls to action.