Enhancing enthusiasm for STEM education with AI: Domain-specific chatbot as personalized learning assistant

Generative AI has been increasingly integrated into educational settings for its potential to facilitate personalized learning. However, evidence-based implementation concepts remain essential to realize its pedagogical value. Therefore, this study investigates the educational efficacy of ADA, a domain-specific AI chatbot customized for secondary mathematics education, in enhancing learning outcomes and student engagement. A cluster-randomized controlled study was conducted with 195 ninth-grade students from German secondary schools in authentic classroom settings. The objective of the study was to compare ADA's personalized support against conventional differentiation materials. The intervention targeted the Heron method for estimating square roots in a single-lesson intervention with a pre-post assessment. As one of the few cluster-randomized controlled studies on custom AI chatbots in secondary mathematics education, this investigation assessed both affective and cognitive aspects of learning simultaneously to provide a comprehensive research picture. The findings indicated a high degree of student acceptance across the dimensions of the Technology Acceptance Model. Additionally, situational interest significantly improved in the chatbot condition. The study found that trends in performance and emotional responses were favorable, while cognitive load increased slightly. The findings emphasize the evident potential of customized AI chatbots to supplement differentiated instruction in the domain of mathematics education. They provide a critical foundation for future research investigating the potential of AI chatbots to enhance student learning through careful integration and customization.