The alignment of knowledge focus during teaching preparation and AI-powered learning-by-teaching and their effects on mathematics learning outcomes in the context of high-stakes state assessments

The learning-by-teaching paradigm, particularly with AI-powered teachable agents, has shown particular promise for mathematics education. Despite this potential, it remains unclear how to best balance conceptual and procedural knowledge in this paradigm to optimize learning. This study addresses that question by examining how students' knowledge focus during two learning phases, preparing for teaching by watching instructional videos and active learning by teaching an AI-powered teachable agent, influences test performance. Specifically, we analyzed the increase in high-stakes state assessment scores over 3 months among 169 middle school students based on the knowledge focus (conceptual vs. procedural) they adopted while watching videos and subsequently teaching the AI agent. Results indicated that the effectiveness of pedagogical practices was moderated by learner preparedness. We found that average students enrolled in the regular-grade courses achieved the highest learning gains when adopting a complementary approach: watching concept-focused instructional videos followed by procedure-focused learning-by-teaching activity. On the other hand, students enrolled in the accelerated-grade courses, who served as the higher-preparedness group, showed the highest learning gains when using a consistent, concept-focused approach across both learning modalities. These findings suggest that the higher-preparedness group would benefit from instruction and practice that both emphasize conceptual knowledge. In comparison, average students would learn more effectively when teaching others in a way that complements, rather than mirrors, how they were initially taught. • For average students, learning is optimized when procedure-focused learning-by-teaching practices are complemented by concept-focused instructions that prepare them for teaching. • Learners who are better prepared academically tend to gain more learning benefits from focusing on conceptual knowledge both when teaching the AI agent and when preparing to teach. • Average students should be encouraged to teach the AI agent with a different knowledge focus than how they were originally taught to deepen their understanding of the learning materials.