Analyzing K-12 AI education: A large language model study of classroom instruction on learning theories, pedagogy, tools, and AI literacy

There is growing recognition among researchers and stakeholders about the significant impact of artificial intelligence (AI) technology on classroom instruction. As a crucial element in developing AI literacy, AI education in K-12 schools is increasingly gaining attention. However, most existing research on K-12 AI education relies on experiential methodologies and suffers from a lack of quantitative analysis based on extensive classroom data, hindering a comprehensive depiction of AI education's current state at these educational levels. To address this gap, this article employs the advanced semantic understanding capabilities of large language models (LLMs) to create an intelligent analysis framework that identifies learning theories, pedagogical approaches, learning tools, and levels of AI literacy in AI classroom instruction. Compared with the results of manual analysis, analysis based on LLMs can achieve more than 90% consistency. Our findings, based on the analysis of 98 classroom instruction videos in central Chinese cities, reveal that current AI classroom instruction insufficiently foster AI literacy, with only 35.71% addressing higher-level skills such as evaluating and creating AI. AI ethics are even less commonly addressed, featured in just 5.1% of classroom instruction. We classified AI classroom instruction into three categories: conceptual (50%), heuristic (18.37%), and experimental (31.63%). Correlation analysis suggests a significant relationship between the adoption of pedagogical approaches and the development of advanced AI literacy. Specifically, integrating Project-based/Problem-based learning (PBL) with Collaborative learning appears effective in cultivating the capacity to evaluate and create AI.