From simplification to sophistication: Secondary students' conceptual change in understanding machine learning model decision making

Abstract As artificial intelligence (AI) reshapes the educational landscape, promoting AI literacy in secondary education has become increasingly important. Understanding how secondary students perceive AI, particularly the machine learning (ML) decision‐making process, is critical for equipping them to navigate an AI‐driven future. While prior research has highlighted the potential of AI education to foster meaningful engagement with AI concepts, little is known about how students conceptualize and understand the underlying mechanisms of ML. This study developed and implemented an AI module within English Language Arts classes in middle and high schools to demystify the ML decision‐making process. A total of 437 students from five schools participated in this research. Qualitative analyses of student responses revealed five distinct categories of understanding and misconceptions: (1) data‐based thinking, (2) divergent thinking, (3) anthropomorphic thinking, (4) awareness of limitations and (5) tale‐spin simplification. By mapping a progression in students' understanding from anthropomorphic interpretations towards a robust, data‐driven perspective, this study fills a critical research gap, offers valuable insights for fostering AI literacy in secondary education contexts, and provides actionable implications for curriculum design and future research aimed at addressing specific misconceptions about ML decision making. Practitioner notes What is already known about this topic Understanding the decision‐making process of machine learning models is essential for secondary learners to prepare them for navigating an AI‐driven world. Educators face significant challenges in making complex AI concepts accessible to younger students. Secondary students often hold misconceptions and fragmented understandings of the machine learning decision‐making process, resulting in a loss of control and reduced interest in further exploring AI concepts. What this paper adds This study developed and implemented an AI module for text mining and narrative modelling for secondary learners to demystify ML decision‐making process. This study introduces a robust coding framework to capture these evolving understandings of machine learning model decision making, including five key categories: ‘data‐based thinking’, ‘divergent thinking’, ‘anthropomorphic thinking’, ‘awareness of limitations’ and ‘tale‐spin simplification’. This study also demonstrates a progression in students' understanding, shifting from anthropomorphic interpretations to data‐driven perspectives. This study highlights the importance of helping students decouple their humanistic associations with language from the statistical, pattern‐based operations that drive language‐based AI models. Implications for Practice and/or Policy Educators and researchers might consider including ‘conceptual decoupling’ as a core competency within AI literacy, emphasizing the skills needed to differentiate human cognitive abilities from data‐driven processing in machine learning models. Educators can address these misconceptions by using analogies that emphasize AI's statistical nature, designing activities that compare human decision making with algorithmic processes and encouraging students to reflect on examples where AI ‘errors’ reveal the limitations of pattern‐based, non‐human reasoning. Educators should integrate culturally relevant examples to deepen their understanding of language patterns in model decision making.