Empirical insights into the influence of ChatGPT-facilitated programming on primary learners’ performances: a mixed methods approach

ChatGPT, an AI-driven chatbot capable of generating code and providing timely, personalized feedback, offers new opportunities for supporting programming learning among younger learners. This study employed a quasi-experimental design to examine the effects of Self-Directed Programming (SDP) and ChatGPT-Facilitated Programming (CFP) learning modes on primary learners’ programming knowledge, skills, and attitudes. In addition, learners’ programming behaviors were explored within the CFP learning mode. A total of 71 primary learners were randomly assigned to either the CFP or SDP class. Learners in the SDP class engaged in self-directed programming activities, whereas learners in the CFP class learned programming through a ChatGPT-facilitated learning platform, CodewithAI. A mixed-methods approach was adopted to collect and analyze multidimensional data. The results revealed a significant difference in programming knowledge performance between the CFP and SDP classes. Behavioral analysis indicated that primary learners in the CFP class frequently sought assistance from ChatGPT when encountering programming challenges, particularly by requesting explanations of code and error messages. In addition, primary learners in the CFP class reported relatively high levels of confidence compared with those in the SDP class. Based on these findings, this study offers implications for the design of primary programming instruction and the development of AI-supported instructional tools. By providing fine-grained empirical evidence, this study contributes to the field of the effectiveness of ChatGPT-facilitated programming learning for primary students.