Exploring conversational agents for novice programmers: a scoping review

Abstract Background Educational mediums are evolving rapidly due to technological advancements and the increasing availability of digital resources. A significant trend in education is the integration of technologies like Conversational Agents (CAs), often referred to as Chatbots, as supplementary tools to enhance learning experiences. Despite their growing application, comprehensive reviews of their design methodologies, implementation strategies, pedagogical foundations, and data management practices in introductory programming education remain scarce. Objective This scoping review aims to explore and map the existing literature on educational CAs developed for programming education, analyzing publication characteristics (e.g., distribution and research designs) and prototype characteristics (e.g., implementation techniques and focus areas), as well as how these systems extend support beyond academic learning. Inclusion criteria The review includes studies on educational CAs designed to assist in teaching introductory computer science and programming courses, published in English between January 2019 and June 2024. Studies focusing on commercial products, code generators, or non-novice programmers were excluded. Methods A total of 743 citations were identified through searches in recognized digital databases. After screening and applying inclusion criteria, 23 studies were selected for this scoping review, following the PRISMA-ScR framework. Results This review provides a comprehensive overview of the use of CAs in programming education, highlighting the diverse technologies, pedagogical approaches, and interaction methods employed to support novice programmers. Conclusions Advancements in large language models (LLMs) and AI technologies have significantly enhanced the potential of CAs to transform programming education by offering personalized assistance, real-time feedback, and adaptive learning experiences. However, challenges related to scalability, usability, and inclusivity must be addressed. In addition, this review proposes actionable recommendations to guide future research and design practices.