Exploring the use of ChatGPT in mathematical problem solving competency education: An analysis of error diagnosis and feedback performance in solving word problems on division of fractions

Recently, several researchers have been exploring how to utilize generative artificial intelligence in mathematical problem solving competency education. However, how generative AI diagnoses students' errors and what feedback they provide to respond to them has not yet been explored in detail. In this study, the error diagnosis and feedback performance of ChatGPT was evaluated to explore its application in problem solving competency education. To achieve this, representative error types in solving word problems involving division of fractions were identified, and scenarios of students with insufficient problem-solving competency, along with structured prompts, were developed. The results indicated that ChatGPT’s performance in diagnosing and providing feedback on errors associated with problem understanding and strategy exploration was satisfactory. However, ChatGPT’s performance in diagnosing and providing feedback on errors associated with implementing and reflection exhibited variability depending on the type and number of errors. Potential applications of ChatGPT in problem solving competency education and topics for future research were discussed.