Exploring the potential of ChatGPT in mathematics education: Analyzing mathematical performance and errors through a multimodal approach

This study explores the potential of utilizing Large Language Models (LLMs) like OpenAI’s ChatGPT in mathematics education by analyzing its mathematical performance using a multimodal approach that includes language and image processing evaluations. A total of 1,836 middle school-level mathematical problem images were input to assess accuracy rates, error rates, and characteristics, patterns, and contexts of errors. The integrity of the solution processes was evaluated in terms of accuracy, relevance, transparency, fluency, and safety. The analysis revealed that ChatGPT generally maintained high accuracy rates in the algebra domain but showed relatively lower accuracy in the geometry domain. Notably, recognition errors were found to be frequent and often triggered a chain of subsequent errors, making them a key area of concern. Strategies to mitigate these errors included enhancing the linguistic, symbolic, and numerical explanations provided with the problems. Furthermore, through an analysis of ChatGPT’s problem-solving processes, the study identified its potential for educational use in areas such as problem- solving, personalized tutoring, problem posing, and grading and error analysis. This study provides a comprehensive multimodal analysis and confirms the potential of ChatGPT in mathematics education, and offers insights and recommendations for future research to ensure its reliable application in educational settings.