Evaluating the Effectiveness of ChatGPT in Solving Math Problems Across Varying Levels of Difficulty and Its Implications on Education

This project aims to make the literature on LLMs, like ChatGPT, more accessible, while also evaluating its effectiveness in solving math problems of varying difficulty. In order to achieve this, I generated and ranked math problems based on their difficulty and the analysis/steps required to complete them correctly. From there, these problems were inputted into ChatGPT to be solved, and patterns of error were documented in order to see what steps or sections of these problems were conducted incorrectly. Additionally, analysis of this information allowed me to determine the benefits of ChatGPT and what areas still need improvement, in terms of mathematical problem solving. Then, based on my findings, I discussed ChatGPT’s implications on math education, and presented ways to turn it into a “teaching tool” rather than a “cheating tool.” Ultimately, this evaluation is essential because of the popularity the service has gained, especially with students, who should be more knowledgeable about the tool that they rely on to complete tasks.