Mathematical Capabilities of LLMs

Large language models (LLMs) have the potential to solve mathematical problems well, but little has been investigated. In this research, we evaluate five leading LLMs — (Gemini, Claude, Mistral, ChatGPT, and Llama — on a set of 50 mathematical problems that cover calculus, algebra, geometry, number theory, and probability. Finally, the study evaluates the accuracy of their solutions and gives the ability to assess their intermediate steps correctly. I created a primary dataset for comparison of the LLM performances and ranked the models according to how well they were able to solve those problems. It illustrates the shortcomings of present LLMs at reasoning and solving for mathematics and suggests what needs to be rectified in the LLMs. Future research will further refine this dataset and monitor the progression of LLM capabilities in solving more complex mathematical problems.