CoinMath: Harnessing the Power of Coding Instruction for Math LLM

Large Language Models (LLMs) have shown strong performance in solving mathematical problems, with code-based solutions proving particularly effective.However, the best practice to leverage coding instruction data to enhance mathematical reasoning remains underexplored.This study investigates three key questions: (1) How do different coding styles of mathematical code-based rationales impact LLMs' learning performance?(2) Can generaldomain coding instructions improve performance?(3) How does integrating textual rationales with code-based ones during training enhance mathematical reasoning abilities?Our findings reveal that code-based rationales with concise comments, descriptive naming, and hardcoded solutions are beneficial, while improvements from general-domain coding instructions and textual rationales are relatively minor.Based on these insights, we propose CoinMath, a learning strategy designed to enhance mathematical reasoning by diversifying the coding styles of code-based rationales.CoinMath generates a variety of code-based rationales incorporating concise comments, descriptive naming conventions, and hardcoded solutions.Experimental results demonstrate that CoinMath significantly outperforms its baseline model, MAmmoTH, one of the SOTA math LLMs. 1 Model Comment A. S. G. M. Avg.Llama-3.1-8BNo 61.7 30.4 32.4 11.5 34.0 Concise 83.3 80.3 73.5 36.4 68.4 Detailed 80.7 83.4 73.6 35.5 68.3 CodeLlama-Python-7B No 82.7 64.4 52.2 27.3 56.7 Concise 83.0 68.7 59.7 28.1 59.9 Detailed 83.3 69.6 55.4 27.2 58.