Emis: Scaling Mathematical Reasoning With Evolutionary Instruction Synthesis

The advancement of mathematical reasoning in Large Language Models (LLMs) is fundamentally constrained by the scarcity of highly complex, Olympiad-level training data. To address this bottleneck, we propose Evolutionary Mathematical Instruction Synthesis (EMIS), a novel framework that autonomously constructs a curriculum of challenging mathematical problems. EMIS follows an evolutionary process that begins with a small set of expert-curated seeds and builds upon a Long Chain-of-Thought (CoT) paradigm. Through multi-round self-reflection, it iteratively proposes, critiques, and refines new problems, systematically enhancing both problem diversity and difficulty. To ensure the correctness of the synthesized problem-solution pairs, a cross-model majority voting mechanism is integrated for robust verification. Augmenting training with data from EMIS yields consistent and significant performance improvement on established benchmarks. Notably, our fine-tuned Qwen2.5-MATH-7B model achieves 25.8% accuracy on the AIME 2024 benchmark, an absolute gain of 19.5 percentage points over the baseline. Our work demonstrates that evolutionary selfimprovement can effectively close the data gap in complex mathematical reasoning, paving the way toward more capable mathematical foundation models.