Beyond English-Only GRPO: Training Language and Auxiliary Reward as Implicit Regularizers in Sub-3B Math Reasoning

A multi-seed empirical study of GRPO post-training of DeepSeek-R1-Distill-Qwen-1.5B under a single-GPU LoRA constraint. We compare four arms (English-only, Vietnamese-translated, English with auxiliary language-consistency reward R5, and a constant-bias ablation) across three random seeds and four math benchmarks. Three findings: (i) R5 arm achieves the only positive AIME-2024 maj@8 effect (+4.4±1.9 pp over base); (ii) vanilla English GRPO shows σ=11.3 pp seed variance on AMC-23 with consistent AIME degradation, confirming single-seed claims at this scale are unreliable; (iii) 10-language MGSM evaluation shows null cross-lingual transfer — all conditions within ±0.5 pp of base. Constant-bias ablation does not fully reproduce R5 effect

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-05-11
DOI
https://doi.org/10.5281/zenodo.20115652
Primary Topic
Cognitive and developmental aspects of mathematical skills
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Beyond English-Only GRPO: Training Language and Auxiliary Reward as Implicit Regularizers in Sub-3B Math Reasoning

Dang Vu
Zenodo (CERN European Organization for Nuclear Research)
Cognitive and developmental aspects of mathematical skills
preprint

Beyond English-Only GRPO: Training Language and Auxiliary Reward as Implicit Regularizers in Sub-3B Math Reasoning

Dang Vu
preprint en

Abstract

A multi-seed empirical study of GRPO post-training of DeepSeek-R1-Distill-Qwen-1.5B under a single-GPU LoRA constraint. We compare four arms (English-only, Vietnamese-translated, English with auxiliary language-consistency reward R5, and a constant-bias ablation) across three random seeds and four math benchmarks. Three findings: (i) R5 arm achieves the only positive AIME-2024 maj@8 effect (+4.4±1.9 pp over base); (ii) vanilla English GRPO shows σ=11.3 pp seed variance on AMC-23 with consistent AIME degradation, confirming single-seed claims at this scale are unreliable; (iii) 10-language MGSM evaluation shows null cross-lingual transfer — all conditions within ±0.5 pp of base. Constant-bias ablation does not fully reproduce R5 effect

Zenodo (CERN European Organization for Nuclear Research)
Quality Education
Cognitive and developmental aspects of mathematical skills
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.