Distilling Open Reasoning Corpora into a 3B-Active MoE on Commodity GPUs: What Transfers, What Doesn't, and a Decontamination Check

We present an honest, reproducible study of budget-constrained knowledge distillation for reasoning, carried out on commodity hardware (a single node of 4x NVIDIA L40S, no NVLink) for $2,723 in GPU compute ($3,091 all-in with tax) against a $3,000 budget. We LoRA-fine-tune (QLoRA, 4-bit NF4, attention-only) the Mixture-of-Experts model Qwen3-30B-A3B-Thinking-2507 (~30B total, ~3B active) on a 15,296-example mixture assembled from three open reasoning-distillation corpora (OpenCodeReasoning, Mixture-of-Thoughts, OpenThoughts3) plus a small model-identity set. To use the four GPUs on a node that lacks NVLink, we run four hyperparameter configurations in parallel, one model per card, as a single-node A/B/C/D comparison rather than a communication-bound tensor-parallel job. The best checkpoint raises overall greedy pass@1 from 84.0 to 88.5 across GSM8K, MATH-500, HumanEval+, and MBPP+. Crucially, the improvement is almost entirely mathematical: MATH-500 rises 53.6 -> 77.2 (+23.6), GSM8K is flat at ceiling, and coding is flat-to-slightly-negative (HumanEval+ -3.7, MBPP+ +3.4). We then run an n-gram-containment plus exact/substring decontamination pass between the entire data pool and every benchmark. The data the released model was actually trained on is clean against all four benchmarks at both n=13 and a stricter n=8; the only leakage (~12% of MBPP+) lives in an auxiliary teacher-generated trace pool that -- as we verify from the winning checkpoint's optimizer-step count -- was never folded into the released model. Our contribution is not a new method but a candid, checked, low-cost recipe, a transfer-asymmetry finding (verified reasoning distillation transfers strongly to mathematics and weakly to code in this regime), and a reproducibility artifact that includes the decontamination tooling most reports omit. All weights, adapters, recipe, and evaluation logs are released under Apache-2.0.