Form over Substance: A Metric-Decomposition Diagnosis of Group Relative Policy Optimization for Arabic Legal Reasoning
Reinforcement learning with verifiable rewards (RLVR) drives much post-training of reasoning models, yet it is judged by a rising reward curve that cannot indicate which of several rule-based verifiers earned the gain. This study makes that observable during training through the weight-zero monitor: the substantive verifier is registered a second time at weight 0.0 and logged alongside the within-group standard deviation of each component, so that it cannot be optimized against. It was applied within an internally registered falsification program of Group Relative Policy Optimization (GRPO) on Qwen3-8B over Saudi commercial rulings. The registered attempt moved the policy without improving it: every greedy completion changed, 54% of held-out dispositions changed, macro-F1 fell by 0.067 (95% CI [−0.136, +0.011]), and repetition halved while the monitor remained flat. Acting on this reading, eight-sample majority voting, which trains nothing, achieved the only significant gain, +0.088 macro-F1 on a frozen 300-case instrument (95% CI [0.043, 0.140]). We derive when a gated composite reward rewards form rather than substance, and we recommend measuring the extractable headroom before allocating compute to policy-gradient training.
Authors
- Abdullah M. Alshanqiti (ORCID: https://orcid.org/0000-0002-6080-5236)
- Ahmad B. Alkhodre (ORCID: https://orcid.org/0000-0001-6168-3552)
- Nawaf Al Enezi
Institutions
- Islamic University of Madinah (SA)
Publication Details
- Journal
- Islamic University Journal of Applied Sciences
- Published
- 2026-10-30
- DOI
- https://doi.org/10.63070/jesc.2026.046
- Primary Topic
- Artificial Intelligence in Law
- Type
- article
- Field-Weighted Citation Impact
- 0.00