Form over Substance: A Metric-Decomposition Diagnosis of Group Relative Policy Optimization for Arabic Legal Reasoning

Reinforcement learning with verifiable rewards (RLVR) drives much post-training of reasoning models, yet it is judged by a rising reward curve that cannot indicate which of several rule-based verifiers earned the gain. This study makes that observable during training through the weight-zero monitor: the substantive verifier is registered a second time at weight 0.0 and logged alongside the within-group standard deviation of each component, so that it cannot be optimized against. It was applied within an internally registered falsification program of Group Relative Policy Optimization (GRPO) on Qwen3-8B over Saudi commercial rulings. The registered attempt moved the policy without improving it: every greedy completion changed, 54% of held-out dispositions changed, macro-F1 fell by 0.067 (95% CI [−0.136, +0.011]), and repetition halved while the monitor remained flat. Acting on this reading, eight-sample majority voting, which trains nothing, achieved the only significant gain, +0.088 macro-F1 on a frozen 300-case instrument (95% CI [0.043, 0.140]). We derive when a gated composite reward rewards form rather than substance, and we recommend measuring the extractable headroom before allocating compute to policy-gradient training.

Authors

Institutions

Publication Details

Journal
Islamic University Journal of Applied Sciences
Published
2026-10-30
DOI
https://doi.org/10.63070/jesc.2026.046
Primary Topic
Artificial Intelligence in Law
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Form over Substance: A Metric-Decomposition Diagnosis of Group Relative Policy Optimization for Arabic Legal Reasoning

Abdullah M. Alshanqiti, Ahmad B. Alkhodre, Nawaf Al Enezi
2 links
Islamic University Journal of Applied Sciences
Artificial Intelligence in Law
article

Form over Substance: A Metric-Decomposition Diagnosis of Group Relative Policy Optimization for Arabic Legal Reasoning

Abdullah M. Alshanqiti, Ahmad B. Alkhodre, Nawaf Al Enezi
article en

Abstract

Reinforcement learning with verifiable rewards (RLVR) drives much post-training of reasoning models, yet it is judged by a rising reward curve that cannot indicate which of several rule-based verifiers earned the gain. This study makes that observable during training through the weight-zero monitor: the substantive verifier is registered a second time at weight 0.0 and logged alongside the within-group standard deviation of each component, so that it cannot be optimized against. It was applied within an internally registered falsification program of Group Relative Policy Optimization (GRPO) on Qwen3-8B over Saudi commercial rulings. The registered attempt moved the policy without improving it: every greedy completion changed, 54% of held-out dispositions changed, macro-F1 fell by 0.067 (95% CI [−0.136, +0.011]), and repetition halved while the monitor remained flat. Acting on this reading, eight-sample majority voting, which trains nothing, achieved the only significant gain, +0.088 macro-F1 on a frozen 300-case instrument (95% CI [0.043, 0.140]). We derive when a gated composite reward rewards form rather than substance, and we recommend measuring the extractable headroom before allocating compute to policy-gradient training.

Islamic University Journal of Applied SciencesVol. VIII(2)
Islamic University of Madinah (SA)
Peace, justice, and strong institutions
Openalex Percentile: Top 4%
Artificial Intelligence in Law
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Form over Substance: A Metric-Decomposition Diagnosis of Group Relative Policy Optimization for Arabic Legal Reasoning — Abdullah M. Alshanqiti, Ahmad B. Alkhodre, et al. · Islamic University Journal of Applied Sciences (2026) | TGRS Research Map | TGRS