Form over Substance: A Metric-Decomposition Diagnosis of Group Relative Policy Optimization for Arabic Legal Reasoning

Reinforcement learning with verifiable rewards (RLVR) drives much post-training of reasoning models, yet it is judged by a rising reward curve that cannot indicate which of several rule-based verifiers earned the gain. This study makes that observable during training through the weight-zero monitor: the substantive verifier is registered a second time at weight 0.0 and logged alongside the within-group standard deviation of each component, so that it cannot be optimized against. It was applied within an internally registered falsification program of Group Relative Policy Optimization (GRPO) on Qwen3-8B over Saudi commercial rulings. The registered attempt moved the policy without improving it: every greedy completion changed, 54% of held-out dispositions changed, macro-F1 fell by 0.067 (95% CI [?0.136, +0.011]), and repetition halved while the monitor remained flat. Acting on this reading, eight-sample majority voting, which trains nothing, achieved the only significant gain, +0.088 macro-F1 on a frozen 300-case instrument (95% CI [0.043, 0.140]). We derive when a gated composite reward rewards form rather than substance, and we recommend measuring the extractable headroom before allocating compute to policy-gradient training.

[1] Z. Shao et al., “DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models,” arXiv:2402.03300, 2024.

[2] D. Guo et al., “DeepSeek-R1 Incentivizes Reasoning in LLMs Through Reinforcement Learning,” Nature, vol. 645, pp. 633–638, 2025, doi: 10.1038/s41586-025-09422-z.

[3] J. Karwowski, O. Hayman, X. Bai, K. Kiendlhofer, C. Griffin, and J. Skalse, “Goodhart’s Law in Reinforcement Learning,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2024. arXiv:2310.09144.

[4] S.-Y. Liu et al., “GDPO: Group Reward-Decoupled Normalization Policy Optimization for Multi-Reward RL Optimization,” arXiv:2601.05242, 2026.

[5] J. Cui, X. Shen, F. Nie, Z. Wang, J. Wang, and Y. Chen, “A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges,” arXiv:2204.04859, 2022.

[6] J. Niklaus, V. Matoshi, P. Rani, A. Galassi, M. Stürmer, and I. Chalkidis, “LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain,” in Findings Assoc. Comput. Linguistics: EMNLP, 2023. arXiv:2301.13126.

[7] I. Chalkidis et al., “LexGLUE: A Benchmark Dataset for Legal Language Understanding in English,” in Proc. 60th Annu. Meeting Assoc. Comput. Linguistics (ACL), Vol. 1: Long Papers, Dublin, Ireland, May 2022, pp. 4310–4330, doi: 10.18653/v1/2022.acl-long.297.

[8] A. Ammar, A. Koubaa, B. Benjdira, O. Nacar, and S. Sibaee, “Prediction of Arabic Legal Rulings Using Large Language Models,” Electronics, vol. 13, no. 4, art. 764, 2024, doi: 10.3390/electronics13040764.

[9] M. B. Kmainasi, A. E. Shahroor, and A. Al-Ghraibah, “Can Large Language Models Predict the Outcome of Judicial Decisions?” arXiv:2501.09768, 2025.

[10] A. Almalki, S. Alsafari, and N. M. Alotaibi, “Legal Judgment Prediction in the Saudi Arabian Commercial Court,” Future Internet, vol. 17, no. 10, art. 439, 2025, doi: 10.3390/fi17100439.

[11] H. Abu Shairah et al., “ALARB: An Arabic Legal Argument Reasoning Benchmark,” in Proc. 3rd Arabic Natural Language Processing Conf. (ArabicNLP), Suzhou, China, Nov. 2025, pp. 389–406, doi: 10.18653/v1/2025.arabicnlp-main.32.

[12] M. Medvedeva, M. Wieling, and M. Vols, “Rethinking the field of automatic prediction of court decisions,” Artif. Intell. Law, vol. 31, no. 1, pp. 195–212, 2023, doi: 10.1007/s10506-021-09306-3.

[13] M. Medvedeva and P. McBride, “Legal Judgment Prediction: If You Are Going to Do It, Do It Right,” in Proc. Natural Legal Language Processing Workshop 2023, Singapore, Dec. 2023, pp. 73–84, doi: 10.18653/v1/2023.nllp-1.9.

[14] F. Kang, M. Kuchnik, K. Padthe, M. Vlastelica, R. Jia, C.-J. Wu, and N. Ardalani, “Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead,” arXiv:2510.01624, 2025.

[15] L. Zhang and J. Li, “Predicting Inference-Time Scaling Gains from Labeled Validation-Set Output Statistics,” arXiv:2606.02981, 2026.

[16] C. Gan, Z. Cai, Y. Liang, H. Wei, S. Ni, and Q. Zhang, “A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent: A Controlled Null and Its Mechanism,” arXiv:2607.12640, 2026.

[17] N. Razin et al., “Vanishing Gradients in Reinforcement Finetuning of Language Models,” in Proc. Int. Conf. Learn. Representations (ICLR), 2024. arXiv:2310.20703.

[18] W. Nie et al., “Gradient Starvation in Binary-Reward GRPO: Why Group-Mean Centering Fails and Why the Simplest Fix Works,” arXiv:2605.07689, 2026.

[19] Q. Yu et al., “DAPO: An Open-Source LLM Reinforcement Learning System at Scale,” in Adv. Neural Inf. Process. Syst. (NeurIPS), 2025. arXiv:2503.14476.

[20] Y. Ichihara et al., “MO-GRPO: Mitigating Reward Hacking of Group Relative Policy Optimization on Multi-Objective Problems,” arXiv:2509.22047, 2025.

[21] M. M. Salmani-Zarchi, Z. Rahimi, H. Faili, and M. J. Dousti, “MDP-GRPO: Stabilized Group Relative Policy Optimization for Multi-Constraint Instruction Following,” in Proc. 64th Annu. Meeting Assoc. Comput. Linguistics (ACL), Vol. 1: Long Papers, San Diego, CA, USA, 2026, pp. 42784–42797, doi: 10.18653/v1/2026.acl-long.1982. arXiv:2606.06058.

[22] Y. Lu and M. Jiang, “Uncovering Cross-Objective Interference in Multi-Objective Alignment,” arXiv:2602.06869, 2026.

[23] R. Xin et al., “Surrogate Signals from Format and Length: Reinforcement Learning for Solving Mathematical Problems without Ground Truth Answers,” arXiv:2505.19439, 2025.

[24] C. Ge, C. H. Yin, H. Liang, and J. Zhang, “Why GRPO Needs Normalization: A Local-Curvature Perspective on Adaptive Gradients,” in 5th Workshop on Mathematical Reasoning and AI (MATH-AI), Neural Inf. Process. Syst. (NeurIPS), Dec. 2025. arXiv:2601.23135.

[25] X. Niu, B. Bai, W. Han, and W. Zhang, “On the Non-decoupling of Supervised Fine-tuning and Reinforcement Learning in Post-training,” arXiv:2601.07389, 2026.

[26] R. Liu, J. Liu, X. Wan, Y. Fu, and L. Pan, “When RL Fails after SFT: Rejuvenating Model Plasticity for Robust SFT-to-RL Handoff,” arXiv:2606.09932, 2026.

[27] I. Shenfeld, J. Pari, and P. Agrawal, “RL’s Razor: Why Online Reinforcement Learning Forgets Less,” arXiv:2509.04259, 2025.

[28] N. Rajani, A. P. Gema, S. Goldfarb-Tarrant, and I. Titov, “Scalpel vs. Hammer: GRPO Amplifies Existing Capabilities, SFT Replaces Them,” in Actionable Interpretability Workshop, Int. Conf. Mach. Learn. (ICML), 2025. arXiv:2507.10616.

[29] S. R. Samineni, D. Kalwar, K. Valmeekam, K. Stechly, and S. Kambhampati, “RL in Name Only? Analyzing the Structural Assumptions in RL Post-Training for LLMs,” arXiv:2505.13697, 2025.

[30] M. Pronesti, A. Belz, and Y. Hou, “Beyond Outcome Verification: Verifiable Process Reward Models for Structured Reasoning,” in Findings of the Association for Computational Linguistics: ACL 2026, 2026. arXiv:2601.17223.

[31] L. Gao, J. Schulman, and J. Hilton, “Scaling Laws for Reward Model Overoptimization,” in Proc. Int. Conf. Mach. Learn. (ICML), 2023.

[32] B. Baker et al., “Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation,” arXiv:2503.11926, 2025.

[33] X. Wang et al., “Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges,” arXiv:2604.13602, 2026.

[34] I. F. Shihab, S. Akter, and A. Sharma, “Detecting Proxy Gaming in RL and LLM Alignment via Evaluator Stress Tests,” arXiv:2507.05619, 2025.

[35] X. He et al., “Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation,” arXiv:2605.21125, 2026.

[36] M. F. Tarek et al., “Reward Hacking Mitigation using Verifiable Composite Rewards,” in Proc. 16th ACM Int. Conf. Bioinformatics, Computational Biology, and Health Informatics (BCB), 2025.

[37] C. Ye et al., “Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Training,” in MATH-AI Workshop, Neural Inf. Process. Syst. (NeurIPS), 2025. arXiv:2509.03403.

[38] E. Skopin and E. Kotelnikov, “Improving Small Language Models for Code Generation with Reinforcement Learning from Verification Feedback,” in Proc. Artificial Intelligence and Natural Language Conf. (AINL), 2026. arXiv:2605.30478.

[39] F. Ariai and G. Demartini, “Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challenges,” ACM Comput. Surv., 2024.

[40] F. Hijazi et al., “ArabLegalEval: A Multitask Benchmark for Assessing Arabic Legal Knowledge in Large Language Models,” in Proc. 2nd Arabic Natural Language Processing Conf. (ArabicNLP), Bangkok, Thailand, Aug. 2024, doi: 10.18653/v1/2024.arabicnlp-1.20.

[41] M. Altherwi, “The Efficacy of Large Language Models (LLM) in Predicting the Outcomes of Commercial Legal Cases based on The Saudi Law: A Preliminary Study,” in Proc. 6th Int. Conf. Computing, Networks and Internet of Things (CNIOT), 2025, pp. 1–5, doi: 10.1109/CNIOT65435.2025.11070426.

[42] N. Alangari, N. Alshenaifi, and H. Almuzaini, “Hierarchical Classification of Arabic Legal Cases Using Transformer Architectures and Large Language Models,” Electronics, vol. 15, no. 14, art. 2988, 2026, doi: 10.3390/electronics15142988.

[43] V. Ovcharov, “UA-Legal-Bench: A Benchmark for Evaluating Large Language Models on Ukrainian Legal Reasoning,” arXiv:2605.29170, 2026.

[44] A. Hochlehnert et al., “A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility,” arXiv:2504.07086, 2025.

[45] A. Yang et al., “Qwen3 Technical Report,” arXiv:2505.09388, 2025.

[46] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-Rank Adaptation of Large Language Models,” in Proc. Int. Conf. Learn. Representations (ICLR), 2022. arXiv:2106.09685.

[47] W. Kwon et al., “Efficient Memory Management for Large Language Model Serving with PagedAttention,” in Proc. 29th ACM Symp. Operating Systems Principles (SOSP), 2023, pp. 611–626, doi: 10.1145/3600006.3613165.

[48] J. Schulman and Thinking Machines Lab, “LoRA Without Regret,” Thinking Machines Lab: Connectionism, Sep. 2025, doi: 10.64434/tml.20250929. [Online]. Available: https://doi.org/10.64434/tml.20250929. Accessed: Aug. 4, 2026.

[49] Hugging Face, “TRL Documentation: Logging (GRPOTrainer logged metrics),” 2026. [Online]. Available: https://huggingface.co/docs/trl/main/logging. Accessed: Sep. 6, 2026.

[50] N. Ho, L. Schmid, and S.-Y. Yun, “Large Language Models Are Reasoning Teachers,” in Proc. 61st Annu. Meeting Assoc. Comput. Linguistics (ACL), Vol. 1: Long Papers, Toronto, Canada, 2023, pp. 14852–14882.

[51] X. Wang et al., “Self-Consistency Improves Chain of Thought Reasoning in Language Models,” in Proc. 11th Int. Conf. Learn. Representations (ICLR), 2023.

[52] N. Thakur, N. Reimers, A. Rücklé, A. Srivastava, and I. Gurevych, “BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models,” in Proc. Neural Inf. Process. Syst. Datasets and Benchmarks Track, 2021. arXiv:2104.08663.

[53] Y. Gao et al., “Retrieval-Augmented Generation for Large Language Models: A Survey,” arXiv:2312.10997, 2023.

[54] Y. Wu et al., “Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration,” in Proc. 2023 Conf. Empirical Methods in Natural Language Processing (EMNLP), Singapore, 2023, pp. 12060–12075.

[55] S. Balloccu, P. Schmidtova, M. Lango, and O. Dusek, “Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs,” in Proc. 18th Conf. European Chapter of the Association for Computational Linguistics (EACL), Vol. 1: Long Papers, St. Julian’s, Malta, Mar. 2024, pp. 67–93, doi: 10.18653/v1/2024.eacl-long.5.

[56] D. Ulmer, C. Hardmeier, and J. Frellsen, “deep-significance: Easy and Meaningful Statistical Significance Testing in the Age of Neural Networks,” in Proc. ML Evaluation Standards Workshop, Int. Conf. Learn. Represent. (ICLR), 2022. arXiv:2204.06815.

[57] S. Riezler and M. Hagmann, Validity, Reliability, and Significance: Empirical Methods for NLP and Data Science, 2nd ed. Cham, Switzerland: Springer, 2024.

[58] E. van Miltenburg, C. van der Lee, and E. Krahmer, “Preregistering NLP Research,” in Proc. 2021 Conf. North Amer. Chapter Assoc. Comput. Linguistics: Human Lang. Technol. (NAACL-HLT), 2021, pp. 613–623.

[59] F. Cuconasu et al., “The Power of Noise: Redefining Retrieval for RAG Systems,” in Proc. 47th Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, Washington, DC, USA, 2024, pp. 719–729.

[60] X. Zhu et al., “The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning,” arXiv:2506.01347, 2025.

[61] B. Bai, X. Wang, P. Ye, and T. Chen, “Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewards,” in ICML Workshop on AI for Math, Jul. 2026. arXiv:2602.02555.

[62] Y. Wang et al., “Learn What’s Left, Not What’s Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization,” arXiv:2608.16072, 2026.

[63] Y. Wang et al., “Reinforcement Learning for Reasoning in Large Language Models with One Training Example,” arXiv:2504.20571, 2025.

[64] R. Shao et al., “Spurious Rewards: Rethinking Training Signals in RLVR,” arXiv:2506.10947, 2025.

[65] M. Wu et al., “Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination,” in Proc. AAAI Conf. Artificial Intelligence, 2026. arXiv:2507.10532.

[66] Y. Yu, “Pass@k Metric for RLVR: A Diagnostic Tool of Exploration, But Not an Objective,” arXiv:2511.16231, 2025.

[67] P. Cordero-Encinar and A. B. Duncan, “Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMs,” arXiv:2510.17472, 2025.

[68] Q. Di, K. Ji, X. Li, H. Zhao, and Q. Gu, “Best-of-Majority: Minimax-Optimal Strategy for Pass@k Inference Scaling,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2026. arXiv:2510.03199.

[69] Z. Cen, Y.-F. Yao, W. J. Han, Z. Liu, and D. Zhao, “Behavior Injection: Preparing Language Models for Reinforcement Learning,” arXiv:2505.18917, 2025.