Strive-R1: Proactive retrieval-augmented reasoning with strategic retrieval control and hierarchical verifiable rewards

Retrieval-Augmented Reasoning (RAR) mitigates the static knowledge limitations of Large Language Models (LLMs) by incorporating external evidence acquisition into reasoning. Recent work has introduced reinforcement learning (RL) to optimize retrieval jointly with reasoning. However, many RL-based RAR methods still organize retrieval reactively, where search is triggered by local uncertainty or immediate evidence gaps. This can weaken coordination across retrieval steps, especially when multiple pieces of evidence must be progressively collected and connected. To address this issue, we propose Strive-R1, a framework for proactive Retrieval-Augmented Reasoning with strategic retrieval control and hierarchical verifiable rewards. Specifically, Strive-R1 realizes strategic retrieval control through a structured Plan–Search–Reflect (PSR) process, in which planning sets an initial retrieval direction, reflection assesses the current trajectory and, when available, the retrieved information to determine the next action, and search acquires external evidence when further retrieval is required. In Strive-R1, PSR is not used as a fixed prompting heuristic, but learned as a multi-step retrieval-control behavior through RL. We further introduce Hierarchical Verifiable Rewards (HVR), which organize the Answer Outcome Reward, Token-Level F1 Reward, Reflection-based Auxiliary Process Signal, and Format Reward into a conditionally activated hierarchy without a learned neural reward model. Experiments on seven QA benchmarks show that Strive-R1 consistently outperforms strong baselines, with larger gains on multi-hop tasks. Additional evaluations indicate that the learned retrieval–reasoning policies can generalize to different domains and retrieval corpora. Ablation studies show that PSR provides the main gains and HVR offers complementary improvements while helping regulate search depth.