Decoupling Outcome-Observation and Actor-Update Horizons in Reinforcement Learning with Verifiable Rewards

Long rollout horizons are increasingly used in reinforcement learning with verifiable rewards (RLVR), yet their effect on policy optimization remains unclear. Existing methods mainly modify how boundary-truncated responses are filtered or penalized, while retaining the assumption that outcome observation and policy optimization should share the same horizon. We find that longer rollouts substantially delay the final answer without consistently improving task performance, while earlier response prefixes can already support correct answers. At the same time, short horizons may miss useful outcome feedback beyond the truncation boundary. Motivated by this mismatch, we propose Natural-Continuation Boundary Return (NCBR), which naturally continues truncated responses for verification while updating only the original responses. Under a frozen continuation policy, the extended outcome estimates the continuation value of the original response, providing a basis for short-response updates. Across three mathematical reasoning benchmarks, NCBR consistently improves both DAPO and GRPO. With DAPO, it achieves a 57.4% relative improvement in AIME'24 Pass@1 over the 2K DAPO baseline, while outperforming Full 8K training with 54.5% fewer GPU-hours.