- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates whether RLVR improves reasoning primarily through better plan selection or more reliable execution. The authors propose a matched plan-conditioned interface to separate these contributions through component replacement. Experiments on arithmetic and search tasks suggest that execution contributes more to RLVR gains in the arithmetic setting. Based on this finding, the authors introduce EF-RLVR, which improves accuracy over standard joint RLVR by 1.28 percentage points across eight paired training seeds.
Critical strengths
1.The matched interface and component-replacement analysis enable controlled comparisons, supported by confidence intervals and independent training-seed confirmation.
2.Separating plan selection from execution provides a useful framework for understanding where RLVR gains originate.
Critical weaknesses
1.All experiments use a single 1.5B model and two controlled two-plan environments. It remains unclear whether the findings extend to larger models, more diverse reasoning tasks, or realistic multi-step agent settings.
2.Although the selection- and execution-directed procedures match rollout counts and optimizer updates, they differ substantially in gradient-bearing token counts and optimization geometry. Therefore, the observed advantage does not necessarily establish that execution-focused optimization is intrinsically superior.
3.In the search setting, neither the original selector objective nor uniform action weighting learns the conditional routing rule. The experiments demonstrate the limitation of the tested objectives but do not identify an effective solution.
Questions for authors
1.The conclusion that execution contributes more than plan selection appears sensitive to the repetition protocol. Under the common-repeat setting, the confidence interval includes zero. How should we interpret the claim that execution is the dominant contributor to RLVR improvements?
2.In the search setting, the selector fails to learn conditional routing despite the demonstrated benefits of routing. What do the authors believe is the main cause of this failure?
3.The selection-directed and execution-directed approaches differ substantially in the number of gradient-bearing tokens. How can we distinguish the effect of optimizing execution from the effect of having a much larger effective optimization budget?