- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes ARCA, an event-driven adaptive rollout collection strategy for coding-agent RL. It dynamically allocates additional rollouts to all-failure groups, stops sampling once a group contains both successful and failed trajectories, and asynchronously refills completed queries. Experiments on SWE-bench Verified report improved training throughput and pass@1 over fixed-size GRPO.
Critical strengths
-
Addresses a practical efficiency problem in long-horizon coding-agent RL.
-
Evaluates on real coding-agent tasks, with additional OOD results on SWE-Gym and Terminal-Bench.
Critical weaknesses
-
Limited novelty. The core ideas substantially overlap with existing work. NGU already investigates asynchronous resampling of unsuccessful queries; AR3PO and AERO study adaptive rollout allocation; Reinforce-Ada and SARA provide more principled formulations of adaptive sampling and stopping decisions. The paper does not clearly establish what is fundamentally new beyond combining existing techniques in a coding-agent setting.
-
Inadequate baselines. Despite extensive related work, the experiments primarily compare against fixed-size GRPO. There are no empirical comparisons with closely related adaptive sampling methods such as NGU, AR3PO, AERO, or even DAPO-style dynamic sampling. Outperforming vanilla GRPO is insufficient to establish the effectiveness of the proposed approach relative to existing methods.
-
Insufficient experimental rigor. The main experiments lack multi-seed evaluations and report peak performance on a small 51-problem evaluation set. More importantly, most GRPO baselines exhibit severe training collapse. It is unclear whether the reported performance gains reflect the effectiveness of ARCA or the instability of the chosen baselines.
-
Unclear presentation and inconsistent experimental evidence. The paper's overall narrative is difficult to follow, and the appendix contains substantial historical results and defensive explanations that obscure the final method and its supporting evidence. A particularly concerning example is the hard-task analysis in Section 5 (Figure 6): Appendix A.5 explicitly states that these results use a historical sampler that resamples both all-correct and all-wrong groups, rather than the proposed drop-all-correct variant. Since these variants have different stopping rules and sampling distributions, the analysis does not directly support the claimed hard-task improvements of the final method.
References
-
NGU: Noukhovitch, M., et al. (2026). Learning to Solve Hard Problems in RL for LLMs by Never Giving Up. arXiv:2609.13443
-
Reinforce-Ada: Xiong, W., et al. (2025). Reinforce-Ada: An Adaptive Sampling Framework under Non-linear RL Objectives. arXiv:2510.04996
-
SARA: Nomand, P., et al. (2026). Early Verdicts, Better Budgets: Sequential Adaptive Rollout Allocation for Compute-Efficient RLVR. arXiv:2607.26253
-
AR3PO: Zhang, Y., et al. (2025). Improving Sampling Efficiency in RLVR through Adaptive Rollout and Response Reuse. arXiv:2509.25808
-
AERO: Zhang, Z., et al. (2026). Train Less, Learn More: Adaptive Efficient Rollout Optimization for Group-Based Reinforcement Learning. arXiv:2602.14338
-
DAPO: Yu, Q., et al. (2025). DAPO: An Open-Source LLM Reinforcement Learning System at Scale. NeurIPS 2025. arXiv:2503.14476
Questions for authors
-
What is the key algorithmic contribution of ARCA beyond NGU, AR3PO, and AERO? Why are these methods not included as baselines?
-
Why do most GRPO baselines collapse so severely? Do the improvements persist against stable, properly tuned baselines and across multiple seeds?
-
Can the authors reproduce the hard-task analysis in Figure 6 using the final drop-all-correct algorithm rather than the historical variant?