- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In asynchronous GRPO-style post-training the queue fixes the order of optimizer batches, and persistent Adam state makes that order matter. The paper formalizes this "batch path", proposes an admission contract with an exact allocator, and audits it with a pre-registered six-seed 0.6B experiment over eight updates. Reversing the two tool-family blocks shifts product-minus-shared success by −23.6 pp; a first-moment reset attenuates this but lowers aggregate success, and a larger FIFO-versus-contract study is null.
Critical strengths
- Clean isolation of the batch-path treatment. Recording all trajectories before assigning availability and byte-identical fixed-batch checkpoints (Appendix B.2–B.3) make "same experience, different path" an estimand, and pre-registration lets the −23.6 pp effect be taken at face value.
- Transparent reporting of failed and unconfirmed extensions. Appendix D and Table 9 document every stopped, invalid or inconclusive study with its frozen decision rule.
Critical weaknesses
- Headline effect is Adam's prediction at T=8; no longer horizon tested. With T=8 below Adam's time constant 1/(1−β₁)=10, idealised constant-gradient accounting gives the first family ≈6.2 update-equivalents and the second ≈1.8, so the −23.6 pp reversal in Table 3 is, in my reading, the expected momentum effect. Absent a β₁=0/SGD ablation or horizon sweep, this confirms only Sweeney (2026)'s momentum-favors-the-first-comer result.
- The reset's "capability cost" follows from preserving the Adam clock. Zeroing m before step 5 leaves the second family at ≈1.8 and removes only first-family coasting, which qualitatively predicts Table 5 (first family −13.80 pp, second −1.37 pp); the claimed "material first-moment dependence" is, in my reading, a design artifact, not a queue property.
- Real queues never linked to learning; contract never shown to help. The closest thing to a natural queue (3B FIFO-versus-contract) is null under the pre-registered rule (+2.08 pp [−1.11, +5.34], Appendix D.1), and the learner-free τ²-bench traces fail the paper's joint rule, leaving "agentic RL" supported by trace timing alone.
Questions for authors
- Report absolute update-8 held-out success for source, balanced and blocked paths; how far do eight steps move the model?
- Do saved Adam diagnostics match the accounting above (≈6.2 vs 1.8; 4.0 vs 1.8 after reset), and did you try a full reset or β₁=0?
- Run a horizon sweep (T up to 512) under measured service-trace imbalance plus a β₁=0/SGD control to see where the interaction decays.