- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
PACE is an admission controller for asynchronous GRPO that decides which completed prompt groups enter training. Pool occupancy sets a smoothed rejection budget, while candidates are ranked by waiting age plus generation age discounted by prefix drift. The score separates policy updates crossed during generation from those crossed while a completed group waits in the pool. The controller admits whole groups, preserving the group-relative advantage structure and the form of the policy loss while changing the distribution of training data. Experiments on Qwen3 dense and MoE backbones include a ReTool agentic setting. Matched full/raw comparisons support a benefit from prefix scoring, while the larger ReTool comparison with Async uses different rollout resources.
Critical strengths
S1. The controller treats the two sources of lag differently, and this is the paper's most useful design idea. Occupancy decides how much to reject, while prefix drift discounts only the generation age and leaves the waiting penalty intact. Because admission operates on whole prompt groups, GRPO's group-relative advantage structure survives the filtering.
S2. A controlled comparison shows that the prefix-drift score adds value beyond raw staleness. With matched resources and the same pool controller, full PACE improves the selected six-benchmark validation mean by 2.09 points in both tested settings: 41.29 versus 39.20 for Qwen3-8B and 47.91 versus 45.82 for ReTool.
Critical weaknesses
W1. The 18.67-point ReTool gain over Async comes from runs with different rollout allocations. PACE uses 4r2t and Async uses 2r2t. Extra capacity can change sample supply and pool backlog, so dividing by GPU-hours does not isolate the admission policy's contribution. The matched full/raw comparison does support a 2.09-point gain from prefix scoring, but that control measures an incremental component benefit and cannot explain the size of the full/Async gap.
W2. The MoE result needs a check that the trainer and rollouter agree numerically. Behavior log probabilities are recorded, but the paper does not compare token scores from the two engines at the same checkpoint. Routing replay is disabled, and prefix scoring is performed on the rollout side. A discrepancy between the engines would enter the policy ratio even without a policy-version change.
Questions for authors
Q1. How large is the full PACE/Async gap with the same rollout/trainer allocation and checkpoint-selection rule? An existing matched comparison, or one matched Async run, with available training-run variation would address the attribution issue. Please also report GPU-hours to a common validation target and identify the runs and timing boundaries used for the cost measurements.
Q2. At a fixed MoE checkpoint and on identical sequences, how large are the differences between trainer and rollouter log probabilities? Please report policy-ratio and clipping statistics as well, and investigate routing only if the difference is material.