- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
- The paper studies the rollout phase of agentic RL and argues that prefill-decode disaggregation is a good fit here, because multi-turn interaction and asynchronous sample injection create a continuous stream of prefill requests mixed with decode.
- Section 3 characterizes the workload on three tasks and names three benefits of PD disaggregation: less interference across data-parallel attention ranks in MoE engines, separate parallelism per stage, and lower inter-token latency for long-tail trajectories.
- Prorsa adds a workload predictor that uses the trajectory lifecycle to forecast prefill tokens and decode KV need over a 10s window, and a role switcher that flips rollout instances between prefill and decode roles.
- Implemented on VeRL + SGLang and evaluated on Qwen3-235B, DeepSeek-V3.2 and MiMo-V2.5-Pro on 128 GPUs. Rollout speedup over PD colocation is 1.11–4.05x on the two agentic tasks; on math it is a slowdown in most settings. Prediction WAPE is 5–14%.
Critical strengths
- The workload characterization is informative. Agentic tasks keep issuing prefill all through a step, the prefill/decode token ratio moves up to 1.74x across steps and a lot within a step, and colocated MoE engines lose a lot of GPU time to cross-DP-rank interference. This is a clear, measured story that people building RL infra can use right away.
- The experiments are at a scale very few groups can run, and the paper is honest when the method loses (math task in Sectio 5.2).
- The predictor is simple and sensible, with online bias correction, and the paper actually measures its accuracy instead of assuming it works.
- The writing and figures are clear and easy to follow.
Critical weaknesses
- Most of the headline speedup comes from PD disaggregation itself, not from the new idea. The authors list that GLM-5, MiniMax-M2 and RollArt already use PD disaggregation for agentic RL rollout. The new part, prediction-guided role switching, is isolated only in Fig. 7, only for MiMo-V2.5-Pro, and gives 1.0x–1.44x over a static PD split. So the real contribution is a good characterization plus a 0–44% gain on one model, while the abstract reads like 4.05x. The static-PD baseline should be shown for all three models and all tasks.
- The only baseline is colocation, and that baseline may be weak. Methods that attack the same interference, like in-place PD multiplexing, or scheduling systems like Libra / Heddle / RollPacker / OrchestrRL are never compared. Recent accepted RL-infra-flavored papers (for example the STRONGER-MAS, or RollArt / StreamRL) position against the strongest alternative, not only against the naive one.
- No evidence that the training outcome is unchanged. Role switching cancels in-flight requests and resumes them on another instance from the persisted KV cache; in async mode the resumed part can run under a newer policy version. The paper never shows a reward curve, a final accuracy, or an end-to-end training time for Prorsa vs colocation, only rollout time. For an ML venue this is a gap.
- Reproducibility is low: a proprietary 1T model, a 128-GPU cluster, no code release mentioned, and the GPU-idle metric excludes scheduling overhead and cross-DP imbalance - more like an MLSys submission.
Questions for authors
- Can you show Fig. 7 static PD vs Prorsa for Qwen3-235B and DeepSeek-V3.2 and for all staleness settings, and add at least one dynamic-scheduling baseline (Libra, Heddle or OrchestrRL) and one PD-multiplexing baseline? This would tell us how much of the gain is really from role switching.
- Does Prorsa change the trained model? Please report training reward / eval accuracy and total time-to-target for at least one task with and without Prorsa, and explain how the cancel-and-resume path interacts with importance ratios when a trajectory continues under a newer weight version.
- Multi-agent workflows create even more prefill churn. Does the predictor and the switcher extend to that case, or is the current design tied to one policy and one engine type?
We sincerely thank you for your thoughtful feedback. We address each question below.
We have conducted additional experiments and added static PD baselines for all models and tasks (Figure 5 in Section 5.2). Prorsa achieves a 1.00–1.88× speedup over static PD, with speedups of up to 1.55× on agentic tasks. We have also added a comparison with a recent-load prediction baseline (Figure 8 in Section 5.4), showing that Prorsa reduces WAPE by a factor of 1.31–2.51 compared with this baseline.
We have added score curves in Figure 11 (Section 5.5). Under the same training settings, Prorsa and PD colocation follow similar learning trends and both reach scores of approximately 80 at the end. Specifically, Appendix C explains how cancel-and-resume retains generated prefixes and KV state to maintain model behavior.
We have added a discussion of multi-agent workflows in Section 4.2. For these workflows, the lifecycle abstraction should track agent branches and aggregate their prefill and decode demand. Moreover, heterogeneous models require separate resource pools and model-specific predictions. We believe that Prorsa can also be applied to multi-agent workflows.
For reproducibility, we have added batch sizes, group sizes, sequence and turn limits, and sampling acceptance rates in Appendix D. We compute the fraction of GPU time lost to P/D interference by dividing the corresponding idle GPU time by the total GPU time across all GPUs over the entire rollout.