- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies admission control for mixed RL rollout serving, where agentic, RLVR, and RLHF rollouts share an asynchronous inference pool. The authors observe that prefix-aware routing (e.g., vLLM Router's cache-aware policy) decides where a request goes but not how many sessions may occupy an instance, so under high concurrency new sessions evict reusable prefixes and throughput collapses. They propose MISA-T, a router-layer policy with three components: a dynamic global session cap driven by a reference KV footprint and contracted under confirmed prefix-hit degradation; per-class soft caps that split protected KV blocks in proportion to demand x footprint (MISA); and residency-time weighting of that demand, including tool/environment gaps for agent sessions (MISA-T). Rollout-only ablations on Step3.7 (196B-A11B) and Qwen3.6-35B-A3B report +53.3% / +43.6% sample throughput over a sweep-tuned vLLM Router. A 50-iteration end-to-end Step3.7 run reports +35.6% rollout throughput, -22.8% iteration time, a consumed mixture slightly closer to the trainer target (TVD 2.71 vs 4.14 pp), and pass@4 within 0.5 pp on three SWE benchmarks.
Critical strengths
S1. The problem is real and the diagnosis is convincing. The paper correctly identifies that cache-aware placement does not bound KV contention across heterogeneous sessions. The high-load row of Table 3 (prefix hit 92.4% -> 4.5%, throughput -55%) and the Appendix A stress test make the failure mode concrete and motivate admission control as a distinct lever from routing.
S2. The ablation cleanly separates the three mechanisms. Session Admission -> MISA -> MISA-T is evaluated on both models under identical controller settings, so the reader can see that global admission alone recovers +12-19%, class allocation adds ~10%, and residency weighting adds the remainder. This is more disciplined than most serving-systems papers.
S3. Experiments are at production scale with end-to-end validation. A 196B sparse-MoE model on multi-node H200 clusters, a second open model on H100s, an end-to-end RL run that tracks iteration time and consumed mixture, and downstream task scores rather than serving metrics alone. The compatibility check with CPU KV offloading (Sec. 5.5) also shows the method composes with existing memory tiers.
Critical weaknesses
W1. Statistical rigor does not match the size of the claims. Table 3 reports only means over three runs with no variance or confidence intervals; Table 2 and Figure 3 are a single 50-iteration run. Several deltas that carry the narrative are ~10% (MISA vs Session Admission on both models, MISA-T vs MISA on Qwen), which is within plausible run-to-run noise for throughput on a shared cluster. The "<0.5 pp" pass@4 claim is given without the actual numbers or any seed variance.
W2. Two central results are unexplained and partly contradict the stated mechanism. (a) On Qwen3.6-35B-A3B, MISA drops prefix-hit rate from 96.3% to 79.2% while increasing throughput; since Sec. 4.2 says class caps never exceed the global cap, class quotas alone should not add eviction pressure, and the paper offers no account of which mechanism admits the extra evicting sessions. (b) Residency weighting is motivated by tool/environment gaps, which are 6.6% of agent task time on Step3.7 and 22.7% on Qwen (Table 4), yet MISA -> MISA-T gains +24.3% on Step3.7 and only +9.7% on Qwen. The gain is largest where the motivating effect is smallest, suggesting the EWMA is mostly capturing decode-length heterogeneity (RLVR p50 output 47K tokens) rather than tool residency. The paper should disentangle these.
W3. The headline throughput numbers are not interpretable without the consumed mixture. Sample rate counts completed rollout units, and classes differ by orders of magnitude in per-sample cost (Table 1). A policy that drains short RLHF/agent samples faster inflates sample rate without serving the trainer's target. The end-to-end run does report mixture (MISA-T over-serves RLHF by +2.6 pp), but Table 3 -- the source of the +53%/+44% headline -- reports no per-class counts or TVD.
W4. Baselines and experimental conditions are underspecified. The only competitor is vLLM Router with a static cap tuned by a coarse sweep (steps of 32/16); the sweep curve is not shown, and no static per-class cap baseline is included, so "dynamic" vs "class-aware" vs "better static split" is not separated. The outer ceiling given to MISA-T is never stated numerically, although Figure 5 shows it running ~150-400 requests versus ~100-130 for the sweep baseline. Table 5 sets the protected budget to 90% of GPU+CPU blocks, yet Sec. 5.5 claims offloading was enabled with "unchanged" settings despite a 1.3 TB CPU tier. The ~10 controller constants in Table 5 have no sensitivity analysis, so it is unclear whether static-concurrency tuning has been removed or relocated.
Questions for authors
-
What is the exact outer concurrency ceiling for the admission-controlled variants in each experiment, and did the vLLM Router sweep extend to it?
-
Why does MISA on Qwen reduce prefix-hit rate to 79.2% when class caps are bounded by the global cap? Which mechanism admits the extra sessions?
-
Please report per-class completed samples and TVD to target for every row of Table 3.
-
Can you separate the contribution of tool/environment residency from decode-length residency, e.g., by weighting with model-execution time only?
-
In Sec. 5.5, is the protected budget recomputed to include CPU blocks, and what is the +35.6% RPM compared against? (It coincides exactly with the end-to-end sample-rate figure.)