- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes AOSR, an asynchronous speculative rollout framework for accelerating on-policy distillation of autoregressive image generators. The key observation is that batch-synchronous speculative rollout suffers from substantial shared-boundary waiting because different sequences make uneven verified progress. AOSR addresses this through sequence-wise asynchronous advancement, adaptive batch execution, and CSR-based cross-step prefetching, achieving clear rollout and end-to-end training speedups while largely preserving generation quality. The work has strong practical motivation and solid empirical validation, but the main contribution is primarily execution-level optimization with limited algorithmic novelty, and its gains may depend on sequence-progress heterogeneity and speculative acceptance behavior. Overall, the system design is effective and practically useful, though the algorithmic contribution is relatively modest.
Critical strengths
The paper identifies shared-boundary waiting as a major source of inefficiency in speculative OPD rollouts, and the proposed asynchronous execution directly addresses this bottleneck with substantial end-to-end training speedups.
Critical weaknesses
-
The main contribution is largely an execution-level redesign of OPD rollout, combining asynchronous advancement, dynamic batch adaptation, speculative decoding, and cross-step prefetching. While effective, the teacher, distillation objective, and optimization procedure remain unchanged, making the work more system-oriented than algorithmically novel.
-
AOSR benefits from heterogeneous sequence-wise progress and idle batch slots; if speculative acceptance rates are low or different sequences progress at similar speeds, the advantage of asynchronous scheduling and CSR may become much smaller. More systematic sensitivity analysis across such regimes would strengthen the paper.
Questions for authors
-
How much of the speedup comes specifically from the proposed asynchronous scheduling, rather than from existing speculative decoding, dynamic batching, and prefetching techniques?
-
How sensitive is AOSR to sequence-progress heterogeneity and speculative acceptance rates? In what regimes does the speedup become marginal?
-
Can the proposed framework generalize beyond autoregressive image OPD, for example to video generation or other rollout-intensive post-training settings?
7 comments
For W2. Sensitivity to Sequence-progress Heterogeneity
We thank the reviewer for this concern. We would like to clarify that limited progress heterogeneity does not necessarily imply poor overall acceleration, as speculative decoding itself also contributes to AOSR's speedup. When sequences advance rapidly, AOSR can remain efficient even with limited asynchronous gains (the impact of speculative acceptance is further examined in Q2). For CSR, more synchronized completion reduces opportunities for cross-step prefetching, potentially limiting its additional benefits.
To examine progress heterogeneity in practice, we analyze 50 OPD training batches.
Table 1. Verified Progress Heterogeneity in Real OPD Rollouts.
| Metric | Value |
|---|---|
| Average Acceptance Rate | 0.742 |
| Sequences with less than 30% progress | 0.162 |
| Sequences with less than 50% progress | 0.356 |
| Sequences with less than 80% progress | 0.756 |
When the first sequence finishes, 35.6% of the remaining sequences have completed less than half of their generation, indicating substantial progress heterogeneity and opportunities for both AOSR and CSR.
We further evaluate synchronization sensitivity by progressively constraining sequences to wait for the slowest sequence in their group, without changing the verification procedure.
Table 2. Impact of Controlled Synchronization on Rollout Latency.
| Constrained Sequences | Rollout Latency (ms) ↓ | Speedup vs. Fully Sync. ↑ |
|---|---|---|
| 0% (AOSR) | 3904.18 | 1.53× |
| 10% | 3924.34 | 1.52× |
| 50% | 4765.14 | 1.25× |
| 80% | 5482.63 | 1.09× |
| 100% (Fully Synchronized) | 5972.81 | 1.00× |
The results show that increasing synchronization constraints reduces the additional benefit of asynchronous advancement. Nevertheless, fully synchronized SJD++ still achieves a 1.34× speedup over AR (5972.81 vs. 8003.57 ms), confirming that reduced asynchronous gains do not necessarily eliminate overall speculative rollout acceleration.
For Q1. Contribution of Asynchronous Scheduling
We would like to clarify that asynchronous advancement alone reduces rollout latency by 30.9% (1.45× speedup) compared with synchronous execution using the same SJD++ verifier. This contribution has already been isolated in the component ablation in Table 4 of our original submission, reproduced below for clarity.
| Variant | Rollout Latency (ms) ↓ |
|---|---|
| GKD (AR) | 8003.57 |
| Batch-Sync. SJD | 8514.44 |
| + SJD++ Draft Refinement | 5972.81 |
| + Async. Advancement | 4125.26 |
| + Batch Adaptation (AOSR) | 3904.18 |
As shown above, asynchronous advancement provides the primary improvement among the execution optimizations after SJD++ refinement, while batch adaptation further reduces latency by 5.4%. The contribution of CSR is reported separately in Table 1 of our original submission, where adding CSR to AOSR improves E2E speedup from 1.52× to 1.63× in the LlamaGen-L/GKD setting.
These results clearly isolate the contributions of draft refinement, asynchronous advancement, batch adaptation, and cross-step prefetching.
For W1. Research Insights and Algorithmic Contributions
We thank the reviewer for recognizing the effectiveness of our method. We would like to clarify that unlike OPD acceleration approaches that reduce or adapt rollout lengths, our work focuses on improving student rollout execution efficiency without changing the distillation procedure. Specifically, the asynchronous advancement, dynamic batch adaptation, and cross-step prefetching in our framework are designed to address the challenges of adapting speculative decoding to OPD rollouts.
As demonstrated in our paper, directly applying batch-synchronous speculative decoding can be slower than standard AR due to heterogeneous verified progress and synchronization overhead. To address this, we develop asynchronous advancement and adaptive batch execution to eliminate synchronization bottlenecks, and further introduce CSR for cross-step computation reuse with post-update re-verification. These problem-driven algorithmic designs are tailored to the characteristics of OPD, achieving substantial rollout and end-to-end training acceleration while maintaining comparable generation quality.