- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DStreamer and DStreamer++, a speculative decoding framework for high-concurrency LLM serving and online RLVR. Building on the shared-KV adapter design of Speculative Streaming (which drafts tokens in parallel without a separate draft KV cache), DStreamer warm-starts the adapter with masked diffusion consistency distillation and then fine-tunes it via GRPO on complete candidate paths to maximize autoregressive acceptance. To handle policy shifts during online RLVR, DStreamer++ continuously updates the adapter on rollout windows. Experiments on Qwen3 and GPT-OSS show up to serving throughput gains over AR (outperforming EAGLE-3 and DFlash) and end-to-end RLVR speedup with no loss in final accuracy.
Critical strengths
-
Practical algorithm-system co-design for online RLVR and high-batch decoding. Reusing the target KV cache directly avoids the memory bottleneck of separate draft models at large batch sizes and long contexts. More importantly, DStreamer++ makes online drafter RL practical in synchronous FSDP training: by sampling multiple candidate paths from a single adapter forward pass and reusing the rollout KV cache in place, online adaptation adds only ~3.8% step-time overhead while effectively preventing acceptance drop over training.
-
Strong empirical validation in realistic high-throughput regimes. Rather than focusing on batch-size-1 latency, the paper evaluates continuous batching up to concurrency 1024, context scaling up to 8K (where baseline drafters OOM), and multi-seed end-to-end RLVR runs measured in wall-clock GPU-hours vs. benchmark accuracy. This convincingly shows that the proposed method delivers real wall-clock speedups where rollout cost actually dominates.
Critical weaknesses
- Training gains are not cleanly separated from the inherited shared-KV architecture. DStreamer adopts its in-model adapter and target-KV reuse directly from Speculative Streaming, so the core novelty lies in the two-stage training recipe. However, the main serving and RLVR benchmarks do not include either the original Speculative Streaming training method or the Stage-1 consistency-distilled checkpoint as baselines. While Figure 3 shows that RL improves over the distilled initialization on a validation set, it does not show how much this translates into actual wall-clock throughput in the main benchmarks. We need a controlled comparison across the original Speculative Streaming recipe, the Stage-1 distilled adapter, and the final RL-trained adapter on the same architecture and data (with matched training budgets), reporting both accepted length and end-to-end throughput. Without this, it is hard to tell how much of the advantage over EAGLE-3 and DFlash comes from the proposed training versus the zero-KV architecture itself.
Questions for authors
- Controlled ablation of the two-stage training recipe on the same shared-KV architecture: Since DStreamer uses the same inference architecture and KV-sharing mechanism as Speculative Streaming, could you provide a controlled comparison across (i) the original Speculative Streaming training recipe, (ii) the Stage-1 consistency-distilled checkpoint (before RL), and (iii) the full Stage-2 RL-trained DStreamer adapter on the same architecture and training data (with training compute/steps matched or clearly reported)? Specifically, how much does each stage improve mean accepted length () and actual vLLM serving throughput on the main benchmarks (e.g., Qwen3-4B on GSM8K and HumanEval)? Seeing these numbers would directly clarify how much of the speedup over EAGLE-3 and DFlash comes from your training recipe versus the zero-KV adapter design itself.
2 comments
Thank you for the additional experiments and clarification, the ablations address my concern about the attribution of the two-stage training gains. After considering the updated evidence and overall strengths, limitation of the paper, I will maintain my original score.
We appreciate the reviewer's suggestion to isolate the training contributions from the inherited shared-KV architecture. We have added a dedicated Component Analysis (Sec. 4.2, Fig. 3; Appendix E.1, Fig. 12), reporting both accepted length and vLLM serving throughput on GSM8K and HumanEval. We compare Speculative Streaming-style direct CE training, consistency-distilled diffusion, and their RL-optimized counterparts using the same shared-KV adapter architecture and training data. Both initializations use 30K optimizer steps, followed by the same 10K-step acceptance-aligned RL procedure.
The controlled comparisons show how each training stage contributes:
Diffusion improves initialization. Training on partially masked windows provides an easier reconstruction task than predicting all future tokens at once. Consistency distillation transfers this benefit to single-pass drafting, improving accepted length from 3.08 to 3.33 on GSM8K (Fig. 3a) and 2.58 to 2.69 on HumanEval (Fig. 12b).
Acceptance RL builds on this advantage. By optimizing complete proposal paths for prefix compatibility rather than individual token predictions, RL further improves both initializations. Diffusion retains its advantage after RL: 3.74 vs. 3.59 on GSM8K (Fig. 3c) and 2.83 vs. 2.72 on HumanEval (Fig. 12b).
Both stages improve serving throughput. Full DStreamer achieves the highest vLLM throughput across evaluated batch sizes on GSM8K (Fig. 3b) and HumanEval (Fig. 12a). At batch size 256 on HumanEval, throughput increases from approximately 13.1K tokens/s with direct CE to 13.9K after diffusion initialization, and 14.6K after acceptance RL. In comparison, direct CE + RL reaches 14.01K tokens/s.
By holding the shared-KV architecture fixed, these comparisons isolate the acceptance and serving-throughput improvements attributable to each training stage. Diffusion provides a stronger single-pass initialization, while acceptance-aligned RL further improves its usefulness under autoregressive verification.