- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces DCOCO, which leverages a parallel drafter like DFlash and directly addressing its main disadvantage, that future tokens are not conditioned on earlier selected tokens. Similarly to DSpark and Domino, DCOCO uses a small auxiliary neural net to model dependencies between drafted tokens and adjusting their logits accordingly. In contrast with DSpark and Domino that iteratively run a sequential corrector, DCOCO calculates the corrected logits in parallel for every pair of consecutive positions, on all their top-K tokens that are suggested by the DFlash backbone. In the experiments, DCOCO consistently achieves the best speedup.
Critical strengths
- The motivation for DCOCO is explained very clearly. The solution is elegant and simple.
- Experiments clearly show that DCOCO achieves the highest speedups compared to the DFlash, DSpark, Domino.
- Evaluations were conducted using vLLM, which is a production-ready, high-performance serving framework, so throughput numbers are likely truthful.
Critical weaknesses
- I'm not convinced the speedup is due to the parallelism of the corrector. The tables in the appendix show that the accepted length is higher than the baselines, which might explain by itself the speedup.
- No comparison to autoregressive drafters like EAGLE.
- The baselines are retrained which raises the concern that they were not trained optimally. Comparison with original checkpoints (in addition) could strengthen the paper.
Questions for authors
What is the evidence that the parallelism of DCOCO is what contributing to the overall speedup, and not the modeling of the corrector itself?
4 comments
Response to Reviewer bpay (2/2)
W2 "No comparison to autoregressive drafters like EAGLE."
Thank you for raising this concern. We initially chose not to include EAGLE3 as a baseline because our method is already compared against three other methods (DFlash, DSpark, and Domino), all of which consistently report outperforming the EAGLE3 autoregressive drafter. We therefore considered an additional comparison with EAGLE3 to be largely redundant.
Nevertheless, we agree that this is a valid concern and have taken this opportunity to evaluate EAGLE3 directly. Below, we present the evaluation results for EAGLE3 trained with the same Appendix A recipe for Qwen3-8B and served with chain drafting, as tree-based drafting is not supported in vLLM 0.27 due to the complexity it introduces in a production system and its lower performance in higher concurrencies. EAGLE3 results are shown alongside the corresponding DCOCO results for ease of comparison. We tuned EAGLE3's number of speculative tokens per evaluation benchmark for throughput and reported the best setting.
Table R6. Throughput (tok/s) / mean accepted length τ of EAGLE3 and DCOCO, Qwen3-8B, T=0.
| Dataset | EAGLE3 | DCOCO |
|---|---|---|
| MATH-500 | 639.3 / 3.96 | 1625.2 / 8.04 |
| AIME-25 | 616.7 / 3.81 | 1436.8 / 7.26 |
| GSM8K | 501.3 / 2.81 | 1175.8 / 6.12 |
| HumanEval | 479.2 / 2.47 | 1075.6 / 5.44 |
| MBPP | 458.9 / 2.37 | 979.1 / 5.21 |
| LiveCodeBench | 482.7 / 3.05 | 1328.5 / 6.98 |
| MT-Bench | 340.4 / 1.67 | 494.7 / 2.93 |
| Arena-Hard-v2 | 330.1 / 1.58 | 486.9 / 2.74 |
| Alpaca | 325.8 / 1.58 | 496.1 / 2.56 |
| SPEED-Bench | 305.8 / 1.39 | 498.4 / 2.56 |
| Average | 448.0 / 2.47 | 959.7 / 4.98 |
Table R7. Throughput (tok/s) / mean accepted length τ of EAGLE3 and DCOCO, Qwen3-8B, T=1 (mean over seeds 42/43/44).
| Dataset | EAGLE3 | DCOCO |
|---|---|---|
| MATH-500 | 573.4 / 3.40 | 1145.5 / 5.75 |
| AIME-25 | 495.0 / 2.72 | 828.7 / 4.49 |
| GSM8K | 488.3 / 2.70 | 1081.3 / 5.44 |
| HumanEval | 454.2 / 2.35 | 910.0 / 4.65 |
| MBPP | 429.7 / 2.24 | 848.3 / 4.43 |
| LiveCodeBench | 472.7 / 2.74 | 1013.7 / 5.43 |
| MT-Bench | 336.3 / 1.62 | 450.8 / 2.58 |
| Arena-Hard-v2 | 308.9 / 1.49 | 436.8 / 2.29 |
| Alpaca | 305.5 / 1.38 | 447.5 / 2.31 |
| SPEED-Bench | 298.7 / 1.35 | 459.5 / 2.37 |
| Average | 416.3 / 2.20 | 762.2 / 3.97 |
On average, DCOCO is 2.1× faster than EAGLE3 at T=0 (959.7 vs. 448.0 tok/s) and 1.8× at T=1 (762.2 vs. 416.3), with roughly twice the accepted length at both temperatures.
W3 "The baselines are retrained which raises the concern that they were not trained optimally. Comparison with original checkpoints (in addition) could strengthen the paper."
We thank the reviewer for raising this concern. We retrained all methods to ensure a controlled comparison: every method shares the same training data, drafter backbone, verifier layers, and serving stack, so that differences in results reflect the methods themselves rather than their training setups. Since DCOCO is built on DFlash, we followed DFlash's published data recipe. The exact samples drawn from the Nemotron Post-Training Dataset v2 are not disclosed, so we drew our own sample according to that recipe and fixed it across all methods. To guard against under-training, we also extended every baseline to twelve epochs under its own published objective (Appendix B, Table 4). Each changed accepted length by at most 1.9%, indicating that the baselines had converged.
Officially released checkpoints are a less controlled reference. They are typically intended for production use and trained on larger data than the published recipe, so a comparison against them would conflate the method with the training budget. Retraining every method under one recipe, together with the convergence check above, is what allows the comparison to isolate the methods themselves.
We thank the reviewer again for these questions. We hope the new experiments, which isolate the contribution of parallel execution, add an EAGLE3 comparison, and clarify our training protocol, address the concerns raised. We would be glad to discuss any remaining questions.
Sincerely, the authors
Thank you for the response. Reviewer Nsr2 pointed out that DSpark is worse than DFlash in Table 1. This result is surprising to me, and it raises the concern that the training recipe for the baselines is not optimal. Furthermore, on Llama-3.1-8B at T=1 on Arena-Hard-v2 (Tables 10 and 11), DCOCO's accepted length tau is only 3.5% above DFlash's (1.499 vs 1.449), yet its throughput is 38% higher (310.7 vs 224.7 tok/s). This does not make sense to me, as DCOCO has DFlash as backbone + your corrector. Therefore, I am keeping my score.
We thank the reviewer for the continued engagement and address both points below.
DSpark vs. DFlash. The reviewer’s observation reflects a central finding of our work. Under our shared training recipe, DSpark does improve on DFlash in accepted length: on Qwen3-8B at T=0, its average accepted length is 8.7% higher (4.254 vs. 3.913). Its throughput is nevertheless lower (765.6 vs. 783.1 tok/s), because at block size 16 the cost of its sequential correction step outweighs the acceptance it gains. Correction improves throughput only when its cost is small relative to the acceptance it adds. This is the motivation for DCOCO’s design, which keeps correction lightweight and parallel so that gains in accepted length carry over to throughput.
Llama-3.1-8B on Arena-Hard-v2. We agree with the reviewer that throughput would generally be expected to track accepted length on a drafter with larger overhead, and this relationship holds for Qwen3-8B and Qwen3-4B. For Llama-3.1-8B, the measured throughput gap is larger than the difference in accepted length alone would suggest. Throughput also depends on factors beyond accepted length, including model-specific layout, serving and kernel behavior, output-length distributions, and request-level aggregation. Characterizing these factors for each model is beyond the scope of this work, and we report the results as measured.
To confirm that these measurements are stable, we repeated the Llama-3.1-8B evaluation five additional times for both DCOCO and DFlash. All runs reproduced the originally reported results to within 1.5%. To support independent verification, we have released anonymized DCOCO and DFlash checkpoints for Llama-3.1-8B. The supplementary material contains the evaluation code and step-by-step instructions for reproducing our setup.
We hope these clarifications address the reviewer’s concerns and would be glad to discuss further.
Response to Reviewer bpay (1/2)
We thank the reviewer for the constructive comments and for recognizing the clarity of the motivation, the simplicity of the method, and the reliability of our vLLM-based throughput measurements.
We thank the reviewer for this question and the reviewer is correct that both factors contribute. Throughput is the number of tokens accepted per drafting round divided by the time a round takes, and DCOCO improves both. To isolate what parallelism contributes, we ran a new experiment that keeps the corrector's modeling fixed and changes only how it is executed.
DCOCO normally scores every pair of adjacent candidates at all positions at once, then picks the draft left to right. The pick at each position depends only on the token chosen at the previous position. So the same scorer can also run one position at a time as a sequential corrector instead of a parallel one, scoring only the 16 transitions from a chosen token at each position. This produces exactly the same draft (verified on every input), so accepted length is unchanged.
We timed the correction step of both versions using the same modules our vLLM integration runs (Qwen3-8B, one B200, one request from HumanEval):
Table R5. Timing table for the parallel and sequential corrector at each block size, Qwen3-8B, T=0 (block sizes above 16 exceed the deployed setting).
With identical drafts, parallel execution makes the correction step faster, and the advantage grows with every extra drafted token, isolating the benefit of parallelism.
We attribute the speedup to better utilization of the GPU. Serving one request, each correction operation is tiny and the GPU's compute units sit mostly idle, so the number of dependent steps sets the time, not the FLOPs. The sequential version needs one step per drafted token, while DCOCO scores all positions in a single step, using "free" FLOPs that would otherwise go unused.
To translate this into end-to-end throughput: a decoding round on Qwen3-8B at T=0 takes about 5.2 ms (4.985 tokens per round at 959.7 tok/s). Executing the same corrector sequentially would add about 0.38 ms per round at block size 16, lowering throughput by roughly 7% with identical drafts. The corrector's modeling raises accepted length, and parallel execution keeps the added scoring off the critical path. The effect grows with load. In Figure 4, Domino keeps its accepted-length gain over DFlash at every concurrency, yet its throughput falls below DFlash as its sequential correction overhead accumulates, while DCOCO's accepted-length and throughput gains move together.