acceptodds
Under review as a conference paper at ICLR 2027

Contextual Correction of Parallel Drafts for Speculative Decoding

Abstract

Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are subsequently verified in parallel by the target model. Recent diffusion-based drafters make this paradigm particularly attractive: deep attention-based neural backbones provide strong predictions, while drafting multiple future positions in parallel avoids the sequential cost of autoregressive generation. This parallelism, however, comes at a cost. Tokens within a draft block are predicted without access to the realized identities of preceding draft tokens, removing intra-block dependencies that would naturally be available under autoregressive generation. We introduce **DCOCO** (**D**Flash **Co**ntextual **Co**rrector), a lightweight mechanism that restores these dependencies while preserving parallel drafting. Rather than committing to each position's top prediction, DCOCO retains multiple candidate tokens and evaluates their contextual compatibility across adjacent draft positions in parallel, enabling contextually informed token selection at low additional cost. Across four verifier backbones and ten benchmarks, DCOCO achieves the highest average throughput in every evaluated configuration, spanning greedy and stochastic decoding with and without extended reasoning, with speedups of up to 6.6× over vanilla decoding. Its advantage persists under concurrent serving: DCOCO outperforms the strongest competing method by 6–23% across 4–32 concurrent requests, while other sequential correctors fall below uncorrected DFlash at these loads. These results demonstrate that recovering intra-block dependencies through parallel contextual scoring delivers consistent acceleration across decoding regimes and serving loads.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Speculative Decoding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_bpayOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces DCOCO, which leverages a parallel drafter like DFlash and directly addressing its main disadvantage, that future tokens are not conditioned on earlier selected tokens. Similarly to DSpark and Domino, DCOCO uses a small auxiliary neural net to model dependencies between drafted tokens and adjusting their logits accordingly. In contrast with DSpark and Domino that iteratively run a sequential corrector, DCOCO calculates the corrected logits in parallel for every pair of consecutive positions, on all their top-K tokens that are suggested by the DFlash backbone. In the experiments, DCOCO consistently achieves the best speedup.

Critical strengths
  1. The motivation for DCOCO is explained very clearly. The solution is elegant and simple.
  2. Experiments clearly show that DCOCO achieves the highest speedups compared to the DFlash, DSpark, Domino.
  3. Evaluations were conducted using vLLM, which is a production-ready, high-performance serving framework, so throughput numbers are likely truthful.
Critical weaknesses
  1. I'm not convinced the speedup is due to the parallelism of the corrector. The tables in the appendix show that the accepted length is higher than the baselines, which might explain by itself the speedup.
  2. No comparison to autoregressive drafters like EAGLE.
  3. The baselines are retrained which raises the concern that they were not trained optimally. Comparison with original checkpoints (in addition) could strengthen the paper.
Questions for authors

What is the evidence that the parallelism of DCOCO is what contributing to the overall speedup, and not the modeling of the corrector itself?

4 comments
AuthorsOpenReview ↗

Response to Reviewer bpay (1/2)

We thank the reviewer for the constructive comments and for recognizing the clarity of the motivation, the simplicity of the method, and the reliability of our vLLM-based throughput measurements.

 

W1, Q1 "I'm not convinced the speedup is due to the parallelism…", "What is the evidence that the parallelism of DCOCO is what contributing to the overall speedup, and not the modeling of the corrector itself?"

We thank the reviewer for this question and the reviewer is correct that both factors contribute. Throughput is the number of tokens accepted per drafting round divided by the time a round takes, and DCOCO improves both. To isolate what parallelism contributes, we ran a new experiment that keeps the corrector's modeling fixed and changes only how it is executed.

DCOCO normally scores every pair of adjacent candidates at all positions at once, then picks the draft left to right. The pick at each position depends only on the token chosen at the previous position. So the same scorer can also run one position at a time as a sequential corrector instead of a parallel one, scoring only the 16 transitions from a chosen token at each position. This produces exactly the same draft (verified on every input), so accepted length is unchanged.

We timed the correction step of both versions using the same modules our vLLM integration runs (Qwen3-8B, one B200, one request from HumanEval):

 

Table R5. Timing table for the parallel and sequential corrector at each block size, Qwen3-8B, T=0 (block sizes above 16 exceed the deployed setting).

Block size8163264
DCOCO, all positions at once (µs)144197293481
Same corrector, one position at a time (µs)3195761,1062,157
Speedup from parallel execution2.2×2.9×3.8×4.5×

With identical drafts, parallel execution makes the correction step faster, and the advantage grows with every extra drafted token, isolating the benefit of parallelism.

We attribute the speedup to better utilization of the GPU. Serving one request, each correction operation is tiny and the GPU's compute units sit mostly idle, so the number of dependent steps sets the time, not the FLOPs. The sequential version needs one step per drafted token, while DCOCO scores all positions in a single step, using "free" FLOPs that would otherwise go unused.

To translate this into end-to-end throughput: a decoding round on Qwen3-8B at T=0 takes about 5.2 ms (4.985 tokens per round at 959.7 tok/s). Executing the same corrector sequentially would add about 0.38 ms per round at block size 16, lowering throughput by roughly 7% with identical drafts. The corrector's modeling raises accepted length, and parallel execution keeps the added scoring off the critical path. The effect grows with load. In Figure 4, Domino keeps its accepted-length gain over DFlash at every concurrency, yet its throughput falls below DFlash as its sequential correction overhead accumulates, while DCOCO's accepted-length and throughput gains move together.

AuthorsOpenReview ↗

Response to Reviewer bpay (2/2)

W2 "No comparison to autoregressive drafters like EAGLE."

Thank you for raising this concern. We initially chose not to include EAGLE3 as a baseline because our method is already compared against three other methods (DFlash, DSpark, and Domino), all of which consistently report outperforming the EAGLE3 autoregressive drafter. We therefore considered an additional comparison with EAGLE3 to be largely redundant.

Nevertheless, we agree that this is a valid concern and have taken this opportunity to evaluate EAGLE3 directly. Below, we present the evaluation results for EAGLE3 trained with the same Appendix A recipe for Qwen3-8B and served with chain drafting, as tree-based drafting is not supported in vLLM 0.27 due to the complexity it introduces in a production system and its lower performance in higher concurrencies. EAGLE3 results are shown alongside the corresponding DCOCO results for ease of comparison. We tuned EAGLE3's number of speculative tokens per evaluation benchmark for throughput and reported the best setting.

 

Table R6. Throughput (tok/s) / mean accepted length τ of EAGLE3 and DCOCO, Qwen3-8B, T=0.

DatasetEAGLE3DCOCO
MATH-500639.3 / 3.961625.2 / 8.04
AIME-25616.7 / 3.811436.8 / 7.26
GSM8K501.3 / 2.811175.8 / 6.12
HumanEval479.2 / 2.471075.6 / 5.44
MBPP458.9 / 2.37979.1 / 5.21
LiveCodeBench482.7 / 3.051328.5 / 6.98
MT-Bench340.4 / 1.67494.7 / 2.93
Arena-Hard-v2330.1 / 1.58486.9 / 2.74
Alpaca325.8 / 1.58496.1 / 2.56
SPEED-Bench305.8 / 1.39498.4 / 2.56
Average448.0 / 2.47959.7 / 4.98

 

Table R7. Throughput (tok/s) / mean accepted length τ of EAGLE3 and DCOCO, Qwen3-8B, T=1 (mean over seeds 42/43/44).

DatasetEAGLE3DCOCO
MATH-500573.4 / 3.401145.5 / 5.75
AIME-25495.0 / 2.72828.7 / 4.49
GSM8K488.3 / 2.701081.3 / 5.44
HumanEval454.2 / 2.35910.0 / 4.65
MBPP429.7 / 2.24848.3 / 4.43
LiveCodeBench472.7 / 2.741013.7 / 5.43
MT-Bench336.3 / 1.62450.8 / 2.58
Arena-Hard-v2308.9 / 1.49436.8 / 2.29
Alpaca305.5 / 1.38447.5 / 2.31
SPEED-Bench298.7 / 1.35459.5 / 2.37
Average416.3 / 2.20762.2 / 3.97

On average, DCOCO is 2.1× faster than EAGLE3 at T=0 (959.7 vs. 448.0 tok/s) and 1.8× at T=1 (762.2 vs. 416.3), with roughly twice the accepted length at both temperatures.

 

W3 "The baselines are retrained which raises the concern that they were not trained optimally. Comparison with original checkpoints (in addition) could strengthen the paper."

We thank the reviewer for raising this concern. We retrained all methods to ensure a controlled comparison: every method shares the same training data, drafter backbone, verifier layers, and serving stack, so that differences in results reflect the methods themselves rather than their training setups. Since DCOCO is built on DFlash, we followed DFlash's published data recipe. The exact samples drawn from the Nemotron Post-Training Dataset v2 are not disclosed, so we drew our own sample according to that recipe and fixed it across all methods. To guard against under-training, we also extended every baseline to twelve epochs under its own published objective (Appendix B, Table 4). Each changed accepted length by at most 1.9%, indicating that the baselines had converged.

Officially released checkpoints are a less controlled reference. They are typically intended for production use and trained on larger data than the published recipe, so a comparison against them would conflate the method with the training budget. Retraining every method under one recipe, together with the convergence check above, is what allows the comparison to isolate the methods themselves.

 

We thank the reviewer again for these questions. We hope the new experiments, which isolate the contribution of parallel execution, add an EAGLE3 comparison, and clarify our training protocol, address the concerns raised. We would be glad to discuss any remaining questions.

Sincerely, the authors

Thank you for the response. Reviewer Nsr2 pointed out that DSpark is worse than DFlash in Table 1. This result is surprising to me, and it raises the concern that the training recipe for the baselines is not optimal. Furthermore, on Llama-3.1-8B at T=1 on Arena-Hard-v2 (Tables 10 and 11), DCOCO's accepted length tau is only 3.5% above DFlash's (1.499 vs 1.449), yet its throughput is 38% higher (310.7 vs 224.7 tok/s). This does not make sense to me, as DCOCO has DFlash as backbone + your corrector. Therefore, I am keeping my score.

We thank the reviewer for the continued engagement and address both points below.

DSpark vs. DFlash. The reviewer’s observation reflects a central finding of our work. Under our shared training recipe, DSpark does improve on DFlash in accepted length: on Qwen3-8B at T=0, its average accepted length is 8.7% higher (4.254 vs. 3.913). Its throughput is nevertheless lower (765.6 vs. 783.1 tok/s), because at block size 16 the cost of its sequential correction step outweighs the acceptance it gains. Correction improves throughput only when its cost is small relative to the acceptance it adds. This is the motivation for DCOCO’s design, which keeps correction lightweight and parallel so that gains in accepted length carry over to throughput.

Llama-3.1-8B on Arena-Hard-v2. We agree with the reviewer that throughput would generally be expected to track accepted length on a drafter with larger overhead, and this relationship holds for Qwen3-8B and Qwen3-4B. For Llama-3.1-8B, the measured throughput gap is larger than the difference in accepted length alone would suggest. Throughput also depends on factors beyond accepted length, including model-specific layout, serving and kernel behavior, output-length distributions, and request-level aggregation. Characterizing these factors for each model is beyond the scope of this work, and we report the results as measured.

To confirm that these measurements are stable, we repeated the Llama-3.1-8B evaluation five additional times for both DCOCO and DFlash. All runs reproduced the originally reported results to within 1.5%. To support independent verification, we have released anonymized DCOCO and DFlash checkpoints for Llama-3.1-8B. The supplementary material contains the evaluation code and step-by-step instructions for reproducing our setup.

We hope these clarifications address the reviewer’s concerns and would be glad to discuss further.

Reviewer_Nsr2OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DCOCO, a lightweight contextual corrector for DFlash. It scores adjacent top-K candidate pairs in parallel, then selects a draft from left to right without another backbone pass. The corrector is jointly trained with the drafter. Experiments report improvements in accepted length and throughput across several target models, decoding temperatures, and concurrency levels.

Critical strengths
  1. The corrector is small and avoids another backbone forward pass. Precomputing candidate-pair scores is a practical way to add local dependencies while retaining efficient drafting.

  2. The evaluation covers several target models and concurrency levels, and includes additional training-budget controls. The reported gains make the method worth investigating.

Critical weaknesses
  1. The relationship to DFlash 2 is not discussed, making the additional contribution difficult to assess. Its path selector also retains top-16 candidates, scores adjacent candidate pairs using context, computes these scores in parallel, and leaves only path selection sequential. DCOCO uses an MLP scorer rather than a gated bilinear scorer, but the benefit of its scoring and training choices over this closely related design is unclear. DiffuSpec also provides relevant earlier motivation for causal path selection from parallel token candidates.

  2. The empirical comparison needs clarification. On Qwen3-4B at T=0, DSpark has a lower average accepted length than DFlash (3.889 vs. 4.162), not just lower throughput. It is unclear what accounts for this result and whether the concurrency experiments retain DSpark's confidence calibration and load-aware scheduling, or Domino's correction-loop optimizations. These details affect the claimed wall-clock advantage. The main and ablation tables also report different accepted lengths for the full DCOCO model under apparently identical settings, which further complicates the assessment.

Questions for authors
  1. Please clarify the relationship between DCOCO and DFlash 2's path selector (https://inco.ai/blog/dflash2/). What benefit comes from DCOCO's MLP scorer and training objective compared with the gated bilinear selector? A comparison on the same backbone, without DFlash 2's additional convolutions, would help isolate this contribution.

  2. Were DSpark's confidence calibration and hardware-aware scheduling, and Domino's fused correction kernels, enabled? How were differences in actual draft-token counts handled? Which training or evaluation differences might explain DSpark's lower accepted length than DFlash on Qwen3-4B?

  3. At T=1, did DFlash, Domino, and DSpark sample their drafts stochastically or select them deterministically? Which proposal distributions were used for verification? This would help interpret their acceptance-length comparisons with DCOCO's point-mass proposals.

  4. Why do the full-model accepted lengths differ between Tables 7 and 6 under apparently identical settings: 5.215 vs. 5.858 on MBPP, and 6.982 vs. 6.153 on LiveCodeBench? Please clarify any differences in checkpoints, evaluation data, or aggregation. Timing variation alone would not explain changes in accepted length.

5 comments
AuthorsOpenReview ↗

Response to Reviewer Nsr2 (1/2)

We thank the reviewer for their insightful comments and suggestions. We address each point below.

 

W1, Q1 "The relationship to DFlash 2 is not discussed…", "Please clarify the relationship between DCOCO and DFlash 2's path selector…"

DFlash 2 was released as a technical blog on August 18, 2026, after the July 17, 2026 cutoff in the ICLR reviewer guidelines, which do not require comparison with later or non-peer-reviewed work. We nevertheless agree that it is highly relevant, and conducted additional experiments to characterize the relationship.

Conceptually, the two selectors differ in what a pair score can depend on. DFlash 2 scores a transition as a bilinear match between context-free token embeddings, gated by a single slot's hidden state. DCOCO scores it with a nonlinear function of candidate representations fused with their own slot contexts, a pair state over both slots' hidden states, and both draft logits. Compatibility is thus judged jointly from both positions and the drafter's confidence, rather than from token embeddings modulated by a single slot's context.

Following the reviewer's suggestion, we implemented the DFlash 2 path selector on the same DFlash backbone as DCOCO (without DFlash 2's convolutions), trained it under the same joint-training setting, and compared the two directly with the drafter held fixed.

 

Table R3. DFlash 2 selector vs. DCOCO on the same backbone. Each cell: throughput (tok/s, per stream) / mean accepted length τ. Qwen3-8B, thinking off, concurrency 1, max 2048 tokens; T=1 is the mean over seeds 42/43/44. Bold: best per metric.

DatasetDFlash 2 (T=0)DCOCO (T=0)DFlash 2 (T=1)DCOCO (T=1)
MATH-5001376.5 / 7.3771625.2 / 8.0391176.8 / 5.8281145.5 / 5.755
AIME-251296.3 / 6.8191436.8 / 7.263914.8 / 4.698828.7 / 4.486
GSM8K1012.4 / 5.4701175.8 / 6.118938.4 / 5.1361081.3 / 5.440
HumanEval945.0 / 5.0681075.6 / 5.443815.1 / 4.542910.0 / 4.646
MBPP868.4 / 4.674979.1 / 5.215773.5 / 4.257848.3 / 4.428
LiveCodeBench1200.3 / 6.7581328.5 / 6.982987.0 / 5.2601013.7 / 5.431
MT-Bench547.7 / 2.746494.7 / 2.930496.4 / 2.527450.8 / 2.581
Arena-Hard-v2474.3 / 2.468486.9 / 2.736407.4 / 2.201436.8 / 2.286
Alpaca431.0 / 2.402496.1 / 2.560392.2 / 2.219447.5 / 2.313
SPEED-Bench442.3 / 2.370498.4 / 2.561401.2 / 2.225459.5 / 2.373
Average859.4 / 4.615959.7 / 4.985730.2 / 3.889762.2 / 3.973

Our DFlash 2 selector improves DFlash's accepted length by 17.9% at T=0 and 19.3% at T=1, indicating a faithful and competitive reimplementation; both methods were evaluated with the same vLLM image and protocol.

 

Table R4. Draft time (ms/round, median over decoding steps) of the DFlash 2 selector and DCOCO on the same backbone. Qwen3-8B, HumanEval (164 prompts), block size 16, concurrency 1.

DFlash 2 (T=0)DCOCO (T=0)DFlash 2 (T=1)DCOCO (T=1)
Draft time (ms/round)0.901.110.891.10

The two mechanisms thus sit at different points of the draft cost–accuracy trade-off: DFlash 2 prioritizes a cheaper correction step, while DCOCO invests additional computation in more accurate scoring. Its accepted-length gain more than compensates, yielding higher average end-to-end throughput (+11.7% at T=0, +4.4% at T=1).

Finally, we thank the reviewer for bringing DiffuSpec to our attention. It is relevant to the motivation of parallel speculative drafting: DiffuSpec uses training-free causal path search over a pretrained diffusion model's candidate lattice, whereas DCOCO jointly trains a contextual transition scorer with the drafter. We will discuss it in the related work of the revision.

AuthorsOpenReview ↗

Response to Reviewer Nsr2 (2/2)

W2.1, Q2.1 "…whether the concurrency experiments retain DSpark's confidence calibration and load-aware scheduling, or Domino's correction-loop optimizations", "How were differences in actual draft-token counts handled?"

Domino's fused correction kernels were enabled. DSpark's adaptive verification (confidence-based scoring with load-aware scheduling) was not available in vLLM 0.27, the version used in our original experiments; it was introduced later as the adaptive-verification option. The original comparison therefore used full-draft verification for every method.

 

Draft-token counts. All methods use block size 16: each round proposes up to 15 drafted tokens after the anchor and commits at most 16, including the verifier's token, so per-round draft lengths are matched across methods.

Adaptive verification belongs to the serving pipeline, not the drafter: it estimates each drafted token's acceptance probability and verifies only the most likely tokens under a load-dependent budget, so it is complementary to any drafter. We therefore evaluated DSpark and DCOCO with and without it on vLLM 0.31 (Qwen3-8B, 10 benchmarks, 3 runs each, output tok/s):

Temperature, requestsDSparkDSpark + AVDCOCODCOCO + AV
T=0, 1768708933838
T=0, 329,2249,66311,29911,556
T=1, 1617574720664
T=1, 327,3478,4738,6989,603
  • With the same verification setting, DCOCO exceeds DSpark at 32 requests by 18–23% without adaptive verification and by 13–20% with it; DCOCO without adaptive verification still exceeds DSpark with it in every setting. We will add these results to the concurrency section.
  • For DCOCO we used the most naive confidence estimate: the softmax probabilities of its corrected draft, with no trained parameters. Even so, at 32 requests and T=1 it raises throughput by 10.4%; trained estimators such as DSpark's confidence head are left for future work.
  • At 1 request, adaptive verification costs both methods 7–10%: a single request's forward pass is memory-bound, so verifying fewer tokens saves no time and only loses tokens that would have been accepted.

 

Q2.2 "Which training or evaluation differences might explain DSpark's lower accepted length than DFlash on Qwen3-4B?"

The same training pipeline, data, epochs, and draft length were used for every backbone, with no backbone-specific tuning for any method. Under this pipeline, DSpark improves on DFlash on Qwen3-8B (+8.7% average accepted length at T=0) but falls below it on Qwen3-4B. We do not have a verified explanation for this backbone dependence and report the results as measured. Importantly, it does not affect the comparison with DCOCO: on Qwen3-4B the strongest baseline is Domino (4.861 vs. DCOCO's 5.012 at T=0), and DSpark would need roughly a 20% gain over DFlash to change the ordering, more than twice the gain it achieves on any backbone in our evaluation.

 

Q3 "At T=1, did DFlash, Domino, and DSpark sample their drafts stochastically or select them deterministically?"

At T=1, all methods, including our DFlash 2 reimplementation, select drafts deterministically and are verified with a point-mass proposal under the same rule (Appendix G). This is vLLM's default speculative-decoding behavior, kept identical across methods for a fair comparison. Temperature applies only to the target's sampling, so accepted-length differences reflect only the tokens each method proposes. We will state this protocol in Section 5.

 

W2.2, Q4 "Why do the full-model accepted lengths differ between Tables 7 and 6…"

The same question was raised by Reviewer J3Vj; we refer the reviewer to our response there (Part 1/2, W1.1–W1.2, Q1.1). In brief, the ablation results were mistakenly taken from a run whose DFlash backbone was trained on a different sample of training prompts. We retrained on the prompt set used for all reported methods and replaced Tables 5–6; all other tables were audited and are unaffected.

 

We thank the reviewer again for these comments. We hope the additional experiments and clarifications address the concerns raised, and we would be glad to discuss any remaining questions.

Sincerely, the authors

Thank you again for your thoughtful comments and constructive feedback. We have addressed the points you raised in our responses and in the revised manuscript. If any concerns remain, we would be glad to clarify them before the discussion period closes.

Reviewer_J3VjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DCOCO adds a small corrector to DFlash to improve parallel drafts. It scores neighboring token candidates and adjusts the draft before verification. The experiments cover several models, tasks, and serving loads.

Critical strengths
  1. The design is simple and keeps the scoring step parallel. The improved drafts also lead to higher throughput, which supports the method’s practical value.

  2. The experiments include concurrent serving and a longer-training comparison. The ablations also help explain the benefits of the corrector and joint training.

Critical weaknesses
  1. The method makes a relatively small change, so consistent results are important. The full-model results differ across tables under apparently similar settings. This also makes the comparison with a frozen drafter hard to judge.

  2. The model is trained on greedy continuations, while sampling can produce different prefixes. I am unsure how the claim of no training–inference mismatch applies in this case.

Questions for authors
  1. What explains the differences in the full-model results across tables? What varies across the repeated greedy runs, and were the frozen-drafter comparisons made under the same conditions?

  2. Why does training the drafter and corrector together help more than keeping the drafter frozen?

  3. Does the no-mismatch argument apply only to greedy decoding? How does it extend to sampling?

3 comments
AuthorsOpenReview ↗

Response to Reviewer J3Vj (1/2)

We thank the reviewer for recognizing DCOCO's acceptance–latency trade-off, concurrent-serving results, and training controls. The questions identify a reporting error in the ablation tables, which we have corrected by retraining, and ask about the scope of the training-distribution argument. We address both below, along with the question on co-adaptation.

 

W1.1, W1.2, Q1.1 "Its contribution therefore depends heavily on consistent performance evidence", "What accounts for the full-model differences between Tables 6 and 7…"

We thank the reviewer for identifying this discrepancy. The full DCOCO results in Tables 6 and 7 should have been identical; they were not because of a reporting error on our side. The ablation results (Tables 5–6) were mistakenly taken from a run whose DFlash backbone used the same recipe but a different sample of training prompts drawn from the same datasets. All ablation variants in that run, including Frozen, shared this backbone, so comparisons within those tables are internally consistent but not comparable with Tables 1 and 7. We audited the checkpoints and evaluation configurations behind every other table; the error is limited to Tables 5–6 and the Appendix B claim derived from them (W1.3).

We retrained the DFlash backbone on the prompt set used for all reported methods, then trained DCOCO and every ablation variant on top of it under the paper's protocol. The retrained models closely reproduce the original results, and DCOCO's improvement over DFlash is preserved. Tables R1–R2 replace Tables 5–6; the revision will add variance margins and throughput for all variants. The two-stage training code is provided in the supplementary material.

All three components contribute, and every effect keeps its direction: removing the candidate representations lowers mean accepted length by 8.6% / 11.9% (T=0 / T=1), removing the pair state by 3.5% / 7.4%, and freezing the backbone by 10.2% / 12.4%. We will update Section 5.1 and Appendix B accordingly.

 

Table R1. Ablations, Qwen3-8B, T=0, mean accepted length τ.

DatasetDCOCO−pair state−cand stateFrozenDFlash
MATH-5008.127.857.287.346.09
AIME-257.377.086.686.655.50
GSM8K6.235.925.245.434.59
HumanEval5.475.315.164.864.17
MBPP5.215.065.004.613.88
LiveCodeBench7.046.846.476.255.52
MT-Bench2.952.862.732.752.43
Arena-Hard-v22.782.712.512.502.28
Alpaca2.602.502.522.442.17
SPEED-Bench2.602.492.462.372.15
Average5.044.86 (−3.5%)4.61 (−8.6%)4.52 (−10.2%)3.88

 

Table R2. Same, T=1 (mean over seeds 42/43/44).

DatasetDCOCO−pair state−cand stateFrozenDFlash
MATH-5006.195.665.325.374.72
AIME-255.084.404.124.243.72
GSM8K5.665.344.834.984.24
HumanEval4.824.474.414.253.70
MBPP4.664.334.044.073.47
LiveCodeBench5.705.135.044.844.26
MT-Bench2.692.582.442.472.23
Arena-Hard-v22.382.262.172.121.96
Alpaca2.402.302.272.171.98
SPEED-Bench2.332.312.222.202.05
Average4.193.88 (−7.4%)3.69 (−11.9%)3.67 (−12.4%)3.23

Repeated T=0 runs. The three T=0 runs re-evaluate a single checkpoint with identical weights, data, and decoding settings. Greedy accepted length is deterministic, so the repetitions capture only run-to-run variation in throughput.

 

W1.3 "…Appendix B's cross-table claim of a 4.4% frozen-backbone gain difficult to interpret."

We agree and withdraw it: the 783.1 tok/s (DFlash, Table 1) and 817.7 tok/s (Frozen, on the other backbone) were not matched. In R1–R2, DFlash, Frozen, and DCOCO share one backbone and one protocol. There, the corrector alone improves mean accepted length by 16.6% at T=0 (3.88 → 4.52) and 13.5% at T=1 (3.23 → 3.67), on all ten benchmarks, and joint training roughly doubles the gain, to 29.9% and 29.6%. Both conclusions of Appendix B hold, and we will restate them using this matched comparison.

AuthorsOpenReview ↗

Response to Reviewer J3Vj (2/2)

Q2 "How do you view the corrector's role in that co-adaptation? I would be interested in the design intuition behind this interaction."

We thank the reviewer for this question, which touches a discussion we find particularly interesting. We see two complementary effects. With the backbone frozen, the corrector can already exploit useful alternatives in DFlash's top-K candidates. Joint training additionally lets the correction objective shape the drafter itself, producing representations and candidate sets better suited to contextual correction.

Medusa reports a related effect: Medusa-1 trains future-token heads over a frozen backbone, whereas Medusa-2 trains heads and backbone jointly and obtains better accuracy and larger speedups [1]. More broadly, auxiliary future-token supervision on a shared backbone has been found to improve the backbone itself [2]. We suspect a similar phenomenon here: the correction objective acts as an additional future-looking training signal that encourages the backbone to expose representations and top-K alternatives more useful for predicting subsequent positions. This would explain why correction helps a frozen drafter while joint training yields substantially larger gains. We emphasize that this is a hypothesis based on the observed co-adaptation, not a property our experiments establish directly.

[1] Cai, Tianle et al. "Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads." ICML 2024. [2] Gloeckle, Fabian et al. "Better & Faster Large Language Models via Multi-token Prediction." ICML 2024.

 

W1.4, Q3 "Is the no-mismatch argument intended only for greedy decoding, or is there an additional argument connecting stochastic accepted prefixes to the training distribution?"

Losslessness is unaffected: DCOCO only determines the proposal, and the verification in Appendix G preserves the target distribution for any proposal. The strict no-mismatch argument applies to T=0, where accepted prefixes coincide with the verifier's greedy continuation and hence with the training prefixes. At T=1, accepted non-argmax tokens can lead the corrector outside these trajectories; we will clarify this scope in the revision.

Empirically, the greedy-trained corrector retains the best average T=1 throughput in all six configurations (Table 2). To probe the effect directly, we trained DCOCO for Qwen3-4B on verifier continuations generated at T=1:

DatasetGreedy-trained, T=0Greedy-trained, T=1T=1-trained, T=0T=1-trained, T=1
MATH-5007.9676.0418.0146.077 ± 0.152
AIME-257.3684.6237.3884.698 ± 0.035
GSM8K6.0825.6106.1555.633 ± 0.175
HumanEval5.3514.9495.6495.241 ± 0.031
MBPP5.2974.6035.4144.828 ± 0.046
LiveCodeBench7.1275.1107.2035.697 ± 0.060
MT-Bench2.9462.6693.0202.723 ± 0.004
Arena-Hard-v22.7162.3932.7872.500 ± 0.037
Alpaca2.5152.3572.5782.407 ± 0.014
SPEED-Bench2.7532.4253.4472.881 ± 0.009
Average5.0124.0785.1664.268

Training on stochastic continuations improves average accepted length by 3.1% at T=0 and 4.7% at T=1, across all ten benchmarks. This suggests that the distribution shift is real but modest, and that it can be reduced through the training data without sacrificing greedy performance. We will include this analysis in the revision.

 

We hope these answers address the reviewer's concerns, and we would be glad to discuss further.

 

Sincerely, the authors

Thank you again for your thoughtful comments and constructive feedback. We have addressed the points you raised in our responses and in the revised manuscript. If any concerns remain, we would be glad to clarify them before the discussion period closes.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of discussion and revised manuscript

As the discussion period draws to a close, we thank the reviewers for their constructive engagement. We have uploaded a revised manuscript that incorporates all results and clarifications from the rebuttal, with every change marked in red. Below is a brief summary of the concerns raised and how they were addressed.

 

Main concerns

  1. Ablation discrepancy (J3Vj, Nsr2): fixed. The ablations had been reported from a backbone trained on a different prompt sample. We retrained all variants on one shared backbone (App. B). Every component still helps, and the corrector alone adds +16.6% / +13.5% accepted length over DFlash (T=0 / T=1); joint training roughly doubles this. The cross-table 4.4% claim is withdrawn.

  2. Baseline fidelity (Nsr2, bpay): clarified and extended.

    • All methods share one training recipe, which converges: 12 epochs change accepted length by ≤1.9%. They are served in one vLLM stack with block size 16 and the same verification rule.
    • The recipe trains baselines effectively: DSpark gains +8.7% accepted length over DFlash on Qwen3-8B but loses throughput to correction overhead. This accuracy–cost trade-off is a central finding of the paper.
    • DCOCO leads with and without DSpark's adaptive verification (App. F).
  3. Positioning vs. DiffuSpec (Nsr2): now discussed in related work. DiffuSpec uses a training-free path search, whereas DCOCO jointly trains a parallel transition scorer.

 

Additional results

  • EAGLE-3: DCOCO is 2.1× / 1.8× faster (App. I).
  • Parallel vs. sequential correction: with identical drafts, sequential execution is 2.9× slower at block size 16 (App. G).
  • DFlash 2 selector (concurrent work): DCOCO is +11.7% / +4.4% faster on the same backbone (App. H).
  • Training on T=1 continuations: +3.1% / +4.7% accepted length (App. A).
  • Llama-3.1-8B reproducibility: five reruns stay within 1.5%, and we released anonymized checkpoints.
  • Protocol clarifications: matched draft lengths and the T=1 verification rule are now stated explicitly (Sec. 5, App. C).

 

We hope the revision addresses the reviewers' concerns, and we remain available for any further questions before the discussion period closes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.