acceptodds
Under review as a conference paper at ICLR 2027

LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding

16 views

Abstract

Speculative decoding accelerates language-model inference by drafting future tokens that the target model verifies in parallel. A diffusion-style drafter, such as DFlash, drafts an entire block in one forward pass. It is trained on the per-position marginals rather than on the joint distribution over the block, so the tokens it emits are individually plausible yet jointly incoherent. We introduce LiLiCorr, a ghtweight kelihood-based model that elates the per-position marginal distributions such a drafter produces. It keeps the top-k tokens at each position as and processes them jointly, producing for each an and an vector. A pair at consecutive positions matches when the earlier candidate's vector aligns, in cosine similarity, with the later one's vector. Training makes the correct pairings score highest while pushing competing ones down, so coherent blocks outscore incoherent ones. The joint distribution over the block, exponential in its length, is never materialized. One lightweight network pass produces all the vectors, and the pairwise scores follow as batched matrix operations, leaving only a cheap greedy walk sequential. We co-train the DFlash drafter with LiLiCorr, so it learns to propose candidates that correlate into longer accepted sequences. Over the vanilla DFlash drafter it builds on, LiLiCorr accepts more and serves faster at every one of the settings we test: nine benchmarks at two target sizes under greedy and temperature-one decoding, together with a throughput sweep over six concurrencies, two input lengths and three output-entropy tiers. It raises acceptance length by to %, while its single-pass scoring head accounts for only about % of the per-block latency. Against three concurrently developed methods that also restore coherence at draft time, every system equally optimized on a common serving stack, LiLiCorr holds the highest throughput in of those settings, ties within a measured noise floor in , and trails in only .

Primary area:
foundation or frontier models, including LLMs
Keywords:
Speculative Decoding, Diffusion Language Models

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_73EqOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LiLiCorr, a lightweight correlation head for parallel diffusion-style speculative drafters. Rather than independently selecting the highest-probability token at each position, LiLiCorr processes a lattice of top-K candidates with a small bidirectional Transformer. It produces incoming and outgoing vectors whose similarities score adjacent candidate pairs. These scores are computed in parallel, followed by inexpensive sequential token selection. LILIcorrs’ head is jointly trained with the DFlash backbone using a prefix-conditioned cross-entropy objective and a target-weighted distractor penalty. Experiments evaluate Qwen3-4B and Qwen3-8B across all benchmarks.

Critical strengths

LILIcorr proposes an efficient approach to improving draft coherence

Meaningful and consistent serving evaluation.This paper reports acceptance length, absolute throughput, and speedup over autoregressive decoding. Experiments use a common SGLang backend on a single H100 and include DFlash, DFlash 2, Domino, and DSpark. The concurrency sweep provides useful evidence beyond batch-one performance.

Exp shows improved performance on dflash2, dflash etc. sota.

Critical weaknesses

The central architectural contribution could potentially have more direct ablation and discussion. The experiments ablate the distractor penalty, convolution, and decoding rule, but do not directly isolate the benefit of allowing every candidate to attend to the entire lattice. Comparisons without cross-slot attention, without target-state fusion, and with a simpler context-gated pairwise scorer would help establish which components are necessary.

Thanks for comparing against Dflash2. Under greedy decoding, Table 1 reports average speedups of 4.53× versus DFlash 2’s 4.44× on Qwen3-8B, and 4.36× versus 4.32× on Qwen3-4B. These correspond to approximately 2.0% and 0.9% improvements in the reported averages. While it shows consistent improvements on dflash and dspark, It would be good to discuss this if possible.

Questions for authors

n/a

2 comments
AuthorsOpenReview ↗

Response to Reviewer 73Eq

We sincerely thank the reviewer for the very positive and encouraging assessment, and for recognising the efficiency of the approach, the value of the serving evaluation and the concurrency sweep. Both suggestions were helpful and have improved the paper. We address them below.

W1: Isolate the architectural contribution (cross-slot attention, target-state fusion, a simpler context-gated pairwise scorer)

We thank the reviewer for this constructive suggestion. We have added a direct comparison with a context-gated pairwise scorer, DFlash2's selector, on a matched backbone and loss (Appendix D, Table 21). All four arms use the DFlash backbone with the same grouped convolutions and training contract. They differ only in the head (DFlash2's selector or ours) and in whether our distractor penalty is used. LiLiCorr+conv has the longer mean acceptance length in all four settings and is faster in three (on 8, 7 and 6 of the 9 benchmarks, with at most one loss). We note that greedy decoding is the more accurate setting for comparing the heads under this training contract. The training responses are generated greedily by the target, and LiLiCorr's oracle-prefix supervision builds on these greedy continuations. At temperature zero, both heads therefore propose for the kind of continuations they are trained on, whereas at temperature one they propose for sampled continuations that differ from them. The only setting without a throughput lead is temperature one with the penalty on both heads. It is a tie (554 against 557 tokens per second), and even there LiLiCorr+conv accepts longer drafts. Under greedy decoding, it is faster both with and without the penalty. For temperature-one serving, a training setup designed for sampling, with target responses generated at temperature one and an oracle prefix adapted to them, would be a better fit. DFlash2's selector scores each pair from the slot's own hidden state, without attending to the other candidates. This comparison therefore measures what cross-slot attention adds, and the gain shows in the longer accepted drafts. Together with the paper's ablations of the distractor penalty, the sublayer convolutions and the decoding rule, it isolates the main components of our head. Removing target-state fusion within our own head would be a valuable finer-grained comparison, and we thank the reviewer for the suggestion. We did not prioritize it because the target state is already available at no extra cost: the DFlash backbone operates by conditioning on the target's hidden states, and our head reuses the hidden states that the backbone already captures. We therefore chose to let the head use this available signal rather than withhold it.

W2: Margins over DFlash2 under greedy decoding are small (4.53× vs 4.44× at 8B, 4.36× vs 4.32× at 4B)

We thank the reviewer for the careful reading of Table 1. We agree this deserves a discussion and have added one to Section 4.2. Table 1 compares base LiLiCorr, which has no convolutions, with DFlash2, whose backbone carries grouped convolutions. The like-for-like counterpart is LiLiCorr+conv (Appendix C). Under greedy decoding at batch one, it is faster than DFlash2 on 7 of the 9 benchmarks at Qwen3-8B and on 5 at Qwen3-4B, and tied on the rest. Its mean speedup is 4.56× against 4.44× and 4.36× against 4.32×, and its mean acceptance length is 5.1% and 4.1% higher. With backbone, data, contract and loss matched so that only the head differs (Appendix D, Table 21), our head has the longer acceptance length in all four settings. It is also faster in three, with a tie at temperature one when both use the penalty. Beyond batch-one greedy decoding, LiLiCorr has the highest throughput at all 36 points of the 8B concurrency sweep. Under temperature-one sampling, it is faster than DFlash2 in 14 of the 18 settings, tied in 2 and slower in 2. We included DFlash2, which appeared as a blog post four weeks before the deadline, because we find the comparison informative, and we appreciate the reviewer's attention to it.

Dear Reviewer 73Eq, thank you again for your time.

As the discussion phase is closing soon, we would appreciate to hear if our response has addressed your concerns, and if there is anything else to discuss.

Reviewer_xaogOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes LiLiCorr, a lightweight reranking model jointly trained with DFlash to improve draft-token coherence while keeping costs low. LiLiCorr retains the top eight candidate tokens at each future position and processes all candidates together using a two-layer bidirectional Transformer. It produces in and out vectors for each candidate and computes correlation scores between candidates at consecutive positions in parallel.

Across the tested benchmarks and settings, LiLiCorr generally achieves higher throughput while maintaining competitive acceptance lengths compared to DFlash2, Domino, and DSpark. Its central contribution is an efficient consecutive draft-token coherence enhancement module for DFlash.

Critical strengths

Clarity of presentation

  • The paper is clearly written and easy to follow. It motivates from the marginals problem. Individual plausible token predictions from a parallel drafter may form an inconsistent sequence. This provides a clear rationale for improving dependencies between draft-token choices while preserving drafting efficiency.

Technical Feasibility

  • The proposed architecture offers an intuitive balance between draft quality and computational cost. Restricting to small top-(K) candidate pools, processing the candidate tokens in a single network pass is the way to save efficiency. The training objective is the way to enhance token coherence by use cross-entropy loss to encourage correct successor candidate to receive higher probability and use distractor panelty to discourage alternatives that the target model consider substantially worse.
Critical weaknesses

W1. The DFlash2 baseline is mis-described, and its reimplementation is unverified.

  • The paper states that DFlash2 scores a pair “from the identities of its two tokens, so the same pair scores alike in every draft” (line 115–116). DFlash2’s primary sources contradict this. The score is S_t(a,b) = U_t(b) + ⟨A(a) ⊙ H(h_t), B(b)⟩, where U_t(b) is the drafter’s own logit and the pair term is gated by H(h_t), a projection of the drafter’s hidden state at slot t. The paper also does not say whether its “third-party reimplementation” (line 686–687) implements this score or the authors’ interpretation of it. If the reimplementation lacks the gate, it may be weaker than the original DFlash2, and the reported DFlash2 results may understate it. If it includes the gate, the paper’s explanation of LiLiCorr’s advantage (line 115–118) is inaccurate.

W2. The architecture’s contribution is not separated from the loss term.

  • DFlash2’s head has lower latency than LiLiCorr’s (0.256 vs. 0.299 ms per block at 8B in Figure 2 and Table 3), so LiLiCorr’s higher speedup in Table 1 must come from longer acceptance. The paper’s loss ablation suggests that much of it comes from the distractor loss. At 8B greedy, LiLiCorr trained without the distractor loss averages an acceptance length of 6.34 in Table 15, only 1.0% above DFlash2’s 6.27 in Table 5. With the distractor loss LiLiCorr reaches 6.44. Removing the loss therefore remove about 60% of the lead. The distractor loss is used only during training, and DFlash2’s official code also scores each slot’s candidates given the predecessor, so the loss could plausibly be applied to DFlash2 at no inference cost. If DFlash2 trained with this loss achieved a longer acceptance length, LiLiCorr’s speedup lead would need to be re-evaluated and could change. This experiment would show whether the lead comes from the proposed architecture or simply from the separate training-loss design.
Questions for authors

Q1. DFlash2 baseline reimplementation

  • As raised in the weaknesses, can the authors release the training code for their DFlash2 reimplementation?

Q2. Candidate-pool size K

  • Appendix A states that K = 8 was chosen after trying K = 4 and K = 16, both of which “served slower than K = 8” (l. 626–629), but it reports no numbers. Can the authors report the results of these runs?

Q3. Coverage of the top-K candidates

  • Can the authors report the per-slot coverage of the current top-8 candidates (Recall@8), that is, how often the target’s token is among the candidates at each slot? If possible, can the authors report the same for K = 4 and K = 16? This would clarify how the size of the candidate pool trades off against acceptance length and throughput.

Q4. Frozen vs. jointly trained drafter

  • The reported results come from training the proposed reranker jointly with DFlash. Since the reranker could in principle be attached to a frozen drafter, can the authors compare three settings: (i) DFlash alone, (ii) a frozen DFlash with the reranker trained on top, and (iii) DFlash and the reranker trained jointly? This would help assess the efficacy of the proposed reranker.
4 comments
AuthorsOpenReview ↗

Response to Reviewer xaog (1/2)

We thank the reviewer for the careful reading and the concrete requests, all of which we have carried out. On DFlash2 itself, we were unsure whether to compare with it. It appeared as a blog post four weeks before the deadline, without training code or a paper, and the ICLR 2027 guidelines do not require the comparison. We included it because we find it informative and because it highlights the structural benefit of our approach. The reviewer's requests have made the comparison more complete.

W1: DFlash2 is mis-described, and the reimplementation is unverified

The reviewer is correct that lines 115 to 116 are inaccurate: DFlash2's pair term is gated by H(ht)H(h_t). We will quote the rule (Section 2) and correct Sections 1 and 2. Our DFlash2 baseline follows the scoring rule and training description published by its authors, including the gate: edge(p→c)=⟨A[p]⊙H(h),B[c]⟩+U(c)\mathrm{edge}(p \to c) = \langle A[p] \odot H(h), B[c] \rangle + U(c). It uses top-16 and rank 256 on the same 16-position verification block as every arm, so the results do not come from a weakened, gate-less variant. The corrected statement of LiLiCorr's advantage is contextual scoring over all candidates at every slot, against a context-gated pairwise score. We now test this directly (W2 and Appendix D, Table 21).

W2: The architecture's contribution is not separated from the loss (distractor penalty applied to DFlash2)

We thank the reviewer for this targeted experiment, which we have run (Appendix D, Table 21). We trained DFlash2 with our distractor penalty on its selector, under the same contract. The penalty also improves DFlash2 (mean τ\tau +1.2% at temperature zero and +3.3% at temperature one), so it is useful beyond our head. The revision states this in Section 3.4. To compare the heads on a matched architecture, we also trained LiLiCorr+conv, which adds DFlash2's grouped convolutions to our backbone, without the penalty. Within each pair, both with or both without the penalty, only the head differs. LiLiCorr+conv has the longer mean acceptance length in all four settings and is faster in three (on 8, 7 and 6 of the 9 benchmarks, with at most one loss). We note that greedy decoding is the more accurate setting for comparing the heads under this training contract. The training responses are generated greedily by the target, and LiLiCorr's oracle-prefix supervision builds on these greedy continuations. At temperature zero, both heads therefore propose for the kind of continuations they are trained on, whereas at temperature one they propose for sampled continuations that differ from them. The only setting without a throughput lead is temperature one with the penalty on both heads. It is a tie (554 against 557 tokens per second), and even there LiLiCorr+conv accepts longer drafts. Under greedy decoding, it is faster both with and without the penalty. For temperature-one serving, a training setup designed for sampling, with target responses generated at temperature one and an oracle prefix adapted to them, would be a better fit. The penalty loss term also helps both LiLiCorr heads (mean τ\tau +1.4% to +2.4% for LiLiCorr and +0.1% to +1.8% for LiLiCorr+conv). The lead over DFlash2 therefore does not come from the additional loss term. It shows the structural benefit of our head. Knowing, through attention, the alternative candidates at every slot lets it represent the correlation between candidates at adjacent slots better than a score conditioned only on the slot's own hidden state.

AuthorsOpenReview ↗

Response to Reviewer xaog (2/2)

Q1: Release the DFlash2 training code

Yes. The DFlash2 implementation we trained is available, together with the training code of every drafter in the paper, in an anonymous repository. Following the ICLR 2027 Author Guide, we share its link with the reviewers and the AC in a separate comment. It follows the scoring rule and training description published by the DFlash2 authors, including the context gate (see W1), with top-16 candidates, rank 256 and the shared 16-position block.

Q2: Report the K=4K = 4 and K=16K = 16 runs

Yes, and we thank the reviewer for this request. At Qwen3-8B under the same training contract, the fully trained K=4K = 4 is slower than K=8K = 8 on all nine benchmarks at both temperatures, while remaining faster than DFlash on all nine. At temperature zero and one, respectively, its mean τ\tau is 3.1% and 3.2% lower than that of K=8K = 8, and its throughput 2.6% and 2.7% lower. On the concurrency sweep of Section 4.3, K=4K = 4 is also slower than K=8K = 8 on average at every concurrency level. The fully trained K=16K = 16 has a higher mean τ\tau than K=8K = 8 (+0.9% at temperature zero, +1.7% at temperature one) and is faster or tied on all nine benchmarks at batch one. On the concurrency sweep of Section 4.3, its τ\tau is higher at all 36 points, but its throughput lead shrinks with concurrency, from +0.9% at c=1c = 1 to −0.7% at c=32c = 32. We therefore keep K=8K = 8 as the overall preferred setting. We report K=4K = 4 and K=16K = 16 in Appendix A (Table 22) and refine the paper's statement about K=16K = 16 accordingly.

Q3: Per-slot Recall@K of the top-K candidates (K=4,8,16K = 4, 8, 16)

Yes (Appendix A, Table 18). We report per-slot Recall@4, @8 and @16 of the drafter backbones of LiLiCorr (trained with K=8), DFlash and DFlash2, without any head, at both target sizes, on the qualitative split of SPEED. The three backbones are within about 0.02 of each other at every slot and pool size, so the pools are nearly identical. The differences in acceptance therefore come from selection within the pool, the component LiLiCorr changes. Coverage decays along the block. With the top-8 pool, the target's token is present at slot 1 about 97% of the time and at slot 15 about 47% (about 35% with top-4 and 58% with top-16). Under teacher forcing, an oracle that picks the target's token whenever it is in the pool would reach an acceptance length of about 8.0 to 8.2 with top-8 and about 9.0 with top-16. Each backbone's own top-1 token reaches 5.0 to 5.3. A wider pool thus raises the ceiling, while the cost of the head grows with K (Q2).

Q4: Frozen versus jointly trained drafter (DFlash alone, frozen DFlash + head, joint)

We add the frozen-drafter setting (Appendix D). Settings (i), DFlash alone, and (iii), joint training, are the paper's DFlash and LiLiCorr rows. Setting (ii) trains the LiLiCorr head on the frozen DFlash drafter, itself trained for six epochs. At 8B and temperature zero, mean τ\tau over the nine benchmarks is 5.59 for DFlash alone, 6.18 and 6.27 after one and six epochs of head training on the frozen drafter, and 6.44 for joint training. With six epochs, the frozen arm is faster than DFlash on all nine benchmarks (by 5.2% to 10.6%) and slower than joint training on eight (by 1.4% to 3.3%), with one tie. Temperature one (three seeds, sampled proposals) gives the same ordering (τ\tau 4.46, 5.11, 5.20 and 5.32). The head alone thus provides most of the gain over DFlash, about 80% of the gain in τ\tau at temperature zero and 86% at temperature one. Joint training adds the rest, which a longer head-only schedule does not recover. The frozen setting also needs a longer schedule: six epochs for the drafter plus one to six for the head, whereas joint training reaches the higher acceptance length within the same six epochs.

Dear Reviewer xaog, thank you again for your time.

As the discussion phase is closing soon, we would appreciate to hear if our response has addressed your concerns, and if there is anything else to discuss.

Thank you to the authors for their detailed clarifications and extensive additional experiments. My main concern has been addressed, and I will raise my score accordingly.

Reviewer_WQHfOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LiLiCorr reranks the draft of a diffusion-style block drafter. It keeps the top-K candidates at each slot, emits an in and an out unit-norm vector per candidate in one pass, scores adjacent pairs by the cosine of the earlier out with the later in, and commits the block by a greedy walk that touches no network weights. It is trained on ground-truth successors with a target-weighted distractor penalty, jointly with the DFlash drafter. On nine benchmarks, two target sizes, greedy and temperature-one decoding, plus a sweep over six concurrencies, two input lengths and three entropy tiers, it reports higher throughput than DFlash in all 72 settings and than three concurrent methods in 63, ties in 6 and trails in 3.

Critical strengths
  1. The lattice construction is concrete enough to check without running anything. At block size 16 and K equal to 8 it fixes 904 pair terms per block, scores them in one forward pass, and leaves only the commit walk sequential. The per-head cost is not asserted but isolated by the identity of block time over acceptance length, and every figure in Table 3 reproduces from Tables 5 and 6 to within 0.011 ms. A reader can redo that arithmetic and it holds, which is what makes the mechanism credible independently of the headline speedups.

  2. Two reported results run against the paper's own interest: the greedy walk beats exact Viterbi path search by 2.6 percent on the same checkpoint (Table 16), and the distractor penalty is ablated with total reranker weight held fixed (Table 15), where it raises acceptance length on all nine benchmarks. Reporting a negative comparison against a stronger search, and running an ablation with the confound controlled, is not routine, and it is the main reason I read the remaining claims as offered in good faith.

Critical weaknesses

Two concerns determine my recommendation. Four further points follow because they bound what the present claims can support.

  1. The differentiation from the closest concurrent method rests on a description of that method that its own source contradicts. Section 2 states that DFlash2 scores a pair from the identities of its two tokens, so that the same two tokens score alike in every context. The scoring rule given in the cited source is S_t(a,b) equal to U_t(b) plus the inner product of A(a) elementwise multiplied by H(h_t) with B(b), where H(h_t) is a context gate computed from the draft hidden state. The score is therefore draft-dependent, and the structural advantage claimed in Section 1 does not follow as stated.

  2. Half of the main table carries a per-block penalty on one baseline that its own block-time identity does not explain. Appendix A defines block time as acceptance length divided by throughput. Computed that way, every arm is flat between greedy and temperature one except DFlash2, which rises by 1.87 to 2.70 ms (20 to 28 percent) on 8B and 1.91 to 2.74 ms (31 to 41 percent) on 4B, while its head is the cheapest in Table 3. At 8B, GSM8K, temperature one, holding DFlash2 at its greedy block time would put it near 690 tokens per second rather than 574, cutting the margin from 25.5 to 4 percent.

  3. The deviation from the target distribution under sampling is asserted for a procedure that is never specified. Section 3.1 says the rejection test preserves the target distribution and Section 4.1 says verification reproduces it however the block was drafted, but the proposal distribution after reranking is not defined: Eq. (2) normalizes a softmax over the K candidates only, Section 3.5 defines an argmax walk, and Table 5 states that the temperature-one panel uses the sampled proposal. How the proposal enters the acceptance test, and how truncation to K is handled when the target puts mass outside it, is left open, and no greedy-equivalence or distributional check is reported.

  4. The headline acceptance gain is given in three ranges, none of which matches the tables. The abstract and Section 1 give 7 to 19 percent, Section 4.2 gives 13 to 19 percent against the DFlash drafter, and Tables 5 and 6 give 12.3 to 25.2 percent over the 36 settings, with six cells above 19 percent. The lower bound of 7 percent appears only in Table 7, the all-argmax panel that the same section excludes from the 72 settings it counts.

  5. The claim that every system is equally optimized is contradicted one sentence later. Appendix B states that each baseline is composed from its own released branch rather than reimplemented, and then that DFlash2 is benchmarked through a third-party reimplementation. In the same paragraph Domino is trained at 512 anchors rather than its released 256, DSpark runs width-matched, and DFlash2 keeps its own selector width against a pool of 8. No per-arm token count, GPU-hour figure or compute estimate is given, so an equal training budget cannot be verified beyond epochs and batch size.

  6. Attribution to co-training, the mechanism named in the abstract, is never tested. No arm freezes the drafter and trains only the head, so the reported gain cannot be separated from what joint tuning of the drafter alone would give, which the three concurrent baselines also perform. K is fixed at 8 and the alternatives at 4 and 16 are described as slower with no numbers, and block size, the in and out allocation, gamma, lambda and sigma receive no sweep.

  7. Two counts do not reproduce. Appendix C reports that the convolution variant serves faster at 26 of the 36 concurrency points; recomputed from Table 14 the split is 25 faster, 2 equal and 9 slower, or 14 wins, 21 ties and 1 loss under the paper's own 1 percent band. Section 4.2 further reports all 44 SPEED cells as faster, but three of them lead by 0.72, 0.79 and 0.98 percent, which the stated tie rule reads as ties.

Revision items I would look for, none of which requires new training: quote DFlash2's scoring rule and restate the difference as attention over the whole lattice against a gated pairwise score rather than as a structural one; give a per-block time decomposition for both decoding regimes and state whether each arm's sampling path was fused and graph-captured; state one acceptance range measured from Tables 5 and 6 and whether it includes the verifier's bonus token; specify the temperature-one sampler, the handling of the K-candidate truncation, and a greedy-equivalence check against the target's own output; add a frozen-drafter arm and K at 4, 8 and 16, report per-arm GPU-hours, and correct the citation details of LLaDA, of the 2026 EAGLE entry and of the DFlash2 blog.

Questions for authors
  1. The cited source for DFlash2 defines its pairwise score with a context gate over the draft hidden state. How should the statement in Section 2, that the same two tokens score alike in every context, be read, and what remains of the structural difference argued in Section 1?

  2. Using block time equal to acceptance length over throughput, DFlash2's block time rises by about 1.9 to 2.7 ms from greedy to temperature one while every other arm stays flat. What accounts for that difference, and was its sampling path fused and graph-captured as the other heads were?

  3. What is the proposal distribution after reranking, and what exactly is the sampled proposal of Table 5? How does it enter the rejection test, and how is the truncation to K candidates handled when the target puts mass outside them?

  4. Does tau include the verifier's bonus token? The DFlash2 source states that its acceptance length includes the verifier's next token, so are the tau values in Tables 5 to 7 on the same footing across arms given the different pool sizes?

  5. What are the token counts and GPU-hours per arm? Is the same training budget asserted in tokens and epochs or in compute, and were the changes to Domino's anchor count and to DFlash2's reimplementation measured separately?

6 comments
AuthorsOpenReview ↗

Response to Reviewer WQHf (1/3)

We thank the reviewer for a careful and checkable review, including re-deriving Table 3 from Tables 5 and 6, and we adopt every revision item the reviewer lists. The two concerns named as decisive, W1 and W2, both concern DFlash2, which we were unsure whether to include in the submitted version. It appeared as a blog post four weeks before the deadline, without training code or a paper, and the ICLR 2027 guidelines do not require the comparison. We included it because we find it informative, and the reviewer's two points have made it more accurate. W1 led to a corrected description and a new matched experiment, and W2 to a serving audit that confirms the observation and corrects the numbers.

W1 / Q1: DFlash2's score is context-gated, so the structural difference does not follow as stated

The reviewer is correct, and we thank them for the precise reading. DFlash2's pair term is gated by H(ht)H(h_t), and our DFlash2 baseline implements this gated rule (top-16, rank 256). We will quote the rule, St(a,b)=Ut(b)+⟨A(a)⊙H(ht),B(b)⟩S_t(a,b) = U_t(b) + \langle A(a) \odot H(h_t), B(b) \rangle, in Section 2 and correct Sections 1 and 2. We restate the difference as the reviewer suggests. In DFlash2, context enters as a per-slot gate on a bilinear form between context-free token embeddings, so the score of a pair at slot t does not depend on the other candidates. In LiLiCorr, every candidate's in- and out-vectors come from attention over all candidates at every slot together with the target states. To test this directly, we compare the two heads with everything else fixed (Appendix D, Table 21). LiLiCorr+conv, which adds DFlash2's grouped convolutions to our backbone, is compared with DFlash2 on the same data and contract. The loss is matched as well, with our distractor penalty on both heads or on neither. LiLiCorr+conv has the longer mean acceptance length in all four settings and is faster in three (on 8, 7 and 6 of the 9 benchmarks, with at most one loss). We note that greedy decoding is the more accurate setting for comparing the heads under this training contract. The training responses are generated greedily by the target, and LiLiCorr's oracle-prefix supervision builds on these greedy continuations. At temperature zero, both heads therefore propose for the kind of continuations they are trained on, whereas at temperature one they propose for sampled continuations that differ from them. The only setting without a throughput lead is temperature one with the penalty on both heads. It is a tie (554 against 557 tokens per second), and even there LiLiCorr+conv accepts longer drafts. Under greedy decoding, it is faster both with and without the penalty. We note that for temperature-one serving, a training setup designed for sampling, with target responses generated at temperature one and an oracle prefix adapted to them, would be a better fit. Since only the head differs within each pair, this supports the difference: attending to the candidates at every slot lets LiLiCorr score adjacent pairs better than a score conditioned only on the slot's own state, which shows directly in the longer accepted drafts.

W2 / Q2: DFlash2's block time rises by 1.9 to 2.7 ms at temperature one. Was its sampling path fused and graph-captured?

We thank the reviewer for this observation, which is correct. The cause was our harness, not DFlash2. In an isolated engine with the same image, checkpoints and settings, DFlash2's sampling is folded into the draft CUDA graph. Its block time is then flat between greedy and temperature-one decoding (9.43 and 9.45 ms per block on 8B GSM8K), as for every other arm. The extra time came from a correctness check on DFlash2's proposal probabilities, which our harness ran at every verification step during timing and which forced a GPU-host synchronisation each time. For our method, the same check ran once, at warm-up. A smaller check of the same kind affected DSpark's sampled arm. We removed both, re-measured the full temperature-one panel for all arms, and replaced the temperature-one rows (Tables 1 and 5 to 12). LiLiCorr is now fastest in 61 of the 72 settings, tied in 6 and slower in 5 (previously 63, 6 and 3). Appendix A (Table 19) gives every arm's per-block time under both regimes, and none changes by more than 0.5 ms.

AuthorsOpenReview ↗

Response to Reviewer WQHf (2/3)

W3 / Q3: The temperature-one proposal after reranking is not specified

We agree that this was missing, and Section 3.5 now specifies it. At temperature TT, LiLiCorr samples each drafted token from the K candidates at its slot. The probabilities come from the pairwise scores with the previously drafted token: qs(c∣cs−1)=softmaxc∈top-K(ψs(cs−1,c)/T)q_s(c \mid c_{s-1}) = \mathrm{softmax}_ {c \in \text{top-}K}\left(\psi_s(c_{s-1}, c)/T\right), with zero mass outside the K candidates. Verification is the standard speculative-sampling rule of Leviathan et al. (2023) and Chen et al. (2023). A drafted token xx is accepted with probability min⁡(1,p(x)/q(x))\min(1, p(x)/q(x)). On rejection, a token is drawn from the normalised residual max⁡(0,p−q)\max(0, p - q) over the full vocabulary, which recovers the target's mass outside the K candidates, so the output follows the target distribution exactly. We also checked the served outputs. At temperature zero, LiLiCorr's responses match the target's own greedy output as often as those of every other arm (97.3% at 8B and 97.4% at 4B, within 0.2 points). At temperature one, its samples differ from autoregressive samples no more than sampling noise, as for DFlash.

W4 / Q4: The headline acceptance range does not match the tables. Does τ\tau include the bonus token?

Thank you for catching this. The ranges came from different subsets. The 13 to 19% of Section 4.2 covered only the greedy rows of Tables 5 and 6. The 7% lower bound in the abstract and Section 1 came from the all-argmax panel of Table 7, which lies outside the 72 settings. In the abstract, Section 1 and Section 4.2, we will state one range over all 36 settings of Tables 5 and 6, using the re-measured temperature-one rows (W2). LiLiCorr's acceptance length is 12.7% to 25.2% higher than DFlash's, and its throughput 6.3% to 19.7% higher (on average 10.1% at temperature zero and 14.2% at temperature one). τ\tau includes the verifier's bonus token for every arm (Section 4.1). It is the mean number of tokens emitted per verification step, computed by one harness for all methods. Every method verifies the same 16 rows regardless of pool size, consistent with DFlash2's convention.

W5 / Q5: “Equally optimized” versus the third-party DFlash2 reimplementation, Domino anchors, DSpark width, DFlash2 selector width, and per-arm compute

Thank you. “Equally optimized” originally referred to serving. Every method is served on the same SGLang stack, image and hardware, with 15 drafted and 16 verified positions. Every head is also brought to the same level of serving optimization: it is folded into the drafter and uses fused kernels and graph capture. We see that the phrase reads as a claim about training, which conflicts with the second sentence the reviewer quotes. We will therefore replace it with a precise statement of what is matched in serving and in training (Appendix A). In training, every arm uses the same data, six epochs and optimizer steps (130,968 at 8B, 130,956 at 4B), global batch 64, fp32 draft weights and 64 H100. All arms see the same 1,397,048 training conversations at 8B (1,396,889 at 4B) for six epochs. DFlash, Domino and DSpark are each composed from their released training branch. DFlash2's official selector training was not released. We therefore trained DFlash2 with an implementation that follows the scoring rule and training description published by its authors, including the context gate and its own selector configuration (top-16, rank 256). We served it with SGLang's DFlash2 implementation. No DFlash2 drafter has been released for Qwen3-8B or Qwen3-4B, so there is no official checkpoint on these targets to compare against. The anchor count is a contract parameter, 512 for every arm including ours. Domino's recipe uses 256. We did not train Domino at 256, because it would then receive half the supervised anchors, and so half the training signal, of every other arm. Width-matched DSpark drafts the same 15 positions. Appendix A (Table 20) gives the training compute of each arm. At 8B, in GPU-hours, it is 1,240 for DFlash, 1,289 for Domino, 1,449 for DFlash2, 1,479 for LiLiCorr, 1,674 for DSpark and 1,965 for LiLiCorr+conv. At 4B it ranges from 910 to 1,283. The DFlash2 implementation we trained is available, together with the training code of every drafter in the paper, in an anonymous repository. Following the ICLR 2027 Author Guide, we share its link with the reviewers and the AC in a separate comment.

AuthorsOpenReview ↗

Response to Reviewer WQHf (3/3)

W6: Co-training is never tested, and K, block size, in/out allocation, γ\gamma, λ\lambda and σ\sigma receive no sweep

We add the requested frozen-drafter arm (Appendix D). It trains the LiLiCorr head on the frozen DFlash drafter, itself trained for six epochs, and is reported beside DFlash alone and joint training. At 8B and temperature zero, mean τ\tau over the nine benchmarks rises from 5.59 (DFlash) to 6.18 after one epoch and 6.27 after six epochs of head training, against 6.44 for joint training. The frozen arm is faster than DFlash on all nine benchmarks and slower than joint training on eight, with one tie. Temperature one gives the same ordering (τ\tau 4.46, 5.11, 5.20 and 5.32). A longer head-only schedule does not recover the gain of joint training, and joint training needs fewer epochs (six epochs, against six for the drafter plus one to six for the head).

On K, we trained pool sizes 4 and 16 under the same contract. K=4K = 4 is slower than K=8K = 8 on all nine benchmarks at both temperatures and, on average, at every concurrency level of the Section 4.3 sweep. K=16K = 16 accepts longer drafts and is faster or tied on all nine benchmarks at batch one. On the concurrency sweep of Section 4.3, however, its throughput lead shrinks with concurrency, from +0.9% at c=1c = 1 to −0.7% at c=32c = 32. We therefore keep K=8K = 8 as the overall preferred setting and report K=4K = 4 and K=16K = 16 in Appendix A (Table 22). Block size is fixed by the serving contract shared by all arms. The remaining hyperparameters, γ\gamma, λ\lambda, σ\sigma and the allocation of the head's output dimension between the in- and out-vectors, were set in preliminary experiments and held fixed for every run. The distractor term is isolated at fixed total head weight (Table 15). Re-measured over all nine benchmarks, both sizes and both temperatures, it raises mean τ\tau by 1.4% to 2.4% for LiLiCorr and by 0.1% to 1.8% for LiLiCorr+conv.

W7: Two counts do not reproduce (conv variant 26 of 36, SPEED 44 of 44)

Thank you. Both counts are corrected under the paper's 1% tie rule (Section 4.2 and Appendices B and C). At 8B, the convolution variant is faster than the base at 14 of the 36 concurrency points, tied at 21 and slower at 1, with a higher acceptance length at all 36. Of the 44 SPEED category cells, with the re-measured temperature-one values, LiLiCorr is faster than every system compared in 39, tied in 4 and slower in 1. The one loss is the math category at Qwen3-8B and temperature one, 2.2% behind DSpark's sampled proposal.

Citations

We will correct the entries for LLaDA, the 2026 EAGLE paper and the DFlash2 blog post.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to all reviewers

We thank the reviewers for their careful reading. The reviewers agree that the paper is clear and that the mechanism is concrete and checkable (R1 reproduced the head-cost decomposition of Table 3 from Tables 5 and 6). R3 also finds the serving evaluation informative beyond batch-one numbers. We address every point in the individual responses and summarise the main changes here.

On DFlash2. We deliberated about whether to compare with DFlash2. It appeared on August 18, 2026, four weeks before the submission deadline, as a technical blog post. It came without released training code, an arXiv version or a peer-reviewed paper, and the ICLR 2027 guidelines do not require a comparison with such work. We chose to include it because we find the comparison informative. It sets LiLiCorr against the strongest concurrent alternative and highlights the structural benefit of our approach. LiLiCorr scores candidate pairs from representations conditioned on all candidates at every slot, rather than through a context-gated pairwise score. We address every comment about it below.

The main additions are a refined description of DFlash2 and a matched comparison of our head with DFlash2's selector, a re-measured temperature-one panel, a frozen-drafter arm, candidate pool sizes K=4K = 4 and K=16K = 16, per-slot coverage, and a precise specification of the temperature-one sampler. Each is described in the reply to the reviewer who asked for it. We have uploaded a revised paper, in which all changes are marked in red.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.