- Rating
- 4
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LiLiCorr, a lightweight correlation head for parallel diffusion-style speculative drafters. Rather than independently selecting the highest-probability token at each position, LiLiCorr processes a lattice of top-K candidates with a small bidirectional Transformer. It produces incoming and outgoing vectors whose similarities score adjacent candidate pairs. These scores are computed in parallel, followed by inexpensive sequential token selection. LILIcorrs’ head is jointly trained with the DFlash backbone using a prefix-conditioned cross-entropy objective and a target-weighted distractor penalty. Experiments evaluate Qwen3-4B and Qwen3-8B across all benchmarks.
Critical strengths
LILIcorr proposes an efficient approach to improving draft coherence
Meaningful and consistent serving evaluation.This paper reports acceptance length, absolute throughput, and speedup over autoregressive decoding. Experiments use a common SGLang backend on a single H100 and include DFlash, DFlash 2, Domino, and DSpark. The concurrency sweep provides useful evidence beyond batch-one performance.
Exp shows improved performance on dflash2, dflash etc. sota.
Critical weaknesses
The central architectural contribution could potentially have more direct ablation and discussion. The experiments ablate the distractor penalty, convolution, and decoding rule, but do not directly isolate the benefit of allowing every candidate to attend to the entire lattice. Comparisons without cross-slot attention, without target-state fusion, and with a simpler context-gated pairwise scorer would help establish which components are necessary.
Thanks for comparing against Dflash2. Under greedy decoding, Table 1 reports average speedups of 4.53× versus DFlash 2’s 4.44× on Qwen3-8B, and 4.36× versus 4.32× on Qwen3-4B. These correspond to approximately 2.0% and 0.9% improvements in the reported averages. While it shows consistent improvements on dflash and dspark, It would be good to discuss this if possible.
Questions for authors
n/a
2 comments
Dear Reviewer 73Eq, thank you again for your time.
As the discussion phase is closing soon, we would appreciate to hear if our response has addressed your concerns, and if there is anything else to discuss.
Response to Reviewer 73Eq
We sincerely thank the reviewer for the very positive and encouraging assessment, and for recognising the efficiency of the approach, the value of the serving evaluation and the concurrency sweep. Both suggestions were helpful and have improved the paper. We address them below.
W1: Isolate the architectural contribution (cross-slot attention, target-state fusion, a simpler context-gated pairwise scorer)
We thank the reviewer for this constructive suggestion. We have added a direct comparison with a context-gated pairwise scorer, DFlash2's selector, on a matched backbone and loss (Appendix D, Table 21). All four arms use the DFlash backbone with the same grouped convolutions and training contract. They differ only in the head (DFlash2's selector or ours) and in whether our distractor penalty is used. LiLiCorr+conv has the longer mean acceptance length in all four settings and is faster in three (on 8, 7 and 6 of the 9 benchmarks, with at most one loss). We note that greedy decoding is the more accurate setting for comparing the heads under this training contract. The training responses are generated greedily by the target, and LiLiCorr's oracle-prefix supervision builds on these greedy continuations. At temperature zero, both heads therefore propose for the kind of continuations they are trained on, whereas at temperature one they propose for sampled continuations that differ from them. The only setting without a throughput lead is temperature one with the penalty on both heads. It is a tie (554 against 557 tokens per second), and even there LiLiCorr+conv accepts longer drafts. Under greedy decoding, it is faster both with and without the penalty. For temperature-one serving, a training setup designed for sampling, with target responses generated at temperature one and an oracle prefix adapted to them, would be a better fit. DFlash2's selector scores each pair from the slot's own hidden state, without attending to the other candidates. This comparison therefore measures what cross-slot attention adds, and the gain shows in the longer accepted drafts. Together with the paper's ablations of the distractor penalty, the sublayer convolutions and the decoding rule, it isolates the main components of our head. Removing target-state fusion within our own head would be a valuable finer-grained comparison, and we thank the reviewer for the suggestion. We did not prioritize it because the target state is already available at no extra cost: the DFlash backbone operates by conditioning on the target's hidden states, and our head reuses the hidden states that the backbone already captures. We therefore chose to let the head use this available signal rather than withhold it.
W2: Margins over DFlash2 under greedy decoding are small (4.53× vs 4.44× at 8B, 4.36× vs 4.32× at 4B)
We thank the reviewer for the careful reading of Table 1. We agree this deserves a discussion and have added one to Section 4.2. Table 1 compares base LiLiCorr, which has no convolutions, with DFlash2, whose backbone carries grouped convolutions. The like-for-like counterpart is LiLiCorr+conv (Appendix C). Under greedy decoding at batch one, it is faster than DFlash2 on 7 of the 9 benchmarks at Qwen3-8B and on 5 at Qwen3-4B, and tied on the rest. Its mean speedup is 4.56× against 4.44× and 4.36× against 4.32×, and its mean acceptance length is 5.1% and 4.1% higher. With backbone, data, contract and loss matched so that only the head differs (Appendix D, Table 21), our head has the longer acceptance length in all four settings. It is also faster in three, with a tie at temperature one when both use the penalty. Beyond batch-one greedy decoding, LiLiCorr has the highest throughput at all 36 points of the 8B concurrency sweep. Under temperature-one sampling, it is faster than DFlash2 in 14 of the 18 settings, tied in 2 and slower in 2. We included DFlash2, which appeared as a blog post four weeks before the deadline, because we find the comparison informative, and we appreciate the reviewer's attention to it.