LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding

Speculative decoding accelerates language-model inference by drafting future tokens that the target model verifies in parallel. A diffusion-style drafter, such as DFlash, drafts an entire block in one forward pass. It is trained on the per-position marginals rather than on the joint distribution over the block, so the tokens it emits are individually plausible yet jointly incoherent. We introduce LiLiCorr, a $\text{Li}$ghtweight $\text{Li}$kelihood-based model that $\text{Corr}$elates the per-position marginal distributions such a drafter produces. It keeps the top-k tokens at each position as $candidates$ and processes them jointly, producing for each an $in$ and an $out$ vector. A pair at consecutive positions matches when the earlier candidate's $out$ vector aligns, in cosine similarity, with the later one's $in$ vector. Training makes the correct pairings score highest while pushing competing ones down, so coherent blocks outscore incoherent ones. The joint distribution over the block, exponential in its length, is never materialized. One lightweight network pass produces all the vectors, and the pairwise scores follow as batched matrix operations, leaving only a cheap greedy walk sequential. We co-train the DFlash drafter with LiLiCorr, so it learns to propose candidates that correlate into longer accepted sequences. Over the vanilla DFlash drafter it builds on, LiLiCorr accepts more and serves faster at every one of the $72$ settings we test: nine benchmarks at two target sizes under greedy and temperature-one decoding, together with a throughput sweep over six concurrencies, two input lengths and three output-entropy tiers. It raises acceptance length by $7$ to $19$%, while its single-pass scoring head accounts for only about $3$% of the per-block latency. Against three concurrently developed methods that also restore coherence at draft time, every system equally optimized on a common serving stack, LiLiCorr holds the highest throughput in $63$ of those settings, ties within a measured noise floor in $6$, and trails in only $3$.