Beyond the Point Mass: Acceptance-Derived Weights and Distributional Losses for Speculative Decoding

In speculative decoding, a lightweight draft model proposes tokens that the target model verifies in parallel, speeding up generation without changing the target distribution. D-PACE trains such drafters with a loss that weights each position by its contribution to the expected accepted length. Both its weights and its cross-entropy (CE) loss use only the draft's probability of the recorded target token, treating the target as a point mass. Under rejection sampling, however, a sampled draft token is accepted with probability equal to the overlap of the draft and target distributions, one minus their total variation (TV). We derive D-PAL (Dynamic Position-wise Acceptance Loss) from the exact expected accepted length, using this overlap as the acceptance signal. This yields overlap-based position weights with log acceptance (LA) as the per-position loss, and the objective reduces to D-PACE for a point-mass target. Since the weights are detached, any per-position loss can replace LA in practice. We show that forward Kullback–Leibler (KL) divergence and Rényi divergence of order $1/2$ match or exceed LA, and that Rényi-$1/2$ provably bounds the true acceptance rate from above and below. Replacing CE with Rényi-$1/2$ improves acceptance length under both D-PACE's weights and ours. On six benchmarks with a Qwen3-4B target, D-PAL raises mean acceptance length over D-PACE from 4.50 to 4.57 at $T=0$ and from 4.20 to 4.28 at $T=1$, and consistently achieves higher throughput at both temperatures.