- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents D-PAL, a training objective for speculative decoding that extends D-PACE by replacing point-mass confidence estimates with full-distribution overlap to compute acceptance-aware position weights. The proposed method derives dynamic position weights from the expected accepted length and explores several local loss functions, including log acceptance, forward KL divergence, and Rényi-1/2 divergence. Extensive experiments across six benchmarks, multiple target models, and decoding configurations demonstrate improvements in accepted sequence length and inference throughput over D-PACE and DFlash.
Critical strengths
- Principled theoretical analysis. The paper derives position weights from the expected accepted length and establishes theoretical bounds relating Rényi-1/2 divergence to acceptance probability, providing insights into its optimization advantages.
- Comprehensive empirical evaluation. The experiments span multiple benchmarks, target models, decoding temperatures, draft depths, and block sizes, with ablation studies examining the contributions of position weighting and local loss functions.
Critical weaknesses
- Insufficient evidence for the benefit of overlap-based weighting. Table 3 suggests that replacing cross-entropy with Rényi divergence accounts for most of the improvements, while overlap-based and confidence-based weighting achieve nearly identical results under matched smoothing settings.
- Mismatch between theoretical analysis and practical verification. The overlap objective characterizes stochastic rejection sampling, whereas the evaluated DFlash implementation employs deterministic argmax proposals.
- Modest empirical gains and limited robustness evidence. D-PAL achieves marginal improvements in mean accepted length over D-PACE (4.50 to 4.56 at T=0 and 4.20 to 4.28 at T=1), corresponding to relative gains below 2%. Furthermore, the use of a single training seed leaves the robustness of these improvements unclear.
Questions for authors
- Does overlap-based position weighting offer a meaningful advantage over confidence-based weighting when the local loss, smoothing parameter, and training configuration are held fixed?
- To what extent does distributional overlap predict actual token acceptance under the deterministic argmax proposals used during deployment, particularly across different decoding temperatures?
- Are D-PAL's improvements over D-PACE significant across multiple independent training seeds?
3 comments
Response for the Rebuttal
Thank you for the additional experiments and clarifications. That said, my core concern regarding the isolated contribution of overlap-based weighting remains unresolved. Under matched settings, the new results confirm that overlap-based weighting yields no statistically significant gain over confidence-based weighting; instead, the observed improvements stem largely from replacing CE with Rényi divergence. Likewise, comparing against fixed position decay does not demonstrate the value of the proposed weighting scheme, since D-PACE already uses dynamic weights.
Overall, although the expanded experiments bolster the empirical evidence, they still fall short of establishing the necessity or efficacy of the paper's central methodological contribution. I will therefore maintain my original rating.
Response to Reviewer KkLn's Follow-up
Thank you for the follow-up. We agree with part of your assessment and clarify our view of the contribution below.
Where we agree. Under matched settings, overlap weights give no statistically significant gain over confidence weights. The comparison with fixed decay shows the value of dynamic weights—which D-PACE introduced—rather than of our specific weight signal. The camera-ready version will state both points clearly.
What we view as the contribution. Our contribution is moving block-drafter training beyond the point-mass assumption to full distributions. D-PACE treats the target as a point mass in both its weights and its CE loss, whereas modern drafters with sampled proposals (such as the current official DFlash implementation and DSpark in SGLang, which verify via rejection sampling at ) accept tokens based on the overlap of the two full distributions. Our derivation connects this exact acceptance rate to D-PACE's dynamic weights—showing D-PACE as the point-mass special case—and enables training smooth distributional losses (such as Rényi-1/2) under the same detached, floored weights. The gain from replacing CE with a full-distribution loss under dynamic weights is therefore the contribution we claim, not a side effect. In the camera-ready version, we will revise the abstract and contributions to frame the paper around this unified formulation rather than presenting the weight signal alone as the headline.
Why the gap over D-PACE is modest. Because D-PAL reduces to D-PACE for a point-mass target, the more concentrated the target distribution is, the smaller the room between a full-distribution objective and point-mass CE. Within that room, the gain is consistent and largest where the derivation directly applies: at the same floor (), D-PAL[Rényi] outperforms D-PACE on 3/3 seeds across all three conditions (+0.079 at , +0.060 with argmax proposals at , and +0.093 with sampled proposals at ).
Experiments completed. As the discussion period nears its close, we note that the experiments your review asked for are complete and reported above: three independent training seeds with paired tests (Question 3); the matched control holding the loss, floor, and training configuration fixed while varying only the weight signal (Question 1); and evaluation under both argmax proposals and sampled proposals (Question 2), along with D-PACE at the matched floor . Thank you for highlighting these points—we would be grateful if you could take these results into account in your final assessment.
Official Comment by Authors: Response to Reviewer KkLn
We thank the reviewer for taking the time to review our paper. We will respond to their questions and concerns below. G1, G2, and G3 refer to the general response above. Weaknesses 1, 2, and 3 correspond to Questions 1, 2, and 3 below.
We ran additional experiments across three training seeds. At a fixed floor (ρ=0.3) and loss (Rényi), replacing confidence weights with overlap weights gives a small, statistically insignificant difference (+0.002 to +0.020 in τ), whereas replacing CE with the Rényi loss raises τ on 3/3 seeds in every condition (p=0.030, 0.085, 0.012):
Testing pt(argmaxqt) as a third weight signal in a one-seed run also makes no measurable difference (G3). We use overlap because it is the exact acceptance rate for sampled proposals (G3); its derivation also motivates the full-distribution losses, where the measured gain comes from. With a fixed decay instead of dynamic weights, the same losses are 0.25 to 0.35 lower in τ (36K training examples, one training seed, ρ=0.5; reply to LtTq). We will make this split explicit in the abstract and contributions.
Overlap is the exact per-token acceptance rate with sampled proposals and lower-bounds it with argmax proposals (Proposition 1); both proposal types are deployed. SGLang 0.5.19 proposes the argmax for original DFlash drafters; the current official DFlash implementation and DSpark in SGLang use sampled proposals at T>0 (G3). We evaluated both:
Yes: D-PAL[Rényi] is above D-PACE on all three seeds (42, 43, 44), significant in all three sampler setups we evaluated (paired t-test from unrounded seed means):
The values cited in the review (4.50 to 4.56, 4.20 to 4.28) are from the single training seed in the submission. Across three seeds, the relative gain in τ is 1.6% at T=0, 1.8% at T=1 with argmax proposals, and 3.7% with sampled proposals, with inference unchanged. D-PAL[LA] and D-PAL[KL] are also above D-PACE on every seed in every condition (G1).