acceptodds
Under review as a conference paper at ICLR 2027

DSpark-OPD: Acceptance-Aligned On-Policy Distillation for Semi-Autoregressive Speculative Drafting

Abstract

Standard rejection-sampling speculative decoding preserves the target distribution for any proposal model, but its speed depends on the length of the draft prefix that survives verification. Semi-autoregressive drafters face two training mismatches: offline training observes target-generated prefixes rather than draft-induced states, and tokenwise objectives do not reflect that an early rejection invalidates the remaining suffix. We present DSpark-OPD, an on-policy post-training method for Markov DSpark. During rollout, the frozen target verifies samples from the deployed Markov-corrected proposal. During replay, target and drafter distributions are evaluated on the same sampled predecessor chain, and optimization is applied to the corrected proposal used by the verifier. We introduce Survival Loss, a differentiable pathwise surrogate based on target–draft overlap that assigns earlier transitions credit for later positions that remain reachable. Across Qwen3-8B and Qwen3-14B, adding Survival Loss to the matched DSpark-OPD-Base objective improves every evaluated dataset–temperature comparison. Under greedy decoding, public-average engine acceptance length rises by 4.7% at 8B and 5.4% at 14B relative to DSpark-OPD-Base; relative to the released DSpark checkpoints, the complete pipeline gains are 12.8% and 6.0%. At concurrency 8, public-benchmark throughput improves by 4.9% and 6.1% over DSpark-OPD-Base, reaching 2.07× and 2.40× target-only throughput. Controlled Qwen3-8B ablations show that Survival Loss remains competitive with local and fixed-decay variants at block length 7 while providing state-adaptive prefix credit without a hand-designed decay schedule.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Speculative Decoding, Semi-Autoregressive Drafting, On-Policy Distillation, Prefix Survival, LLM Inference Acceleration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_w4hbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DSpark-OPD, an on-policy post-training method for the existing Markov-corrected DSpark speculative drafter. Target-assisted rollouts collect draft-generated predecessor chains, and replay evaluates the target and the corrected drafter distribution on those same chains. The method combines acceptance-aware forward/reverse KL distillation with Survival Loss, which aggregates products of target–draft overlap over verification-reachable prefixes.

Evaluation uses Qwen3-8B and Qwen3-14B, block length seven, seven public benchmarks, and a private Business dataset. Relative to the matched OPD-Base model, adding Survival Loss improves reported public-average greedy acceptance length by 4.7% and 5.4%, respectively, and throughput at concurrency eight by 4.9% and 6.1%.

Critical strengths

The main experiments report positive acceptance-length improvements over OPD-Base for every evaluated dataset–temperature pair at both model scales. Importantly, the evaluation includes end-to-end throughput rather than relying only on acceptance length as a proxy.

The results make the approach potentially useful in practice, although the incremental gains should be distinguished from the larger speedups attributable to the complete speculative-decoding system.

Critical weaknesses

Table 2(a) shows that local overlap and fixed-decay overlap slightly outperform Survival Loss in all three public-average columns and all three Business columns at the common coefficient. For example, under greedy decoding, public-average acceptance length is 5.158 for local overlap, 5.160 for fixed decay, and 5.137 for Survival Loss; the corresponding Business values are 4.393, 4.412, and 4.292. The paper appropriately describes these variants as competitive, but this evidence supports the usefulness of overlap augmentation more directly than the benefit of its particular prefix-credit construction. Avoiding a positional-decay hyperparameter also does not distinguish Survival Loss from uniform local overlap.

More broadly, I could not find the serving hardware/backend configuration, training-set sizes, essential optimization and rollout hyperparameters, or repeated-run uncertainty estimates in the submitted manuscript. Training relies on private Business partitions, making a sufficiently detailed reproducible recipe particularly important. Without these details, it is difficult to assess the robustness and reproducibility of the relatively modest incremental public-benchmark gains.

Questions for authors
  1. Could you reconcile Figure 2(b) with Table 2 and the definition τ+=1+∑kPr⁡(r≥k)\tau_+=1+\sum_k\Pr(r\ge k)? Do the figure and table use different checkpoints, weighting conventions, scheduling settings, or evaluation populations? Please also clarify whether “prefix survival at position k” means accepting through position k or merely reaching its verification decision.

  2. Given that L1 and L2 outperform L3 in all aggregate columns of Table 2(a), what evidence distinguishes the benefit of adaptive prefix credit from that of adding dense overlap supervision? Are repeated-run comparisons available? Otherwise, would you narrow the interpretation to overlap augmentation being effective, with Survival Loss one competitive formulation?

  3. Please provide the hardware and inference backend, training-data quantities, key optimization/rollout settings including rollout temperature and λc\lambda_c and available run-to-run variability. Please also confirm that OPD-Base and OPD+Survival share the same training budget, confidence-head supervision, and inference scheduling configuration.

1 comment
AuthorsOpenReview ↗

Figure 2(b) reach metric, separate evaluation runs, and shared settings

Thank you for the thoughtful review and for taking the time to compare Figure 2(b) with Table 2. We appreciate your emphasis on reproducibility and on distinguishing the incremental throughput gains from the full-system speedups.

Figure 2(b) and Table 2. Each bar in Figure 2(b) measures reaching the verification decision at position kk. Position kk must be offered for verification and the first k−1k-1 draft tokens must be accepted. This includes rounds in which the verifier rejects position kk. The tail-sum identity for Eq. (2) uses acceptance through kk, Pr⁡(r≥k)\Pr(r\ge k), over the same verification rounds and with the same averaging weights as τ+\tau_+. Figure 2(b) and Table 2(a) use the same L0 and L3 checkpoints in separate evaluation runs. Section 5.1 and the figure caption now specify both the metric and the evaluation provenance.

Objective comparison. Overlap augmentation improves acceptance length under all three weighting rules in Table 2(a). L1 and L2 lead the fixed-coefficient aggregate comparison. Survival Loss is a competitive formulation that derives its prefix weights from the current target–draft agreement, as quantified by Eq. (16). Sections 5.3 and 6 now state the common improvement and the gradient property separately. Table 3 reports incremental throughput gains of 4.9% and 6.1% over OPD-Base and full-system speedups of 2.07× and 2.40× against target-only decoding.

Training and serving protocol. The controlled 8B variants share the initial checkpoint, rollout protocol, optimization settings, and inference scheduling configuration. Section 4.3 and Appendix C now specify that L0 and L3 share the base OPD and confidence-head objectives, with λs=0\lambda_s=0 for L0 and 11 for L3 at the common comparison point. Confidence training uses stop-gradient overlap targets and updates only the confidence head. Across both model scales, the target is frozen, thinking is disabled, the draft cap is seven, and rollout offers the full block for verification with confidence-based truncation disabled. The rejected-stream KL uses γ=0.9\gamma=0.9 and βrej=2\beta_{\mathrm{rej}}=2. SFT and OPD use disjoint Business training partitions, with a separate held-out test set. The reported acceptance values are evaluation-run point estimates for the listed checkpoints.

Reviewer_XYjEOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper post-trains a Markov-corrected DSpark drafter with OPD in the style of Draft-OPD. It introduces the replay of the deployed proposal and Survival Loss. The full pipeline improves acceptance length by about 5% over the matched OPD-Base objective and improves end-to-end throughput.

Critical strengths
  • Correct/practical alignment point - I think the problem is a good problem and the authors are targeting the correct mechanism / area to align.
  • Consistent gains with real system measurements - the auxiliary lost consistently improves performance in Table 1, using metrics that make sense.
  • Honest ablations.
Critical weaknesses
  • The ablations do not support Survival Loss: In Table 2(a), both uniform overlap (L1) and fixed-decay overlap (L2) outperform the proposed Survival Loss (L3) across all aggregate comparisons. Therefore, the wins might just come from overlap supervision. This is the biggest weakness for me.
  • Incomplete ablation comparisons: Table 2(b) shows that increasing the Survival Loss coefficient substantially improves performance, but equivalent coefficient sweeps are missing for L1 and L2. Given the above, it is difficult to determine whether Survival Loss provides any advantage over these simpler alternatives.
  • Not enough isolation of the method’s components: The paper builds on Draft-OPD and DSpark, but no ablation directly compares Markov-corrected replay with teacher-forced replay or backbone-logit optimization.
  • Statistical reliability and reproducibility: no reporting multiple seeds, confidence intervals, or sufficient training and serving details, despite relatively small differences between the overlap objectives. The discrepancies between corresponding configurations in Tables 1 and 2 also need explanation.
  • Evaluation of credit-assignment: All experiments use block length 7, and Figure 2 compares position-wise survival only between L0 and L3. Evaluating longer blocks and including L1 in the position-wise analysis would better establish if there’s a real win there.
  • The headline wins are incremental: Although the reported gains are consistnet across table 1, adding Survival Loss improves throughput by only 4.9% and 6.1% over OPD-Base at concurrency 8. Given that simpler overlap objectives perform equally well or better in the reported ablations, these relatively small gains do not justify the additional complexity to me.
Questions for authors
  • Do L1 and L2 still outperform Survival Loss when all three objectives receive comparable coefficient tuning and are evaluated across multiple seeds?
  • Can the authors provide an ablation isolating the benefit of Markov-corrected same-prefix replay from the existing on-policy distillation framework?
  • How does Survival Loss compare with simpler overlap objectives at longer draft lengths, where prefix-aware credit assignment should potentially be more beneficial?
2 comments
AuthorsOpenReview ↗

Block-7 loss comparisons, Markov replay, and serving gains

Thank you for the detailed feedback and for highlighting the practical motivation and end-to-end measurements. Your questions about matched comparisons and evaluation settings helped us clarify what each experiment measures.

Coefficient comparison and repeatability. Table 2(a) compares the three objectives at the shared coefficient of 1. L1 and L2 lead L3 in the six aggregate columns, and all three improve public acceptance length by 4.5–6.0% and Business acceptance length by 9.0–13.4% over L0. Table 2(b) measures L3's coefficient sensitivity with the training and evaluation protocol held fixed. Its greedy public average rises from 5.137 at coefficient 1 to 5.209 at 4 and 5.240 at 16. Section 5.3 now reports the common-coefficient comparison and labels the sweep as a within-L3 result. Section 5.1 identifies the acceptance results as checkpoint-level point estimates.

Markov replay. We build on Draft-OPD's target-assisted rollout and acceptance-aware KL. The DSpark adaptation preserves both the predecessor tokens and the corrected proposal distribution used during verification. A teacher-forced predecessor changes the input to the Markov correction in Eq. (3). A backbone-logit objective scores the proposal before that correction. Table 2 uses the same replay rule across L0–L3. Each variant replays its own recorded predecessor chain and scores the corrected proposal. This isolates the auxiliary objective on top of deployed-policy replay. The direct control for the correction would swap only the logits in the distillation loss on identical recorded prefixes. The direct control for the predecessor would hold the correction head fixed and swap the replay prefix. Draft-OPD, §4.2

Per-position and length comparisons. We evaluate both model scales at a maximum draft length of seven. In the separate position-wise evaluation, L3 improves on L0's public reach rate at position 7 from 29.1% to 33.6% and its Business rate from 11.4% to 18.7%. Section 5.4 now identifies Figure 2(b) as a comparison against the shared replay baseline L0, and Table 2(a) provides the aggregate comparison among L1/L2/L3. Section 6 explains how Eq. (16) applies to larger blocks, where earlier positions can accumulate credit from more retained terms.

Serving gains and table provenance. At concurrency 8, Table 3 measures the increment from OPD-Base to OPD+Survival, from 2,100 to 2,203 tokens/s at 8B (+4.9%) and from 1,473 to 1,563 at 14B (+6.1%). Prefix products are computed during training, so the serving architecture and verification algorithm are those of DSpark. For the corresponding 8B rows, Table 1's OPD-Base and OPD+Survival use the same checkpoints as Table 2's L0 and L3. We ran the main comparison and the controlled ablation evaluations separately. Each table reports measurements from its own evaluation, and improvements are computed within each table. Section 5.1 now records the checkpoint correspondence and the separate evaluation runs.

Thank you for the rebuttal and clarifications. I appreciate the authors' efforts to address my concerns. However, the main empirical issues remain unresolved.

"L1 and L2 lead L3 in the six aggregate columns"

This remains my central concern. The results support the value of overlap supervision, but not the specific advantage of Survival Loss. The coefficient sweep for L3 does not resolve this without comparable tuning for L1 and L2, which the authors did not provide.

"The direct control for the correction would swap only the logits in the distillation loss on identical recorded prefixes."

I agree this is an appropriate ablation, but it has not been performed. The benefit of Markov-corrected replay over alternative replay/optimization strategies therefore remains unquantified.

"Section 6 explains how Eq. (16) applies to larger blocks"

While theory is nice, this needs empirical evidence. Figure 2 compares L3 only against L0, and the experiments remain limited to block length 7. Neither establishes whether prefix-aware credit assignment improves upon simpler overlap objectives.

"We ran the main comparison and the controlled ablation evaluations separately."

This does not address statistical reliability. Multiple seeds or uncertainty estimates are particularly important given the small differences among the objectives. Key training and serving details also remain missing.

I acknowledge that Survival Loss adds no inference-time overhead and that the reported throughput gains over OPD-Base are encouraging. Nevertheless, without comparisons against L1/L2 under matched tuning and serving conditions, these improvements cannot be attributed specifically to prefix-aware credit assignment.

The rebuttal improves clarity but does not resolve my central concerns. My score remains unchanged.

Reviewer_EJebOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DSpark-OPD, an on-policy distillation framework for semi-autoregressive speculative drafting. The method adapts the draft model using on-policy trajectories and introduces a Survival Loss to align training with expected accepted length through prefix-level credit assignment. Experiments on Qwen3 models demonstrate improvements in acceptance length and decoding throughput over existing baselines.

Critical strengths
  1. Proposes an on-policy distillation framework for semi-autoregressive speculative drafting.

  2. Introduces a Survival Loss to optimize accepted length through adaptive prefix-level credit assignment.

Critical weaknesses

The proposed Survival Loss consistently underperforms the simpler Local Overlap objective across all three temperatures in Table 2(a). Since Local Overlap also requires no hand-designed decay hyperparameter, the experiments fail to demonstrate a clear advantage of the proposed prefix-aware credit assignment.

Questions for authors

Could the authors expand the related work on acceptance-oriented training objectives for speculative decoding (e.g., https://arxiv.org/abs/2602.23881) and clarify the advantages of the proposed Survival Loss over existing loss functions?

2 comments
AuthorsOpenReview ↗

Survival Loss prefix credit, Local Overlap results, and LK Losses

Thank you for your careful reading and for recognizing the on-policy distillation framework for semi-autoregressive drafting. We appreciate your comparison with Local Overlap and the pointer to LK Losses.

Survival Loss gives an early draft position credit for the later prefixes it helps preserve. In Eq. (16), each retained prefix contributes a gradient to every position it contains, weighted by the other overlaps on that prefix. This yields state-dependent credit from the same target and draft distributions used by the verifier. In Eq. (14), the expected sum of the retained prefix scores lower-bounds expected accepted length. The loss is used only during post-training.

Table 2(a) shows that both objectives improve on the base loss at maximum draft length seven. At coefficient 1, Local Overlap has the higher public-average score at each temperature (5.158/4.856/4.450 versus 5.137/4.836/4.431) and leads on Business as well. Both formulations avoid a positional-decay hyperparameter. The revised abstract and ablation discussion report the common improvement and describe the prefix-level training signal provided by Survival Loss.

Section 2 now discusses DistillSpec's on-policy TV analysis and LK Losses' adaptive KL–TV blend and negative log overlap. In our setting, Eq. (12) gives reached Local Overlap an exact on-policy value interpretation, and its fixed-batch gradient gives each reached overlap the same direct coefficient. The new paragraph explains how Survival Loss uses these local overlaps in its prefix-dependent gradient. DistillSpec · Samarin et al., LK Losses

Thank you for the clarification. However, my main concern remains unresolved, as the authors have not demonstrated a clear theoretical or empirical advantage of Survival Loss over the simpler Local Overlap objective. Additionally, the revisions are not highlighted in the manuscript, making it difficult to identify the changes. Therefore, I will maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.