- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DSpark-OPD, an on-policy post-training method for the existing Markov-corrected DSpark speculative drafter. Target-assisted rollouts collect draft-generated predecessor chains, and replay evaluates the target and the corrected drafter distribution on those same chains. The method combines acceptance-aware forward/reverse KL distillation with Survival Loss, which aggregates products of target–draft overlap over verification-reachable prefixes.
Evaluation uses Qwen3-8B and Qwen3-14B, block length seven, seven public benchmarks, and a private Business dataset. Relative to the matched OPD-Base model, adding Survival Loss improves reported public-average greedy acceptance length by 4.7% and 5.4%, respectively, and throughput at concurrency eight by 4.9% and 6.1%.
Critical strengths
The main experiments report positive acceptance-length improvements over OPD-Base for every evaluated dataset–temperature pair at both model scales. Importantly, the evaluation includes end-to-end throughput rather than relying only on acceptance length as a proxy.
The results make the approach potentially useful in practice, although the incremental gains should be distinguished from the larger speedups attributable to the complete speculative-decoding system.
Critical weaknesses
Table 2(a) shows that local overlap and fixed-decay overlap slightly outperform Survival Loss in all three public-average columns and all three Business columns at the common coefficient. For example, under greedy decoding, public-average acceptance length is 5.158 for local overlap, 5.160 for fixed decay, and 5.137 for Survival Loss; the corresponding Business values are 4.393, 4.412, and 4.292. The paper appropriately describes these variants as competitive, but this evidence supports the usefulness of overlap augmentation more directly than the benefit of its particular prefix-credit construction. Avoiding a positional-decay hyperparameter also does not distinguish Survival Loss from uniform local overlap.
More broadly, I could not find the serving hardware/backend configuration, training-set sizes, essential optimization and rollout hyperparameters, or repeated-run uncertainty estimates in the submitted manuscript. Training relies on private Business partitions, making a sufficiently detailed reproducible recipe particularly important. Without these details, it is difficult to assess the robustness and reproducibility of the relatively modest incremental public-benchmark gains.
Questions for authors
-
Could you reconcile Figure 2(b) with Table 2 and the definition ? Do the figure and table use different checkpoints, weighting conventions, scheduling settings, or evaluation populations? Please also clarify whether “prefix survival at position k” means accepting through position k or merely reaching its verification decision.
-
Given that L1 and L2 outperform L3 in all aggregate columns of Table 2(a), what evidence distinguishes the benefit of adaptive prefix credit from that of adding dense overlap supervision? Are repeated-run comparisons available? Otherwise, would you narrow the interpretation to overlap augmentation being effective, with Survival Loss one competitive formulation?
-
Please provide the hardware and inference backend, training-data quantities, key optimization/rollout settings including rollout temperature and and available run-to-run variability. Please also confirm that OPD-Base and OPD+Survival share the same training budget, confidence-head supervision, and inference scheduling configuration.
Figure 2(b) reach metric, separate evaluation runs, and shared settings
Thank you for the thoughtful review and for taking the time to compare Figure 2(b) with Table 2. We appreciate your emphasis on reproducibility and on distinguishing the incremental throughput gains from the full-system speedups.
Figure 2(b) and Table 2. Each bar in Figure 2(b) measures reaching the verification decision at position k. Position k must be offered for verification and the first k−1 draft tokens must be accepted. This includes rounds in which the verifier rejects position k. The tail-sum identity for Eq. (2) uses acceptance through k, Pr(r≥k), over the same verification rounds and with the same averaging weights as τ+. Figure 2(b) and Table 2(a) use the same L0 and L3 checkpoints in separate evaluation runs. Section 5.1 and the figure caption now specify both the metric and the evaluation provenance.
Objective comparison. Overlap augmentation improves acceptance length under all three weighting rules in Table 2(a). L1 and L2 lead the fixed-coefficient aggregate comparison. Survival Loss is a competitive formulation that derives its prefix weights from the current target–draft agreement, as quantified by Eq. (16). Sections 5.3 and 6 now state the common improvement and the gradient property separately. Table 3 reports incremental throughput gains of 4.9% and 6.1% over OPD-Base and full-system speedups of 2.07× and 2.40× against target-only decoding.
Training and serving protocol. The controlled 8B variants share the initial checkpoint, rollout protocol, optimization settings, and inference scheduling configuration. Section 4.3 and Appendix C now specify that L0 and L3 share the base OPD and confidence-head objectives, with λs=0 for L0 and 1 for L3 at the common comparison point. Confidence training uses stop-gradient overlap targets and updates only the confidence head. Across both model scales, the target is frozen, thinking is disabled, the draft cap is seven, and rollout offers the full block for verification with confidence-based truncation disabled. The rejected-stream KL uses γ=0.9 and βrej=2. SFT and OPD use disjoint Business training partitions, with a separate held-out test set. The reported acceptance values are evaluation-run point estimates for the listed checkpoints.