acceptodds
Under review as a conference paper at ICLR 2027

PC-VOT: Prefix-Coupled Verifier-Induced Optimal Transport for Draft Training

Abstract

Speculative decoding accelerates LLM inference by using a lightweight draft model to propose candidate tokens that are verified in parallel by the target model. Diffusion drafters further improve drafting efficiency by generating an entire candidate block in a single forward pass. However, a fundamental training-decoding discrepancy remains: conventional objectives train the drafter to match the target distribution pointwise, while decoding depends on the verifier-induced decision regions that determine which draft proposals are accepted. We propose *Prefix-Coupled Verifier-Induced Optimal Transport (PC-VOT)*, a verifier-aligned training framework for diffusion drafters. PC-VOT constructs a verifier-induced feasible set around the Bayes-optimal proposal under stochastic verification and minimizes the point-to-set transport required to reach it. It then couples the transport cost across positions according to prefix reachability and the remaining speculative progress at risk. Theoretical analysis validates that PC-VOT provides a lower bound on expected acceptance length. Extensive experiments across seven math, coding, and dialogue benchmarks show that PC-VOT improves the average decoding speedup by up to 11.2% over DFlash baseline and 6.0% over D-PACE baseline.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Speculative Decoding, Draft Model Training

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ZJPQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PC-VOT, a training objective for diffusion drafters used with stochastic top-1 verification. For each position, it selects a target-probability maximizer, constructs a set of draft distributions whose probability margins favor that token, and defines the local loss as the minimum transport distance to that set. Then, the local losses are weighted by prefix reachability and the number of remaining draft positions. Under a positive margin assumption along the target's greedy trajectory, Theorem 3 bounds the expected accepted draft length from below relative to that trajectory. Experiments compare continued training from DFlash checkpoints across several benchmarks, target-model/mode configurations, and two decoding temperatures (0 and 1), reporting improvements in accepted length and wall-clock speedup without additional inference-time components.

Critical strengths

1. Theory: local-supervision idea with a valid connection to the accepted length.

The paper clearly distinguishes between top-1 verification and exact distribution matching. This distinction motivates the formal definition of their local-supervision objective ("verifier-induced point-to-set optimal-transport cost Diβ(qi;ti⋆)D_i^\beta(q_i; t_i^\star)", Eq. (8)), which avoids unnecessary restriction on the draft training (as discussed around Lemma 2). Moreover, by combining this with prefix reachability and remaining-horizon factors, this work proposes an exact PC-VOT training objective (Eq. (17)). In particular, the prefix reachability is motivated by the accepted-length analysis (also presented in Section 3 of the D-PACE paper [Wu et al., 2026]). The paper also presents a lower bound on the PC-VOT's mean acceptance length of the form "(max of mean acceptance length) - (scaled PC-VOT objective)" in its Theorem 3. I read all proofs in Appendix C and found no significant errors.

2. Experiment: consistent gain in accepted draft length & decoding speed

Table 1 reports improvements in both speedup and accepted length across the evaluated configurations, with the same initialization, architecture, training data, optimization schedule, and decoding pipeline. The gains are robust across their ablation studies (T∈0,1T\in\\{0,1\\}, β∈0.1,0.3,0.5,1.0\beta\in \\{0.1, 0.3, 0.5, 1.0\\}, and two factors of prefix coupling). In particular, I find that PC-VOT without prefix coupling already outperforms most other compared methods (DFlash, KL, LK, D-PACE), suggesting that their local objective is still useful. One caveat is that I am not sure how the authors obtained the speedups they report in the abstract (up to 11.2% over DFlash and 6.0% over D-PACE).

Critical weaknesses

1. Thm. 3 does not establish a particular advantage of PC-VOT, and the usefulness of its lower bound remains unclear.

  • The guarantee extends to other local losses, whose constants must be compared fairly.

    I think the existence of a lower bound on the accepted draft length (Thm. 3) alone does not directly imply the particular strength of the PC-VOT objective. The paper does not establish when the bound improves on the trivial bound of zero, or compare it with bounds obtained from other local losses using their own constants.

    After reading the proof of Thm. 3, I see it extends to a broader class of local objectives (beyond Dβ(⋅)D^\beta(\cdot)) with prefix coupling. Consider a local objective Fi(qi)F_i(q_i) at the draft prefix, define LF(ϕ)=E[∑i=1nwiFi(qϕ,i)]\mathcal L_F(\phi)=\mathbb E[\sum_{i=1}^n w_iF_i(q_{\phi,i})] using the paper's prefix coupling weights. Suppose Fi≥κF(γ)>0F_i\ge\kappa_F(\gamma)>0 for some κF(γ)\kappa_F(\gamma) whenever the draft selects a different token from a target with a unique maximizer and margin at least γ\gamma; in particular, I would say that Thm. 3 chooses/proves κDiβ(γ)=βγ2\kappa_{D^\beta_i} (\gamma) = \frac{\beta\gamma}{2}. Then, for the first disagreement ℓ\ell in Eq. (38), continuing from Eq. (39) and averaging the resulting bound, we have:

    \le w_\ell \le \frac{w_\ell F_\ell}{\kappa_F(\gamma)} \le \frac{\sum_{i=1}^n w_iF_i}{\kappa_F(\gamma)};\\ \therefore \mathbb E[L_{\mathrm{acc}}] \ge n^\star-\frac{\mathcal L_F(\phi)}{\kappa_F(\gamma)}.$$ For total variation ($F_i(q_i)=\operatorname{TV}(q_i,p_i)$), since $q_\ell(\hat t_\ell)\ge q_\ell(t_\ell^\star)$, I find $\kappa_{F_i} (\gamma) = \frac{\gamma}{2}$. It implies \$\mathbb E[L_{\mathrm{acc}}] \ge n^\star-\frac{2}{\gamma}\mathcal L_{\mathrm{TV}}(\phi).\$ Thus, TV also has the same sort of guarantee with no $\beta$ in its coefficient; I beilieve similar guarantees for KL, reversed KL, CE, and more can be derived, too. At first glance, applying Lem. 2 and simply replacing $D_i^\beta(q_i;t_i^\star)$ by TV in Eq. (18) while retaining $2/(\beta\gamma)$ gives a weaker bound, so I thought the OT-based local objective can be regarded as a better choice than TV; but my derivation above reveals that it makes an unnecessarily weak comparator. Hence, I do not read Thm. 3 as a **theoretical** guarantee that training with PC-VOT yields greater accepted length than training with TV or KL.
  • The margin assumption in Thm. 3 also needs a more informative treatment.

    For γ>0\gamma>0, Eq. (18) improves over zero-acceptance only if LPC-VOT(ϕ)<βγn⋆/2\mathcal L_{\mathrm{PC\text{-}VOT}}(\phi)<\beta\gamma n^\star/2. However, even a tie between target maximizers at even one reference position makes the defined γ\gamma zero. More generally, γ\gamma can be arbitrarily small, where the Thm. 3 becomes vacuous; arguably, this happens often in practice, as natural languages have many ambiguities (e.g., synonyms).

    I think we can accommodate the problem of a "tie" (in target probabilities) by changing γ\gamma to the gap between the largest and second-largest target probability values. But this change alone might be insufficient; as I independently checked, you should also change n⋆n^\star to nmin⋆n^\star_{\rm min}, the minimal mean accepted lengths across all possible greedy target sequences (up to length nn). This removes the zero gap caused solely by tied maxima, although small positive gaps can still weaken the bound. I would expect either a treatment of this scope or convincing evidence that the original bound is informative before treating Thm. 3 as a substantial theoretical contribution.

  • The ablations do not resolve the corresponding empirical question.

    Unlike Table 3 (removing prefix-coupling weights wiw_i while retaining the same sampled local loss), the paper does not provide an ablation on replacing the OT local objective with full-vocabulary TV or KL while retaining wiw_i's. So I think the benefit of using (exact) Diβ(qi;ti⋆)D_i^\beta(q_i;t_i^\star) in the training objective is not well separated, even numerically. Note that I acknowledge the relevant β=1\beta=1 case in Table 2: all sampled endpoints then collapse to pˉi\bar p_i, giving prefix-weighted TV⁡(qi,pˉi)\operatorname{TV}(q_i,\bar p_i). However, the paper uses a sparse variant of the training objective as explained in Appendix D (which I will discuss more on later), where changing β\beta modifies both the margins and the OT endpoint probabilities. Although the reported gains from β=1\beta=1 to β=0.5\beta=0.5 are about 0.9% and 1.5% in average speedup at the two temperatures, I believe it does not isolate the claimed advantage over ordinary local supervision with the same prefix weighting.

2. The actual implemented training objective (Appx. D) differs substantially from the exact objective Eq. (17), which is not fully convincing.

  • The main text should have identified the implemented objective.

    The "training objective" paragraph in Section 3.3 presents the full form of the exact/idea PC-VOT objective in Eq. (17) and points to Appx. B for the full pseudocode of the training pipeline. However, it omits the actual training-loss implementation from the main text, which is explained in Appx. D. It includes many details that belong in the main text and, more importantly, the optimizer hyperparameters; e.g., the stop-gradient operation for the prefix-reachability factor, sub-sampling of the support and feasible distributions, some valid reparameterizations, and so on. Although I do not think they should provide a full theoretical guarantee on their practical training objective, the paper should clearly mention it in the main text, especially near Eq. (17). This is mainly because computing and backpropagating through the exact objective seems computationally prohibitive (mostly due to the prefix-reachability factors RiR_i and the OT formulation of DβD^\beta).

  • The original local objective is tractable and allows backpropagation.

    Quoting Appendix D, it says: "Directly constructing and optimizing over the verifier-feasible set on the full vocabulary simplex is computationally intractable because of the large vocabulary size and the continuous family of feasible distributions." I am not quite convinced by this claim, as I derived a closed-form formula for Diβ(qi;ti⋆)D_i^\beta(q_i;t_i^\star) that has the same computational cost as sorting over the vocabulary, which is quite tractable. First of all, for the specified ground cost ci(a,b)=1a≠bc_i(a,b)=\mathbf1\\{a\ne b\\}, we already have WCi(qi,q~i)=TV⁡(qi,q~i)W_{C_i}(q_i,\tilde q_i)=\operatorname{TV}(q_i,\tilde q_i): it is the standard maximal-coupling connection; see, e.g., Section 4 in Sun et al. (2023).

    Now, for simplicity, fix any ii and simplify the notation: p=pi,q=qi,t=ti⋆,Dβ(q;t)=Diβ(qi;ti⋆)p= p_i, q=q_i, t=t_i^\star, D^\beta (q;t) = D^\beta_i(q_i;t_i^\star), and so on. Let vocab size be M≥2M\ge 2. For competitors v≠tv\ne t, define r(v):=β[p(t)−p(v)]−[q(t)−q(v)]r(v):=\beta[p(t)-p(v)]- [q(t)-q(v)] and sort them as r(v1)≥⋯≥r(v(M−1))≥0=:r(t)r{(v_1)}\ge\cdots\ge r(v_{(M-1)})\ge 0 =: r(t). Then, with [x]+=max⁡x,0[x]_+=\max\\{x,0\\}, I claim

    =\left[\max_{1\le k\le M-1} \frac{\sum_{j=1}^k r(v_j)}{k+1}\right]_+.$$ Here is a short derivation. Observe that the function $f(x):=x-\sum_{v\ne t}(r(v)-x)_+$ is continuous and strictly increasing on $\mathbb R$. It is nonpositive at zero and nonnegative at $1-q(t)$; thus, its root $\delta$ exists uniquely in that interval; i.e., $\delta = \sum_{v\ne t}(r(v)-\delta)_+$. Also, note that $\sum_{j=1}^k r(v_j)\le(k+1)\delta$ for every $k$; the equality holds when $k$ exactly counts the competitors $v\ne t$ with $r(v)>\delta$, unless that there is not such a competitior, where $\delta=0$. This proves $\delta = \left[\max_{1\le k\le M-1}\frac{1}{k+1}\sum_{j=1}^k r(v_j)\right]_+$. Next, take any feasible $\tilde q\in\mathcal A^\beta(t)$ and let $d=\operatorname{TV}(q,\tilde q)$. Since no coordinate can increase by more than $d$, $\tilde q(t)\le q(t)+d$, and feasibility implies $\tilde q(v)\le q(t)+d-\beta(p(t)-p(v)) = q(v) + d - r(v)$. Consequently, $$d=\sum_v(q_i(v)-\tilde q_i(v))_+ \ge\sum_{v\ne t_i^\star}(r(v)-d)_+; \quad f(d)\ge0\quad\Longrightarrow\quad d\ge\delta.$$ Now, it remains to show that this lower bound is attained in $\mathcal A^\beta(t)$, Indeed, we set $q^\star$ by $q^\star(t) = q(t) + \delta$ and $q^\star(v) = q(v) - [r(v) - \delta]_+$ for $v\ne t$; we can show it is a valid & feasible probability in $A^\beta(t)$ that satisfies $TV(q, q^\star)=\delta$. This proves $D^\beta(q;t) = \delta$. Once $p_i$ and $q_i$ are available, the local forward calculation takes $O(|V|\log |V|)$ operations, the explicit backward calculation takes $O(|V|)$ operations, and the auxiliary storage is $O(|V|)$ per position. This avoids a costly $|V|\times |V|$ transport matrix. For a length-$n$ block, these local computation costs become $O(n|V|\log |V|)$ and $O(n|V|)$, excluding model computation and rollout generation. Since the exact loss is computable, I would want a controlled training comparison with this loss before accepting the proposed explanation for the gains. If the sampled surrogate works better, that would also be informative, but the paper would then need to explain and evaluate the benefit of its restrictions rather than attribute it to exact minimum decision repair.
Questions for authors

Please see Strengths and Weaknesses; I have noted all my confusions and additional thoughts there. However, to list some important action items for the authors;

  1. "I am not sure how the authors obtained the speedups they report in the abstract (up to 11.2% over DFlash and 6.0% over D-PACE)." -> How do you report this?
  2. Can you verify and numerically implement/evaluate the full sorting-based implementation of DβD^\beta I briefly derived in W2?
  3. Can you derive a more robust and less vacuous lower bound on acceptance lengths as per the discussion I provide in W1?
  4. Can you develop a more concrete theoretical guarantee about the strength of the PC-VOT objective in terms of mean draft acceptance length compared to the other objectives? Or, at least, can you discuss the regime where PC-VOT is meaningfully beneficial, based on the theoretical analysis of the accepted-length lower bounds?

Q1 is a clarification question; Q2-4 are suggestions for further improving the current manuscript (or for future work).

Reviewer_vq1YOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes PC-VOT, a training objective for diffusion drafters in speculative decoding. Instead of replicating the distribution given by the target model, the drafter only needs to produce the correct top-1 token with a sufficient probability margin with Point-to-set optimal transport. It further introduces prefix coupling to assign higher weights to earlier positions based on prefix reachability and the remaining speculative horizon. The authors evaluate it on seven math, coding, and dialogue benchmarks, reporting average decoding speedup improvements of 5.2% over D-PACE.

Critical strengths
  1. The paper studies a meaningful topic, improving diffusion-drafter training for speculative decoding without adding inference-time components.
  2. The experimental evaluation is broad, with comparisons against several objectives under a shared drafter architecture and training setup.
Critical weaknesses
  1. The analysis assumes that the drafter deterministically proposes, and acceptance depends on whether this token matches a token sampled from the target distribution, does not establish generality to standard residual-sampling speculative decoding. Although the paper evaluates at T=1, this experiment does not address the concern regarding the generality of the proposed formulation.
  2. The necessity of OT remains unclear, as a simpler margin-based loss with the same prefix weighting could potentially achieve similar gains. The current ablations do not isolate the contribution of point-to-set OT itself.
Questions for authors

Q1: Can the authors evaluate the proposed method under standard residual-sampling speculative decoding?

Q2: Can the authors compare PC-VOT against a simpler margin-based loss using the same prefix weighting to isolate the contribution of point-to-set OT?

Q3: How much of the reported improvement comes from the point-to-set OT objective itself, rather than the relaxation of distribution matching or the prefix-coupling weights?

2 comments

Thank you for addressing my concerns. I have a follow-up question about residual sampling, which is particularly relevant in high-entropy settings. Could you evaluate the existing TV-trained Qwen3-4B drafter under the same residual-sampling setup and report its mean acceptance length and speedup compared to PC-VOT? If time permits, I would also be interested in seeing the same comparison on Qwen3-8B.

Thank you for the detailed responses. However, the speedup gains over a simple TV objective under top-1 verification is modest (approximately 1.8%). Meanwhile for residual sampling, PC-VOT slightly underperforms TV (approximately -2.4%). My main concern remains, I will maintain my original rating.

Reviewer_kyqqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a new speculative decoding approach by relaxing the requirement that the draft distribution must match the target distribution. Instead, it only requires the drafter to satisfy a feasible region. To train a drafter in this way, the authors propose point-to-set optimal transport. The proposed method is tested on several models and benchmarks and shows improvement over Dflash and D-pace.

Critical strengths
  • The identified problem that the requirement for exact matching between the drafter and verifier is too strong in speculative decoding is reasonable.
  • The proposed solution is theoretically justified and well-motivated.
  • The empirical results demonstrate improvement over state-of-the-art baselines.
Critical weaknesses
  • My main concern is that the cost of the proposed method might be high. Optimal transport is typically very expensive and it is not entirely clear whether the proposed method makes any approximation to reduce the cost. If so, what will the approximation error be? It will also be better to discuss the cost of training and inference, and also compare with the cost of existing approaches.

  • The proposed approach seems not related to diffusion models. The idea of point-to-set transport can be applied to AR model drafters as well. Is there any reason to focus on diffusion drafters? Could the approach be applied to general speculative decoding?

  • If it is applicable to AR speculative decoding too, it will be interesting to add empirical results and compare with some baselines.

Questions for authors

See weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.