acceptodds
Under review as a conference paper at ICLR 2027

XPress: Parallel Refinement for Diffusion Drafters in Speculative Decoding

26 views

Abstract

Block-diffusion drafters like dFlash generate an entire block of draft tokens in a single forward pass, drastically reducing the overhead of multiple-token drafting in speculative decoding. The crucial final step of the single-pass discrete denoising process involves using the logit distribution at each position to sample conditionally independent tokens. The resulting draft is thus a set of per-position marginals, rather than a joint distribution: no draft token is guaranteed to depend on its predecessors. Such independently sampled marginals tend to produce sequences with tokens that are individually likely, but jointly improbable under the target model's distribution, which verifies each token conditionally. This can cause early rejection and limits acceptance length. To address this, we propose XPress as a means to restore the missing causality in diffusion drafters. XPress is a lightweight causal refiner that reconciles the whole diffusion block at once through parallel refinement, restoring and propagating causal dependencies across the draft without a token-by-token loop. On Qwen3-8B, across seven math, code, and chat benchmarks, XPress raises acceptance length by about 30% on average (up to +56%) and its end-to-end decoding throughput by about 1.3 on average (up to 1.5) compared to the original dFlash diffusion drafter.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Speculative Decoding, Diffusion Language Model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_88ktOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes XPress, a causal refiner that restores missing causal dependencies in diffusion drafters to improve the acceptance rate of block-diffusion drafters in speculative decoding. The method addresses a key limitation of dFlash: although it achieves higher drafting throughput, its relatively low acceptance rate limits the overall efficiency of speculative decoding. By reintroducing causal information into the diffusion drafter, XPress aims to improve the acceptance rate while preserving the benefits of parallel token generation. Experiments on Qwen3-series demonstrate improvements in both acceptance length and end-to-end throughput.

Critical strengths
  1. Demonstrated performance improvements. XPress improves the average acceptance length of the dFlash drafter by approximately 35% and achieves an average end-to-end throughput improvement of approximately 1.3× on Qwen3-8B, demonstrating its practical effectiveness.
  2. Addresses a meaningful and well-motivated challenge. dFlash generates an entire block of tokens in parallel, improving drafting throughput. However, unlike autoregressive drafting, each token is not explicitly conditioned on its preceding tokens, which can result in a lower acceptance rate and consequently limit the overall throughput of speculative decoding. The paper identifies this important trade-off and proposes a method to reintroduce causal information into the diffusion drafter.
  3. Clear presentation and motivation. The paper is well-written and easy to follow. The motivation is clear, and the problem is well-defined, making the proposed approach straightforward to understand.
Critical weaknesses
  1. Potentially high training overhead. XPress requires joint training with the draft model, which may introduce substantial computational overhead. However, the paper does not provide sufficient information about the associated training costs, making it difficult to assess the overall cost-effectiveness of the proposed approach.
  2. Incomplete isolation of the benefits of causal modeling and parallel refinement. The paper does not sufficiently disentangle the contributions of causal modeling and parallel refinement. More comprehensive ablation studies are needed to clarify the independent benefits of these mechanisms and identify which components are primarily responsible for the observed improvements.
  3. Limited analysis of practical overhead. Although the paper characterizes refinement latency, it lacks a sufficiently comprehensive analysis of the additional parameter count, GPU memory consumption, and how the refinement overhead scales with block size and vocabulary size. These factors are important for assessing the practical deployment costs and scalability of XPress.
  4. Limited generalizability of the experimental evaluation. The proposed method is evaluated only with dFlash as the diffusion drafter, and the experiments are restricted to the Qwen model family. Its effectiveness across different diffusion drafting architectures and target model families remains unverified, limiting the generalizability of the reported results.
Questions for authors
  1. What is the computational cost of training XPress? Specifically, how many GPUs and how much training time are required?
  2. How does XPress perform with other diffusion drafters and target model architectures?
  3. What motivates the decision to jointly train XPress with the draft model? Would it be possible to train the causal refiner independently, potentially reducing the training cost?
  4. Domino and DSpark have already introduced causal correction mechanisms. XPress primarily employs Jacobi-style parallel refinement and a broader scope of causal mixing. Could the authors clarify the independent contributions of these mechanisms relative to existing approaches? Specifically, how much improvement can be attributed to each component individually?
Reviewer_xBXnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

XPress adds a lightweight learned causal refiner to a parallel diffusion drafter for speculative decoding of an autoregressive target. The refiner combines preceding sampled tokens, drafter hidden states, and a block-global summary through low-rank feature fusion and causal position mixing, then adds a correction to the base logits. Jacobi iterations refine all draft positions in parallel. Training includes target supervision, an acceptance-oriented distributional objective, a base-drafter anchor, and self-conditioned consistency.

The paper proves finite convergence for fixed features and analyzes target-distribution preservation for finite-iteration stochastic drafts under specified fixed-noise and proposal-probability conditions. Evaluation covers Qwen3-4B/8B, seven workloads, greedy and temperature-one decoding, and batched vLLM execution. On Qwen3-8B, reported mean speedups over autoregressive decoding are 6.2x versus 5.9x for the matched Markov head at temperature zero, and 5.2x versus 4.7x at temperature one.

Critical strengths
  1. A useful method with evidence against a close, matched baseline. Table 1 compares with a Markov head retrained under the same data, loss schedule, and step budget. XPress improves acceptance and reported throughput across the tested configurations, with stronger gains at temperature one. This comparison makes the incremental benefit more credible than comparison against dFlash alone.

  2. An explicit treatment of parallel refinement and sampling correctness. Sections 4.1-4.2 and Appendices B-D distinguish causal token dependence, finite convergence, and losslessness without full convergence. This is consequential because it explains how parallel draft refinement can remain compatible with exact target sampling under the specified proposal bookkeeping. The batched serving results additionally show that the implementation is useful beyond a single active sequence relative to dFlash.

Critical weaknesses
  1. The high-batch claim lacks the strongest matched comparator and uncertainty for narrow gains. Tables 2-3 omit the retrained Markov head at batch sizes 16 and 32. Meanwhile, some single-sequence greedy gains over that head are only a few percent, and the paper acknowledges clock drift of several percent while reporting five-round means without intervals. This leaves the size and reliability of the incremental systems contribution less established than the headline improvement over dFlash.

  2. The implementation of the sampling guarantee is not sufficiently auditable. For finite K, Appendix D uses proposal probabilities from the final Jacobi update, conditioned on the previous iterate, together with fixed, position-independent noise rows. Recomputing probabilities using the final draft prefix would generally be a different proposal. The paper does not show enough verifier detail or direct validation to establish that the measured implementation follows the proved construction. This is a request to connect a plausible theorem to the implementation, not an identified contradiction in the theorem.

Questions for authors
  1. Evidence needed for assessment: What are the throughput and latency differences against the matched Markov head in vLLM at batch sizes 16 and 32, and what paired timing uncertainty accompanies the principal comparisons? This would determine whether the claimed incremental serving benefit is established.

  2. Clarification: Which exact probabilities are passed to rejection sampling at temperature one: those retained from the final update using the previous iterate, or probabilities recomputed from the final draft? Please specify the noise reuse and any sampling filters, and identify available correctness checks. An implementation mismatch here would materially affect the soundness assessment.

Reviewer_entBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces XPress, which adds a causal correction head to a DFlash drafter and uses Jacobi iterations to refine the draft block in parallel. The head combines drafter hidden states with preceding draft tokens to adjust the base logits, using a causal mixer to capture dependencies within the block. The drafter and correction head are trained jointly, with supervision on both the base and refined predictions and an additional self-conditioned training pass. Experiments on Qwen3-4B and Qwen3-8B cover greedy decoding, stochastic sampling, and batched inference in vLLM. XPress improves acceptance length and throughput over the Markov-head baseline, with throughput gains over DFlash in batched vLLM inference.

Critical strengths
  1. The design is principled, introducing causal dependencies through parallel refinement without repeating the expensive drafter computation.
  2. Experiments show consistent improvements in acceptance length and end-to-end throughput over the Markov-head baseline under matched training settings.
  3. The ablations help explain the roles of the training losses, hidden-state inputs, and causal mixer.
Critical weaknesses
  1. The comparison with the Markov head in Figure 4 changes both the correction architecture and decoding schedule. A Markov-plus-Jacobi control would better isolate the benefit of the proposed head.
  2. Although Table 4 shows a benefit from refinement with the drafter frozen, a plain DFlash control with the same data and additional training budget is not reported. The batched experiments (Tables 2–3) also omit the Markov baseline.
  3. The timing results are averaged over five rounds but lack variability estimates, making the modest greedy gains over Markov harder to assess.
Questions for authors
  1. How does the Markov head perform with Jacobi refinement across iteration budgets, in terms of acceptance length and measured end-to-end throughput?
  2. How much of the gain remains after further training plain DFlash on the same data for the same number of steps? Does the throughput advantage over Markov persist at batch sizes 16 and 32?
  3. Could the authors report paired throughput gains over Markov across the five measured rounds, together with standard deviations or confidence intervals?
1 comment

Thank you for the detailed rebuttal and for running the additional experiments. The new results address the concerns I raised in my review and strengthen the empirical support for the paper’s conclusions.

I see XPress as a solid incremental contribution to speculative decoding. In light of this, I will keep my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.