acceptodds
Withdrawn from ICLR 2027

Confidence-Adaptive Permutation-Tree Self Speculative Decoding for Diffusion Large Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion-based Large Language Models (dLLMs) offer a promising alternative to autoregressive models by enabling parallel decoding and bidirectional context modeling. However, existing speculative decoding methods for dLLMs rely on linear or heuristic verification structures that fail to capture the full space of valid generation orders within a block, leading to premature rejection of correct tokens and suboptimal acceptance rates. In this work, we propose CAPSpec (Confidence- Adaptive Perm-tree Speculative Decoding), a training-free framework targeting block-based diffusion models. CAPSpec exploits the key insight that, within a fixed-length block, the n remaining masked positions admit n! possible decoding orders, forming a full permutation tree whose size grows factorially. By first constructing this complete permutation tree over a block’s decoding trajectories and then applying Adaptive Confidence Partitioning (ACP)—which clusters candidate positions by their draft-stage confidence and prunes low-value branches before verification—CAPSpec compresses the factorial-sized search space into a compact tree that retains all high-probability trajectories. A single batched forward pass over the pruned tree evaluates the retained trajectories and identifies a unique accepted path. Extensive experiments on LLaDA-8B-Instruct across mathematics, code, and dialogue benchmarks demonstrate that CAPSpec achieves up to 3.08× wall-clock speedup over step-by-step baselines, establishing a new state of the art for training-free speculative decoding in block-based diffusion language models.

Primary area:
generative models
Keywords:
Speculative Decoding, Diffusion Large Language Models (dLLMs), Permutation Tree, Confidence-Adaptive Partitioning, Parallel Verification

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_F7mHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Existing speculative methods for dLLMs fail to capture all generation orders, which could lead to suboptimal acceptance rates. The paper proposes considering the permutation tree. To make this computationally feasible, the paper proposes several tricks to prune the tree. The speed improvements on several models are demonstrated.

Critical strengths
  1. The method is training-free and relevant to accelerating dllm inference. The speedup is promising.
Critical weaknesses
  1. I could not find any benchmark accuracies for CAPSpec in the submission. Did I miss it? Reporting accuracies is important to empirically prove that CAPSpec is lossless. In particular, the pruning still uses static draft confidence. The confidences of other tokens will likely change after a new token is committed. Using the precomputed confidences as pruning scores for all intermediate states may lead to accuracy degradation.
  2. It seems like ACP does not perform noticeably better than the baseline K1.
Questions for authors
  1. Can the authors provide the accuracy table and an explanation of why CAPSpec might be lossless?
  2. Can the authors clarify the relationship between the number of accepted nodes (e.g., in Table 2) and the number of mean accepted tokens (MAT) per forward pass?
Reviewer_MBpjOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

CAPSpec proposes training-free self-speculative decoding for block-based diffusion language models. It constructs candidate token-reveal orders as a permutation tree, prunes shallow branches by confidence clustering, and applies narrower top-k branching at deeper levels. Logical histories reaching the same filled block share a physical segment in packed verification. The model checks candidate edges in a verification pass, and the algorithm commits a selected path plus a bonus token.

The motivation is that revealing one token can change the confidence ranking of remaining positions, making a fixed initial order waste useful speculation. Experiments span five models and four workloads, reporting 1.27-3.08x throughput over native one-token-per-step decoding. Comparisons with width-one and width-two speculative variants show smaller incremental gains and some cases where width one is faster. The main evaluation does not report CAPSpec's task quality or complete-output identity with the reference decoder.

Critical strengths
  1. The paper identifies a meaningful limitation of fixed-order speculation. Context updates can reorder position confidences, so representing alternative reveal orders is a reasonable way to recover otherwise rejected draft tokens. This motivates a potentially useful contribution to self-speculation for diffusion models.

  2. Pruning and deduplication target the main practical cost of that idea. Reusing evaluations of identical filled states and reducing low-value branches could make richer speculation affordable. Tables 1-2 expose the acceptance-versus-verification-cost trade-off, including cases where a simple single path wins. These are substantive positive aspects, though they depend on correct state evaluation and a well-defined reference.

Critical weaknesses
  1. Token agreement is insufficient to preserve the declared confidence-greedy policy. Section 3.2 chooses the next unresolved position by its current maximum confidence. Section 4.4 checks only whether the draft token agrees with the parent's token argmax at the proposed position. Appendix A.1 selects the first accepted child using original draft confidence. If two tokens still match locally but their current position ranking has reversed, this rule can choose a different reveal order; later token predictions can consequently change. The uniqueness proof shows that a path is selected deterministically, not that it is the reference path. This is central to whether the speedup preserves the target computation.

  2. Packed-state equivalence and output quality are not established. Section 4.4 lets shared prefix/suffix queries attend to the anchor segment. In a multilayer bidirectional transformer, those shared hidden states can depend on which branch's block is filled, so non-anchor branches can receive context different from an independent reference forward. Preventing direct sibling attention does not establish the needed equivalence. No packed-versus-independent logit test, complete-output agreement result, or CAPSpec task-quality table resolves this issue. Without exactness evidence or an explicit approximate-decoder quality evaluation, the reported throughput cannot establish acceleration at preserved quality.

Questions for authors
  1. Clarification: Does the implementation require an accepted edge's position, as well as its token, to match the parent's current confidence-greedy choice over all relevant unresolved positions? If so, where is that rule specified? If not, what reference policy is preserved?

  2. Clarification: How are branch-dependent shared prefix/suffix states handled across transformer layers, and are original position IDs reused for each segment? Does the reference use the same context-cache approximation, or are the segments intended to reproduce independent full-context forwards?

  3. Evidence needed for assessment: Can you provide packed-versus-independent logits and complete-output agreement with Step, including ranking-swap cases? If output identity is not intended, please provide CAPSpec's quality relative to Step and K1 at the reported speed settings. These results are needed to interpret the principal claim, not merely to extend its scope.

Reviewer_BxENOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

CAPSpec is a training-free self-speculative decoding framework for block-based diffusion language models. An initial model evaluation drafts tokens and ranks masked positions by confidence. After committing an anchor, the method represents candidate reveal orders with a permutation tree, prunes shallow branches through ACP, and reduces the branching width at greater depths. Distinct logical paths that produce identical filled-position sets share physical segments in a packed verification pass. The decoder then commits a selected accepted path and a bonus token. Across 5 model variants and 4 benchmarks, the paper reports 1.27-3.08x throughput improvements over native one-token-per-step decoding and the highest throughput in most of the comparisons with K1/K2 alternatives. The main contribution is the allocation and reuse of verification computation across alternative position orders.

Critical strengths
  1. A useful decomposition of the verification problem. Sections 4.1-4.4 separate reveal-order histories from the token snapshots on which model computation is performed. Preserving logical parent chains while sharing computations for duplicate snapshots is a useful design idea for diffusion decoding, where different position orders can reach the same state. The framework uses the existing model for drafting and verification and does not require a separately trained drafter, making the approach practically attractive if its quality and implementation are validated.
  2. Evidence that selective branching can improve the tested verification tradeoff. The evaluation includes 5 model variants and mathematics, code, and dialogue benchmarks. ACP achieves the highest reported throughput in most settings. In Table 2, on HumanEval with LLaDA-8B-Instruct, ACP retains 36.8 logical nodes on average versus 110.2 for K2, while increasing the accepted-node count from 4.5 to 5.0.
Critical weaknesses
  1. The speedups are not accompanied by an established generation-quality comparison. Section 3.2 defines the reference Step decoder by selecting the currently highest-confidence masked position. In contrast, Section 4.4 accepts an edge when its drafted token matches the parent-conditioned argmax. These rules can choose different next positions: if A initially ranks above B, both drafted tokens still match their parent-conditioned argmax, and B now has higher confidence, CAPSpec can choose A while Step chooses B.
  2. Figure 2 measures the quality of changing the native per-step token count, while it does not establish CAPSpec's quality. Tables 1-2 and the appendices do not report corresponding task accuracy for Step, K1, K2, and ACP. The reported speedups do not establish acceleration at comparable output quality.
  3. In the paper, k_t=1 is chosen as a quality reference, partially supported by figure 2, however, broader practical claims also need comparison with other decoding schedules at matched quality. While K2 denotes tree width, a k_t=2 baseline is not supplied.
Questions for authors
  1. Is CAPSpec intended to reproduce the confidence-greedy decoder in Section 3.2, or to define a different position-selection policy? If equivalence is intended, how does Eq. (14) handle cases where current position confidences reorder but multiple drafted tokens still pass the token-match check?
  2. What are the task-quality results for Step, K1, K2, and ACP under the same prompts, generation limits, and stopping criteria used for the throughput measurements? If CAPSpec changes the reference output, how does its speed-quality tradeoff compare with native schedules that commit more than one token per step?
  3. How would the algorithm be modified for a k_t=2 setting?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.