acceptodds
Desk rejected at ICLR 2027

SpecFold: Folding Multi-Branch Redundancy for Faster Speculative Decoding in Diffusion Language Models

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion large language models (DLLMs) generate text through iterative block denoising, and multi-branch speculative decoding accelerates this process by verifying a main branch together with multiple draft branches in a single forward pass. While prior DLLM acceleration methods primarily exploit temporal redundancy across denoising steps, we identify a complementary redundancy axis within each speculative verification pass: multi-branch computational redundancy. During speculative verification, draft branches inherit most tokens from their parents while unmasking a small set of additional positions, causing large portions of hidden states to remain highly similar across branches. We propose SpecFold, an algorithm–system co-design that exploits this multi-branch redundancy to reduce the cost of multi-branch speculative verification. Algorithmically, SpecFold performs token-level residual gating and selectively reuses parent computation through folded attention and FFN while preserving residual hidden states. Systemically, a Triton kernel implementation translates this fine-grained reuse into end-to-end throughput gains through efficient sparse multi-branch execution. SpecFold is orthogonal to temporal caching and compatible with existing DLLM speculation strategies. Across two DLLM families, five models, and five standard benchmarks, SpecFold achieves up to throughput over Spiffy and up to over vanilla decoding, while maintaining comparable task performance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Language Models, Efficient Inference, Speculative Decoding, Cross-Branch Redundancy, Activation Reuse, Decoding Algorithms, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_8sC1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SpecFold reuses QKV and FFN computation across diffusion branches with similar hidden states. Custom kernels handle reuse and partial recomputation. Tests on Fast-dLLM-v2 and Nemotron-Labs-Diffusion show speedups over Spiffy.

Critical strengths
  • Reusing work across branches is a useful idea beyond reducing denoising steps.
  • The paper shows real speedups on block-diffusion models and includes several useful ablations.
Critical weaknesses
  • Accuracy does drop in some cases. On Fast-dLLM-v2-7B HumanEval, it falls from 64.63% with Spiffy to 61.59%. This is a speed–quality trade-off, not lossless acceleration.
  • Gains shrink with model size. The best gain over Spiffy on the 14B model is only 1.13x. A runtime breakdown would help show how much the reuse logic itself costs.
Questions for authors
  1. Do the accuracy drops persist across runs? Can a lower reuse threshold recover quality?
  2. How much time goes to gating, packing, and looking up reused states?
Reviewer_nJXDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SpecFold to reduce the cost of multi-branch speculative decoding in block diffusion language models. It uses parent–child hidden-state differences to selectively reuse QKV projections and FFN outputs, while correcting attention for changed keys and values and retaining each branch’s residual stream. A Triton implementation supports sparse execution. Experiments on five models and five benchmarks report up to 1.59× throughput over Spiffy and 1.95× over vanilla decoding.

Critical strengths
  1. The paper identifies a useful opportunity to reuse computation across related draft branches. Correcting attention for changed keys and values while retaining branch-specific residual streams is a thoughtful way to exploit this similarity without simply copying entire branch states.

  2. The implementation translates the proposed reuse into measured throughput gains across two model families and several model sizes. The threshold, draft-budget, and hardware experiments provide useful evidence about the method’s practical behavior.

Critical weaknesses
  1. The contributions of the main components are not sufficiently isolated. The ablations vary thresholds and draft graphs, but do not separately assess attention correction, FFN reuse, or residual preservation. A dense comparison using comparable kernel optimizations is also missing. In addition, Section 4.1 disables block caching, and it is unclear whether the Spiffy baseline includes its draft-pruning procedure. These details matter for understanding which parts of SpecFold drive the gains and how much benefit remains over an optimized baseline.

  2. Quality preservation needs stronger evidence. On Fast-dLLM-v2-7B, HumanEval accuracy falls from 64.63% with Spiffy to 61.59% with SpecFold, without uncertainty estimates. Appendix C addresses numerical differences in Spiffy rather than the additional approximation from folding. The δ→0 validation also needs clarification: the pre-attention gate is reused for the FFN, although attention can change a token’s state even when its layer input matches the parent’s. Thus, δ=0 does not obviously recover dense computation.

Questions for authors
  1. Can you provide component ablations that isolate the effects of attention correction, FFN reuse, and residual preservation on both accuracy and latency?

  2. Does the Spiffy baseline include draft pruning and comparable kernel optimizations? Which caching mechanism is disabled in Section 4.1, and how much incremental speedup remains when it is enabled?

  3. Does the implementation use a separate FFN gate or special handling of δ=0? Please clarify how the claimed recovery of dense computation is validated.

  4. Can you provide paired uncertainty estimates for the main accuracy differences, especially Fast-dLLM-v2-7B on HumanEval, and fixed-input comparisons that separate folding error from numerical differences?

Reviewer_o58qOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SpecFold, an algorithm-system co-design to reduce the verification cost of multi-branch speculative decoding in diffusion LLMs. The key observation is that, inside one verification pass, most hidden states of a child draft stay close to those of its parent. SpecFold uses a per-token residual gate to reuse the parent's QKV and FFN outputs at these positions, corrects the attention for the few changed tokens, and implements the reuse with Triton kernels on top of Spiffy's draft graphs. On Fast-dLLM-v2 and Nemotron-Labs-Diffusion across five benchmarks, the paper reports up to 1.59× throughput over Spiffy and 1.95× over vanilla decoding.

Critical strengths
  1. Cross-branch redundancy inside one verification pass is a new and reasonable axis for DLLM acceleration, complementary to temporal caching and to reducing NFE.
  2. The method is training-free and keeps the draft policy unchanged, so it can be plugged into existing multi-branch speculators, and the speedup is measured end-to-end with real kernels.
Critical weaknesses
  1. The method gives up the lossless property of speculative decoding. Speculative verification is useful because accepted drafts follow the target model exactly, but folded drafts reuse parent QKV and FFN outputs, so their cached logits are approximations. The paper does not analyze how this error changes the output.

  2. Limited novelty. Recomputing only the tokens whose hidden states changed is the same idea as adaptive activation caching for DLLMs (e.g., dLLM-Cache), applied across sibling drafts instead of across denoising steps. The contribution is mainly the kernel engineering.

  3. The source of the speedup is unclear. SpecFold runs on its own Triton kernels while the baselines use the standard dense implementation, and on the smaller models a SpecFold call with all drafts is even cheaper than a vanilla call without drafts, which folding alone cannot explain. A dense baseline on the same kernels is missing.

Questions for authors

Please refer to the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.