acceptodds
Under review as a conference paper at ICLR 2027

DepCap: Adaptive Block-Wise Parallel Decoding for Efficient Diffusion LM Inference

Abstract

Diffusion language models (DLMs) offer a promising alternative to autoregressive language generation by enabling parallel decoding and global sequence refinement. To unlock this potential, DLM inference must balance generation quality and decoding speed. Recent block-wise DLM decoding methods improve this trade-off by performing diffusion-based decoding sequentially in blocks. However, existing block-wise methods often rely on signals that are not well aligned with two key decisions in DLM inference. For block partitioning, fixed schedules and current-step local signals do not explicitly capture cross-step prediction changes, while confidence-based parallel decoding does not explicitly consider interactions between token predictions. In this paper, we argue that block-wise DLM inference requires decision-matched signals: cross-step signals for block partitioning and token-level conflict signals for parallel decoding. Based on this view, we propose DepCap, a training-free framework that instantiates the cross-step signal as the influence of the last decoded block and uses it to adaptively determine the next block boundary. Within each block, DepCap combines token confidence with a pairwise conflict score to select tokens for parallel decoding, balancing decoding speed and generation quality. DepCap is plug-and-play across DLMs and compatible with existing block-wise KV-cache strategies. An information-theoretic analysis characterizes the deviation from additive last-block influence, motivating the proposed block-partitioning criterion. Experiments across four DLM backbones and four reasoning and coding benchmarks show favorable quality-speed trade-offs, with up to 5.63 speedup over vanilla decoding.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Language Models, Efficient Inference, Non-Autoregressive Generation, Adaptive Decoding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_sMZZOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes DepCap, a training-free decoding method for diffusion language models that combines adaptive block size (DepGA-Block) with cross-position confidence-aware parallel decoding (CAP-Decoding). DepGA-Block determines block boundaries using changes in predictive distributions and predictive entropy, while CAP-Decoding selects candidates through confidence thresholds and pairwise cross-prediction scores. Experiments on different models and different benchmarks evaluate the accuracy, TPS, and NFEs of DepCap and other baselines.

Critical strengths
  1. The research question about the trade-off between generation quality and decoding speed is important.
Critical weaknesses
  1. The contribution is marginal, and the designs are not intuitive. The authors wrap an essentially simple method, which combines dynamic block size and decoding with cross-position confidence interaction, in complicated concepts and confusing formulations. However, from the efficiency perspective, this extremely complicated design only results in modest and setting-dependent gains over existing parallel-decoding baselines like Swordsman and Fast-dLLM.

  2. The storyline is unclear. The figures are hard to understand. The authors seem to just stack a lot of boldface points, experimental results, and experimental settings, while the analysis does not provide sound support for their claims. For example, in Section 5.2, I do not understand what it means by "DepCap combines naturally with existing block-wise cache techniques", given the fact that after applying cache, Fast-dLLM turns out to outperform DepCap, while it has lower TPS when there is no cache. Besides, experimental results about hyperparameter analysis are located in the appendix, and there are only a few confusing paragraphs in Section 5.4. If the authors really believe this subsection answers an important question, please just place the results there. Overall, the writing of this manuscript is way below high quality.

Questions for authors
  1. What is Throughput in Figure 2? TPS? And what do the marks in Figure 2 left refer to? TPS values?

  2. How is TPS calculated? By one evaluation run or multiple runs? If by one run, is the random seed fixed for all methods? Why are the TPS values under the same configuration in Table 1 and Table 11 different?

5 comments
AuthorsOpenReview ↗

Response to Reviewer sMZZ (1/2)

We thank you for taking the time to review our paper and for your thoughtful feedback. We address your questions and concerns below.

(1) About the contribution and design motivation.

The key distinction is to use how decoded context changes future predictions, rather than only how uncertain those predictions currently are. Predictive entropy describes the current uncertainty, but does not capture the change induced by the last decoded block. DepGA-Block combines these two signals to adapt the next boundary as generation proceeds. Within the selected block, CAP-Decoding extends confidence-based decoding: a lower candidate threshold admits more tokens, while conflict-aware filtering selects which candidates to commit together, balancing parallelism and generation quality. When two positions support each other's predicted tokens, resolving one first may help disambiguate the other. This motivates using the pairwise score as a lightweight proxy for potential conflicts.

These decisions address two different aspects of parallel decoding: how far the next block should extend, and which tokens within it should be committed together. Together, they aim to balance generation quality and decoding speed as predictions evolve. Both use predictive distributions already available during decoding, without retraining or additional model forward passes. Sections 4.1–4.3 of the revised manuscript explain this intuition before introducing the scoring rules and distinguish DepCap from existing adaptive methods.

(2) About the gains over existing adaptive decoding methods.

Our objective is to improve the quality–speed trade-off, rather than maximize either generation quality or throughput in isolation. Table 1 evaluates this trade-off across three backbones and four benchmarks using the same DepCap configuration. DepCap achieves higher observed accuracy and throughput simultaneously in 8 of the 12 settings compared with AdaBlock and 6 compared with Swordsman. These improvements span all three backbones and both mathematical reasoning and code generation, rather than being confined to a single model–task setting. Together, these results support DepCap’s competitive quality–speed trade-offs across settings. Section 5.2 of the revised manuscript makes this cross-setting analysis explicit.

(3) About cache compatibility and Figure 2.

Caching provides additional acceleration for DepCap while retaining its adaptive decoding policy. In Figure 2 of the submitted version, Dual Cache increases DepCap’s throughput. Fast-dLLM is faster with Dual Cache in that comparison, while DepCap is more accurate.

We expand the cache evaluation to two models and two tasks, comparing DepCap, Fast-dLLM, and AdaBlock with No Cache, Prefix Cache, and Dual Cache (Table a). Both Prefix Cache and Dual Cache increase DepCap’s throughput in all four model–task settings. Prefix Cache provides 2.39–3.08× acceleration on GSM8K and 1.31–1.39× on HumanEval. The table reports accuracy alongside throughput for each cache mode.

Table a. Expanded cache comparison. New evaluations on LLaDA-8B-Instruct and Dream-v0-base-7B using NVIDIA A100-SXM4-80GB GPUs (rather than the A100-PCIe-40GB GPUs used in the original paper). Each cell reports accuracy (%) / TPS on the full benchmark. HumanEval accuracy is pass@1.

ModelDatasetMethodNo CachePrefix CacheDual Cache
LLaDAGSM8KFast-dLLM78.24 / 24.3777.63 / 66.1577.63 / 60.31
LLaDAGSM8KAdaBlock78.85 / 22.5179.30 / 52.7379.00 / 50.53
LLaDAGSM8KDepCap78.77 / 25.4278.62 / 60.7077.18 / 56.69
LLaDAHumanEvalFast-dLLM42.68 / 66.0543.90 / 91.4937.20 / 70.95
LLaDAHumanEvalAdaBlock40.85 / 59.0541.46 / 74.6740.24 / 62.55
LLaDAHumanEvalDepCap44.51 / 69.0439.02 / 90.7938.41 / 71.71
DreamGSM8KFast-dLLM74.22 / 16.3773.01 / 51.7773.69 / 51.99
DreamGSM8KAdaBlock74.15 / 16.2673.54 / 50.5873.69 / 49.22
DreamGSM8KDepCap74.68 / 16.9673.84 / 52.2473.92 / 47.91
DreamHumanEvalFast-dLLM53.05 / 47.3853.05 / 68.3148.78 / 60.04
DreamHumanEvalAdaBlock51.83 / 47.3550.61 / 68.4052.44 / 59.20
DreamHumanEvalDepCap50.00 / 51.3248.17 / 71.1947.56 / 63.92

In the submitted Figure 2, “Throughput” denotes TPS, and the left-panel bars and markers show accuracy and TPS, respectively. We replace that figure with Table 2 in the revised manuscript, explicitly reporting accuracy, TPS, and cache speedups relative to each method's No Cache result. Section 5.2 discusses the expanded comparison.

AuthorsOpenReview ↗

Response to Reviewer sMZZ (2/2)

(4) About the hyperparameter analysis.

Section 5.4 now includes the key hyperparameter results in Table 5. We explain how the maximum block size and candidate threshold affect parallelism and the quality–speed trade-off. Appendix H reports the additional hyperparameter results.

We also identify a parameter-forwarding error in the Dream candidate-threshold sweep and remove that panel and its interpretation. The revised threshold sweep in Appendix H.3 (Table 14) reports LLaDA-8B-Instruct results.

(5) About TPS calculation and measurement reliability.

TPS is the total number of generated tokens divided by the measured generation time, including decoding-policy computation but excluding model loading and downstream scoring. The original main-table entries use one evaluation run per configuration, with temperature-zero decoding and fixed NumPy, PyTorch, and few-shot sampling seeds of 1234 across methods (the default settings in lm-eval).

The TPS differences between Tables 1 and 11 in the submitted version reflect independent timing measurements of the same default setting. We checked the original records and confirmed identical generated outputs, accuracy, generated-token counts, and NFE. The revised hyperparameter tables reuse the main-table measurements for their default rows. To assess timing variability, we conduct five repeated evaluations of DepCap and Fast-dLLM, as summarized in Table b.

Table b. Throughput stability over five repetitions. Results use LLaDA-8B-Instruct on NVIDIA A100-SXM4-80GB GPUs, rather than the A100-PCIe-40GB GPUs used in the submission. Entries report mean TPS ± sample standard deviation on the first 512 GSM8K test examples and all 164 HumanEval examples. Fast-dLLM uses its original implementation with fixed block length 32.

MethodGSM8KHumanEval
Fast-dLLM24.12 ± 0.1366.34 ± 0.24
DepCap25.14 ± 0.1369.32 ± 0.16

DepCap is faster in all five repetitions on both workloads, with approximately 4% higher mean throughput and low run-to-run variation. These measurements support the repeatability of the throughput improvement on the evaluated workloads. Section 5.1 and Appendix C clarify the timing definition. The five-repeat comparison also appears in Appendix F.3 (Table 9).

(6) About writing and organization.

Thank you for pointing out these writing problems. We clarify the design intuition and explain what each experimental comparison shows. Specifically:

  • To make the formulation easier to follow, Sections 4.1–4.3 first explain the two decisions: how far to extend the next block and which tokens to decode together. We explain last-block influence, remaining uncertainty, and conflict filtering before introducing their formulas.
  • To clarify the experimental storyline, Sections 5.2–5.3 explain the overall quality–speed results and the roles of the two components, with a conclusion for each comparison.
  • To improve readability, Table 2 replaces the original Figure 2 with clearly labeled accuracy, TPS, and cache speedups. Section 5.2 analyzes the results across two models and two tasks.
  • To support the hyperparameter discussion with results, Section 5.4 now includes Table 5 and explains how the maximum block size and candidate threshold affect decoding behavior.

The corresponding manuscript revisions are marked in blue.

Thank you for providing detailed responses. However, I do not believe the claim that DepCap achieves a better quality-speed trade-off is well supported. As we can observe in Table a, compared with Fast-dLLM, DepCap either trades a non-negligible performance drop for a comparable or marginally higher TPS (and even gets both lower TPS and lower accuracy in some configurations), or gets comparable or lower TPS when the performance remains comparable. The only configuration that shows a better trade-off is LLaDA on HumanEval without cache, but this is far from convincing.

Reviewer_VCMBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DepCap is a training-free framework for adaptive block-wise diffusion decoding. Its block selector combines the prediction change induced by the most recently decoded block with predictive entropy; its within-block decoder combines token confidence with a pairwise cross-support score used as a conflict proxy. An information-theoretic analysis characterizes a correction to additive last-block influence under an idealized joint distribution. Experiments span four diffusion backbones and four reasoning/coding benchmarks, with additional cache, ablation, runtime, and long-generation studies. The paper reports up to 5.63× speedup over vanilla decoding and an average 3.57× throughput speedup over two dense LLaDA backbones and four tasks, with task-dependent quality trade-offs.

Critical strengths
  1. The framework separates two important decisions and supplies inexpensive signals for each. Using cross-step evidence for block boundaries and a pairwise signal for within-block selection is a coherent response to limitations of fixed blocks and independent confidence thresholds. The method requires no model training and can be combined with block-wise caching, making the contribution practically relevant if its quality–speed advantage is sustained (pp. 4–6, 8).
  2. The validation goes beyond a single model or a single headline speedup. Four backbones and multiple mathematics/code tasks, together with component ablations and cache measurements, provide useful evidence of applicability across decoding settings. The runtime profiling also makes the additional policy cost visible. This breadth is a meaningful argument for the framework’s empirical value, even though not every setting improves both quality and speed (pp. 7–9, 16–18, 22).
Critical weaknesses
  1. The novelty of the early-convergence observation is insufficiently established. Learn2PD already identifies redundant decoding after tokens match the final output and trains a lightweight predictor for early commitment. The paper should distinguish its contribution more clearly from this prior work, particularly regarding Observation 2 and LPS, and narrow its repeated “for the first time” claims. See Learn2PD, Sections 3.1.2–3.2.
  2. The LPS training procedure introduces an unresolved training–inference mismatch. During replay, incorrect predictions are replaced with tokens from the completed reference sequence, whereas incorrect commitments remain irreversible during inference. Matching decoding configurations does not eliminate this difference. The paper should evaluate false-positive rates and error accumulation on actual inference trajectories, beyond the oracle-corrected validation traces.
  3. The ablations do not sufficiently establish the necessity of learned block selection. Table 2 primarily compares LBS against random selection, which is a weak control. Stronger comparisons should include entropy- or confidence-based selection with identical filters and comparable block sizes. Moreover, Table 8 shows that EOS removal alone increases throughput from 42.25 to 90.31 TPS, highlighting the need to separate gains from learned routing and handcrafted rules.
  4. The evaluation provides limited support for broad deployment claims. Experiments mainly use batch size 1 and generation lengths of 256 or 512 tokens, without comprehensive speed–accuracy curves or uncertainty estimates. Furthermore, all eight semi-autoregressive configurations in Table 4 show reduced throughput after adding LBS. Longer outputs, batched serving, and carefully tuned baselines are needed to establish the method’s practical acceleration benefits.
Questions for authors

Please refer to critical weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer VCMB

Thank you for your thoughtful feedback. We are puzzled by some aspects of the feedback, such as the references to LPS/LBS, oracle-corrected replay, and EOS-removal experiments, as these are not part of our work. We have identified several evaluation suggestions that remain relevant to our work and address them below.

(1) About stronger baselines.

Our main experiments already compare DepCap with AdaBlock and Swordsman, both of which use adaptive block partitioning. The results in Table 1 demonstrate DepCap's competitive quality–speed trade-offs against these adaptive decoding methods.

To further examine the quality–speed trade-off, we vary Fast-dLLM's confidence threshold while keeping the block length fixed at 32. Table a compares these settings with default DepCap on GSM8K and HumanEval.

Table a. Fast-dLLM confidence-threshold sweep versus DepCap. Results use LLaDA-8B-Instruct on NVIDIA A100-SXM4-80GB GPUs (rather than the A100-PCIe-40GB GPUs used for the original measurements). Each cell reports accuracy (%) / TPS on the full test set. HumanEval accuracy is pass@1. The threshold sweep implements Fast-dLLM's confidence-based rule in the same evaluation code as CAP-Decoding.

MethodBlock policyThresholdGSM8KHumanEval
Fast-dLLMFixed 320.7577.18 / 35.4237.20 / 89.66
Fast-dLLMFixed 320.8078.09 / 31.5939.63 / 83.04
Fast-dLLMFixed 320.8577.86 / 27.9440.24 / 75.48
Fast-dLLMFixed 320.9078.24 / 24.4742.68 / 67.06
DepCapAdaptiveDefault78.77 / 25.4244.51 / 69.04

On these two benchmarks, DepCap achieves higher observed accuracy at comparable throughput to Fast-dLLM at 0.90. Lower confidence thresholds increase throughput but do not reach DepCap's observed accuracy in the tested sweep. This illustrates DepCap's objective of balancing generation quality and decoding speed through adaptive decoding. Section 5.4 (Table 4) reports this threshold sweep, and Appendix H.3 discusses the selection-policy comparisons.

(2) About statistical reliability.

To assess timing variability, we conduct five repeated evaluations of DepCap and Fast-dLLM. DepCap is faster in all five repetitions on both workloads, with approximately 4% higher mean throughput and low run-to-run variation.

Table b. Throughput stability over five repetitions. Results use LLaDA-8B-Instruct on NVIDIA A100-SXM4-80GB GPUs. Entries report mean TPS ± sample standard deviation on the first 512 GSM8K test examples and all 164 HumanEval examples. Fast-dLLM uses its original implementation with fixed block length 32.

MethodGSM8KHumanEval
Fast-dLLM24.12 ± 0.1366.34 ± 0.24
DepCap25.14 ± 0.1369.32 ± 0.16

Our focus is an improved quality–speed trade-off through adaptive decoding. On LLaDA-8B-Instruct/GSM8K in Table 1, fixed-block Fast-dLLM at lengths 16, 32, and 64 achieves 78.3%, 77.3%, and 77.5% accuracy at 18.3, 19.6, and 20.1 TPS. DepCap achieves 78.8% at 21.4 TPS, improving both observed metrics over all three settings. The corresponding paired bootstrap 95% confidence intervals for accuracy differences are [−1.36, 2.27], [−0.23, 3.26], and [−0.38, 2.96] percentage points. These intervals include zero, so the evidence supports a favorable observed trade-off, not statistically significant accuracy gains. Appendix F.4 of the revised manuscript reports the accuracy uncertainty. The five-repeat comparison also appears in Appendix F.3 (Table 9).

(3) About longer generation and batched inference.

Our evaluation already includes generation lengths up to 1,024 tokens. Appendix H.1 (Table 12) of the revised manuscript compares DepCap with Fast-dLLM, AdaBlock, and Swordsman on Math-500 and HumanEval at this length. On HumanEval, DepCap achieves the highest throughput among these methods at 19.174 TPS, matching AdaBlock's 44.51% pass@1. On Math-500, it provides an intermediate quality–speed trade-off between Fast-dLLM and AdaBlock, with higher accuracy than the former and higher throughput than the latter. Our current focus is per-request decoding efficiency at batch size 1, with batched inference left for future work.

AuthorsOpenReview ↗

Kind Reminder Regarding the Rebuttal Discussion

Dear Reviewer VCMB,

This is a kind reminder that the discussion phase will end soon. If you have any additional questions or concerns, please feel free to let us know. If you find that our response has addressed your concerns, we would greatly appreciate your consideration in updating your assessment.

Thank you again for your time and valuable feedback.

Best,

The Authors

Reviewer_dbZ8OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DepCap, a training-free decoding policy for block-wise masked diffusion language models. It frames inference as "decision-signal matching": each per-step decision should use the signal it naturally requires. DepCap makes two decisions with two components:

  • DepGA-Block decides where the next block ends. It uses the change in predictions at future positions after the previous block is decoded (a cross-step KL signal, termed last-block influence), traded off against predictive entropy.
  • CAP-Decoding decides which masked tokens to commit in parallel inside the block. It combines confidence with a pairwise conflict score computed from cross-position prediction probabilities, and selects tokens greedily so that no conflicting pair is committed together.
Critical strengths

S1. Practicality and ease of integration.

  • The method is training-free and can be integrated into existing block-wise diffusion LM decoding frameworks.
  • Its own operations add minimal overhead: about 0.88 ms per iteration, or roughly 0.57% of measured GPU time.
  • It is compatible with prefix and dual KV caching.
  • Why it matters: the training-free, plug-and-play design makes the method practical for existing inference systems, with no additional training or substantial architectural changes. Its lightweight implementation also allows it to be combined with complementary acceleration techniques such as KV-cache optimization, which further improves efficiency. Together, these properties improve the method's practicality, flexibility and potential for real-world deployment.

S2. A new perspective on block-wise diffusion inference, with theoretical motivation.

  • The paper offers an interesting view of block-wise diffusion LM inference through decision-signal matching. Cross-step signals guide block partitioning, and token-level conflict signals guide parallel decoding.
  • It provides an information-theoretic analysis of the dependency score and interprets the influence of the most recently decoded block.
  • Why it matters: this perspective offers an alternative way to understand and analyze block-wise diffusion LM inference, and may open new directions for future research. The theoretical analysis also helps motivate the approach and justify its design choices. The authors acknowledge that the analysis is only loosely tied to the implemented decision rule, but it still provides valuable conceptual insight and a promising foundation for further investigation.
Critical weaknesses

W1. Limited improvement over existing confidence-based decoding.

  • The large speedups (3.57× on average, up to 5.63×) are measured against one-token-per-step vanilla decoding.
  • Against Fast-dLLM confidence decoding (|B₀| = 32), the improvement is marginal:
    • DepCap's throughput averages about 1.03× that of Fast-dLLM, ranging from 0.92× to 1.09× across the 12 model–task settings (Table 1).
    • In the LLaDA-1.5/MBPP column, where DepCap reports its 5.63× speedup, Fast-dLLM itself reaches 6.11× over vanilla decoding.
    • DepCap is more accurate than Fast-dLLM in 9 of the 12 settings and less accurate in 3.
  • Why it matters: the limited gains over a strong existing baseline weaken the practical significance of the method. The paper offers an interesting perspective, but the modest empirical improvements make it difficult to establish a compelling advantage over existing techniques.

W2. Lack of statistical reliability measures.

  • No variability measures. The evaluation reports no confidence intervals, standard deviations or other measures of statistical variability.
  • Benchmark sampling is the relevant uncertainty. Decoding is deterministic (temperature zero), so accuracy differences of one to two points need paired tests or confidence intervals.
  • Throughput varies between runs by about as much as the claimed gain. Configurations that are identical across tables, with matching accuracy and NFE, show throughput differences of up to about 4%. For example, LLaDA-8B gives 22.2 vs. 21.574 TPS on Math-500 and 59.8 vs. 57.147 TPS on HumanEval (Table 1 vs. Table 11). This is comparable to the 3% average gain over Fast-dLLM.
  • Limited scope. KV-cache compatibility is shown for a single model and task. All runs use batch size 1, and batched inference is left to future work.
  • Why it matters: without measures of variability, it is difficult to tell whether the observed improvements are statistically meaningful or due to run-to-run fluctuation. This concern is particularly important when the throughput improvement over Fast-dLLM is only about 3%. Results across multiple runs, with confidence intervals, would provide stronger evidence that the improvements are robust and significant.

Minor remark on motivation. Apart from the theoretical analysis of the dependency score, the motivation for the specific design choices could be stated more directly in the main text, in particular the cross-step KL signal for block boundaries and the pairwise conflict score.

Overall. Despite these weaknesses, I consider the paper technically sound and the problem of efficient inference for diffusion language models highly relevant and practically important. The weaknesses temper, but do not outweigh, the method's practicality and the value of its perspective, which is why I lean toward acceptance.

Questions for authors
  1. How much does DepCap gain over Fast-dLLM at matched accuracy or matched thresholds? For example, how does confidence-only decoding at τ = 0.8 compare with CAP-Decoding? This would isolate the effect of conflict filtering.
  2. What is the distribution of the block sizes chosen by DepGA-Block? How does a fixed block size equal to the mean adaptive size perform?
  3. Can the authors report paired significance tests or confidence intervals, and the run-to-run variability of the throughput measurements?
6 comments
AuthorsOpenReview ↗

Response to Reviewer dbZ8 (1/3)

We thank you for taking the time to review our paper and for your thoughtful feedback. We address your questions and concerns below.

(1) About the comparison with Fast-dLLM (W1). Unlike fixed-block Fast-dLLM, DepCap adapts the next block boundary during decoding using the influence of the last decoded block on predictions at future positions and the remaining uncertainty in those predictions. The motivation is that a fixed block length cannot account for changes in the model’s predictions as generation proceeds, and adapting the boundary lets us explore a better balance between generation quality and decoding speed.

Table 1 shows why the comparison should not be limited to a block size of 32. On LLaDA-8B-Instruct/GSM8K, 16-token Fast-dLLM is more accurate than the 32-token setting, while the 64-token setting is both faster and more accurate. DepCap achieves 78.8% accuracy at 21.4 TPS, exceeding all three fixed-block settings on both metrics in this case. The trade-off is different on LLaDA-1.5/MBPP, as you noted: 32-token Fast-dLLM is faster (11.6 vs. 10.7 TPS), while DepCap is more accurate (40.6% vs. 37.6%). Across the 12 settings in Table 1, DepCap has higher observed accuracy in 9 and averages 1.03× the throughput of 32-token Fast-dLLM. We clarify this quality–speed comparison in Section 5.2 of the revised manuscript. Our response (3) examines the contribution of adaptive boundaries separately.

(2) About confidence thresholds and the quality–speed trade-off (Q1).

DepCap offers higher observed accuracy at comparable throughput to Fast-dLLM at a confidence threshold of 0.90 on GSM8K and HumanEval. To examine whether adjusting the confidence threshold can achieve a similar trade-off, Table a compares Fast-dLLM across four thresholds with default DepCap. We also include fixed-32 CAP-Decoding at its default setting to compare the selection policies while keeping the block partition unchanged.

Table a. Fast-dLLM, fixed-block CAP-Decoding, and full DepCap. The table reports results on LLaDA-8B-Instruct using NVIDIA A100-SXM4-80GB GPUs (rather than the A100-PCIe-40GB GPUs used in the original submission). Each cell reports accuracy (%) / TPS on the full GSM8K test set (1,319 examples) and HumanEval (164 examples). HumanEval accuracy is postprocessed pass@1. The threshold sweep implements Fast-dLLM's confidence-based rule in the same evaluation code as CAP-Decoding.

MethodBlock policyThresholdGSM8KHumanEval
Fast-dLLMFixed 320.7577.18 / 35.4237.20 / 89.66
Fast-dLLMFixed 320.8078.09 / 31.5939.63 / 83.04
Fast-dLLMFixed 320.8577.86 / 27.9440.24 / 75.48
Fast-dLLMFixed 320.9078.24 / 24.4742.68 / 67.06
CAP-DecodingFixed 32Default77.79 / 26.8342.07 / 70.84
DepCapAdaptiveDefault78.77 / 25.4244.51 / 69.04

CAP-Decoding extends confidence-based decoding by separating candidate admission from parallel selection. A lower candidate threshold admits more tokens, while conflict-aware filtering selects which candidates to commit together, balancing parallelism and generation quality. On HumanEval with fixed 32-token blocks, CAP-Decoding increases pass@1 from 39.63% to 42.07% relative to Fast-dLLM at 0.80, with the corresponding throughput trade-off. Combined with adaptive partitioning, full DepCap reaches 44.51% at comparable throughput to Fast-dLLM at 0.90. These comparisons appear in Section 5.4 (Table 4) of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer dbZ8 (2/3)

(3) About adaptive block lengths and fixed-block controls (Q2).

DepGA-Block selects both longer and shorter blocks as predictions evolve, rather than simply using a smaller fixed block size. Table b shows that although average block lengths are below 32, approximately 29%–32% of interior blocks exceed 32 tokens on GSM8K and HumanEval. The case study in Appendix E (Figure 2) illustrates this variation within a single generation, with blocks of 51 and 44 tokens followed by smaller blocks of 21 and 16 tokens.

Table b. Block-length statistics for DepCap. Results use LLaDA-8B-Instruct on NVIDIA A100-SXM4-80GB GPUs across the full test sets. Mean length includes all blocks. Interior-block statistics exclude the initial block and the final block, whose length can be truncated by the generation budget.

DatasetMean block lengthInterior medianInterior rangeInterior blocks >32
GSM8K21.03178–9229.1%
HumanEval22.10188–11532.1%

To test whether smaller fixed blocks can achieve a similar trade-off, we evaluate fixed lengths of 21 and 23, close to the observed adaptive means, while retaining the same CAP-Decoding policy and 8-token initial block.

Table c. Fixed-block controls versus adaptive partitioning. Results use LLaDA-8B-Instruct on NVIDIA A100-SXM4-80GB GPUs. Each cell reports accuracy (%) / TPS on the full test set. HumanEval accuracy is pass@1.

Block policyGSM8KHumanEval
Fixed 21 + CAP-Decoding78.62 / 25.3142.68 / 68.06
Fixed 23 + CAP-Decoding79.15 / 25.3444.51 / 68.61
DepGA-Block + CAP-Decoding (DepCap)78.77 / 25.4244.51 / 69.04

DepGA-Block selects block boundaries online without prescribing a fixed block length for the entire generation. The fixed lengths of 21 and 23 are chosen to approximate the mean block lengths observed under DepCap. Their comparable performance shows that these lengths provide effective operating points, which DepCap reaches without prescribing either length in advance. Meanwhile, its individual blocks vary substantially, including both shorter and longer blocks, as illustrated by the distributions and the case study. The distributions and fixed-block comparisons appear in Section 5.3 (Table 3) of the revised manuscript.

(4) About the TPS differences between tables and statistical reliability (W2, Q3).

The TPS differences between Tables 1 and 11 in the submitted version reflect independent timing measurements of the same default setting. We checked the original records and confirmed identical generated outputs, accuracy, total generated-token counts, and NFE. The revised hyperparameter tables reuse the main-table measurements for their default rows. To assess timing variability, we conduct five repeated evaluations of DepCap and Fast-dLLM. DepCap is faster in all five repetitions on both workloads, with approximately 4% higher mean throughput and low run-to-run variation.

Table d. Throughput stability over five repetitions. Results use LLaDA-8B-Instruct on NVIDIA A100-SXM4-80GB GPUs. Entries report mean TPS ± sample standard deviation on the first 512 GSM8K test examples and all 164 HumanEval examples. Fast-dLLM uses its original implementation with fixed block length 32.

MethodGSM8KHumanEval
Fast-dLLM24.12 ± 0.1366.34 ± 0.24
DepCap25.14 ± 0.1369.32 ± 0.16

Our focus is an improved quality–speed trade-off through adaptive decoding. On LLaDA-8B-Instruct/GSM8K in Table 1, fixed-block Fast-dLLM at lengths 16, 32, and 64 achieves 78.3%, 77.3%, and 77.5% accuracy at 18.3, 19.6, and 20.1 TPS. DepCap achieves 78.8% at 21.4 TPS, improving both observed metrics over all three settings. The corresponding paired bootstrap 95% confidence intervals for accuracy differences are [−1.36, 2.27], [−0.23, 3.26], and [−0.38, 2.96] percentage points. These intervals include zero, so the evidence supports a favorable observed trade-off, not statistically significant accuracy gains. Appendix F.4 of the revised manuscript reports the accuracy uncertainty. The five-repeat comparison also appears in Appendix F.3 (Table 9).

AuthorsOpenReview ↗

Response to Reviewer dbZ8 (3/3)

(5) About the motivation for the predictive signals.

The motivation is to decide how far to decode and which tokens to decode together. For block boundaries, we ask whether newly decoded context provides enough guidance to extend the next block, or whether the remaining uncertainty calls for a shorter block. Cross-step KL and entropy capture these two considerations. For parallel decoding, CAP-Decoding extends confidence-based selection: a lower candidate threshold admits more tokens, while conflict-aware filtering determines which candidates to commit together. When two positions support each other's predicted tokens, resolving one first may help disambiguate the other. This motivates using the pairwise score as a lightweight proxy for potential conflicts. Sections 4.1–4.3 of the revised manuscript explain these motivations alongside the corresponding formulas.

(6) About cache compatibility across models and tasks.

Caching provides additional acceleration for DepCap while retaining its adaptive decoding policy. We extend the cache evaluation to LLaDA-8B-Instruct and Dream-v0-base-7B on GSM8K and HumanEval, comparing DepCap, Fast-dLLM, and AdaBlock under No Cache, Prefix Cache, and Dual Cache.

Both Prefix Cache and Dual Cache increase DepCap’s throughput in all four model–task settings (Table e). Prefix Cache provides 2.39–3.08× acceleration on GSM8K and 1.31–1.39× on HumanEval. The table reports accuracy alongside throughput for each cache mode. Section 5.2 (Table 2) and Appendix G (Table 10) of the revised manuscript include these comparisons.

Table e. Cache compatibility across models and tasks. New evaluations on LLaDA-8B-Instruct and Dream-v0-base-7B using NVIDIA A100-SXM4-80GB GPUs (rather than the A100-PCIe-40GB GPUs used in the original paper). Each cell reports accuracy (%) / TPS on the full benchmark. HumanEval accuracy is pass@1.

ModelDatasetMethodNo CachePrefix CacheDual Cache
LLaDAGSM8KFast-dLLM78.24 / 24.3777.63 / 66.1577.63 / 60.31
LLaDAGSM8KAdaBlock78.85 / 22.5179.30 / 52.7379.00 / 50.53
LLaDAGSM8KDepCap78.77 / 25.4278.62 / 60.7077.18 / 56.69
LLaDAHumanEvalFast-dLLM42.68 / 66.0543.90 / 91.4937.20 / 70.95
LLaDAHumanEvalAdaBlock40.85 / 59.0541.46 / 74.6740.24 / 62.55
LLaDAHumanEvalDepCap44.51 / 69.0439.02 / 90.7938.41 / 71.71
DreamGSM8KFast-dLLM74.22 / 16.3773.01 / 51.7773.69 / 51.99
DreamGSM8KAdaBlock74.15 / 16.2673.54 / 50.5873.69 / 49.22
DreamGSM8KDepCap74.68 / 16.9673.84 / 52.2473.92 / 47.91
DreamHumanEvalFast-dLLM53.05 / 47.3853.05 / 68.3148.78 / 60.04
DreamHumanEvalAdaBlock51.83 / 47.3550.61 / 68.4052.44 / 59.20
DreamHumanEvalDepCap50.00 / 51.3248.17 / 71.1947.56 / 63.92

(7) About batched inference.

Our current evaluation focuses on per-request decoding efficiency at batch size 1. Extending adaptive block partitioning to efficient batched inference is a direction for future work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Responses and Manuscript Revisions

We thank the reviewers for their careful assessment and constructive feedback.

DepCap adapts block boundaries and within-block token selection to balance generation quality and decoding speed, without additional training or model forward passes. We appreciate the reviewers’ recognition of the importance of the quality–speed trade-off (dbZ8, sMZZ), practical, training-free design and cache compatibility (dbZ8, VCMB), coherent block and token selection design (dbZ8, VCMB), and evaluation across multiple backbones and tasks (VCMB).

Our responses and revised manuscript address the main concerns in four areas:

  • Contribution and adaptive block selection. We clarify how prediction changes induced by the last decoded block guide boundary selection, distinguishing DepCap from fixed-block decoding and existing adaptive methods based on current-step signals. We add block-length distributions and fixed-length controls on GSM8K and HumanEval. The control lengths of 21 and 23 approximate DepCap’s observed means and achieve comparable trade-offs, while DepCap selects boundaries online without prescribing either length throughout generation.

  • Quality–speed comparisons. We complement the existing AdaBlock and Swordsman comparisons with a Fast-dLLM confidence-threshold sweep and fixed-block CAP-Decoding results, assessing generation quality and decoding speed jointly. We also clarify how CAP-Decoding combines lower-threshold candidate admission with conflict-aware filtering.

  • Measurement reliability. We clarify the TPS definition and the independent timing measurements behind the original cross-table differences. Five repeated evaluations on the first 512 GSM8K test examples and all 164 HumanEval examples show approximately 4% higher mean throughput for DepCap, with low run-to-run variation. We also provide paired accuracy confidence intervals to distinguish observed improvements from statistical uncertainty.

  • Cache compatibility and presentation. We expand the cache evaluation to LLaDA and Dream on GSM8K and HumanEval, including Fast-dLLM and AdaBlock. Both cache modes further accelerate DepCap across all four settings, with accuracy reported alongside throughput to characterize the resulting trade-offs. The revised manuscript also presents the design intuition more directly, replaces the cache figure with an explicit comparison table, and brings key hyperparameter results into the main text.

The revised manuscript marks changes in blue, and we provide detailed responses to each reviewer’s concerns. We hope these clarifications and additional results improve the manuscript’s clarity and readability and strengthen the empirical support for DepCap’s contribution. Please let us know if you have any further questions.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.