- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes DepCap, a training-free decoding method for diffusion language models that combines adaptive block size (DepGA-Block) with cross-position confidence-aware parallel decoding (CAP-Decoding). DepGA-Block determines block boundaries using changes in predictive distributions and predictive entropy, while CAP-Decoding selects candidates through confidence thresholds and pairwise cross-prediction scores. Experiments on different models and different benchmarks evaluate the accuracy, TPS, and NFEs of DepCap and other baselines.
Critical strengths
- The research question about the trade-off between generation quality and decoding speed is important.
Critical weaknesses
-
The contribution is marginal, and the designs are not intuitive. The authors wrap an essentially simple method, which combines dynamic block size and decoding with cross-position confidence interaction, in complicated concepts and confusing formulations. However, from the efficiency perspective, this extremely complicated design only results in modest and setting-dependent gains over existing parallel-decoding baselines like Swordsman and Fast-dLLM.
-
The storyline is unclear. The figures are hard to understand. The authors seem to just stack a lot of boldface points, experimental results, and experimental settings, while the analysis does not provide sound support for their claims. For example, in Section 5.2, I do not understand what it means by "DepCap combines naturally with existing block-wise cache techniques", given the fact that after applying cache, Fast-dLLM turns out to outperform DepCap, while it has lower TPS when there is no cache. Besides, experimental results about hyperparameter analysis are located in the appendix, and there are only a few confusing paragraphs in Section 5.4. If the authors really believe this subsection answers an important question, please just place the results there. Overall, the writing of this manuscript is way below high quality.
Questions for authors
-
What is Throughput in Figure 2? TPS? And what do the marks in Figure 2 left refer to? TPS values?
-
How is TPS calculated? By one evaluation run or multiple runs? If by one run, is the random seed fixed for all methods? Why are the TPS values under the same configuration in Table 1 and Table 11 different?
5 comments
Response to Reviewer sMZZ (2/2)
(4) About the hyperparameter analysis.
Section 5.4 now includes the key hyperparameter results in Table 5. We explain how the maximum block size and candidate threshold affect parallelism and the quality–speed trade-off. Appendix H reports the additional hyperparameter results.
We also identify a parameter-forwarding error in the Dream candidate-threshold sweep and remove that panel and its interpretation. The revised threshold sweep in Appendix H.3 (Table 14) reports LLaDA-8B-Instruct results.
(5) About TPS calculation and measurement reliability.
TPS is the total number of generated tokens divided by the measured generation time, including decoding-policy computation but excluding model loading and downstream scoring. The original main-table entries use one evaluation run per configuration, with temperature-zero decoding and fixed NumPy, PyTorch, and few-shot sampling seeds of 1234 across methods (the default settings in lm-eval).
The TPS differences between Tables 1 and 11 in the submitted version reflect independent timing measurements of the same default setting. We checked the original records and confirmed identical generated outputs, accuracy, generated-token counts, and NFE. The revised hyperparameter tables reuse the main-table measurements for their default rows. To assess timing variability, we conduct five repeated evaluations of DepCap and Fast-dLLM, as summarized in Table b.
Table b. Throughput stability over five repetitions. Results use LLaDA-8B-Instruct on NVIDIA A100-SXM4-80GB GPUs, rather than the A100-PCIe-40GB GPUs used in the submission. Entries report mean TPS ± sample standard deviation on the first 512 GSM8K test examples and all 164 HumanEval examples. Fast-dLLM uses its original implementation with fixed block length 32.
| Method | GSM8K | HumanEval |
|---|---|---|
| Fast-dLLM | 24.12 ± 0.13 | 66.34 ± 0.24 |
| DepCap | 25.14 ± 0.13 | 69.32 ± 0.16 |
DepCap is faster in all five repetitions on both workloads, with approximately 4% higher mean throughput and low run-to-run variation. These measurements support the repeatability of the throughput improvement on the evaluated workloads. Section 5.1 and Appendix C clarify the timing definition. The five-repeat comparison also appears in Appendix F.3 (Table 9).
(6) About writing and organization.
Thank you for pointing out these writing problems. We clarify the design intuition and explain what each experimental comparison shows. Specifically:
- To make the formulation easier to follow, Sections 4.1–4.3 first explain the two decisions: how far to extend the next block and which tokens to decode together. We explain last-block influence, remaining uncertainty, and conflict filtering before introducing their formulas.
- To clarify the experimental storyline, Sections 5.2–5.3 explain the overall quality–speed results and the roles of the two components, with a conclusion for each comparison.
- To improve readability, Table 2 replaces the original Figure 2 with clearly labeled accuracy, TPS, and cache speedups. Section 5.2 analyzes the results across two models and two tasks.
- To support the hyperparameter discussion with results, Section 5.4 now includes Table 5 and explains how the maximum block size and candidate threshold affect decoding behavior.
The corresponding manuscript revisions are marked in blue.
Thank you for providing detailed responses. However, I do not believe the claim that DepCap achieves a better quality-speed trade-off is well supported. As we can observe in Table a, compared with Fast-dLLM, DepCap either trades a non-negligible performance drop for a comparable or marginally higher TPS (and even gets both lower TPS and lower accuracy in some configurations), or gets comparable or lower TPS when the performance remains comparable. The only configuration that shows a better trade-off is LLaDA on HumanEval without cache, but this is far from convincing.
Response to Reviewer sMZZ (1/2)
We thank you for taking the time to review our paper and for your thoughtful feedback. We address your questions and concerns below.
(1) About the contribution and design motivation.
The key distinction is to use how decoded context changes future predictions, rather than only how uncertain those predictions currently are. Predictive entropy describes the current uncertainty, but does not capture the change induced by the last decoded block. DepGA-Block combines these two signals to adapt the next boundary as generation proceeds. Within the selected block, CAP-Decoding extends confidence-based decoding: a lower candidate threshold admits more tokens, while conflict-aware filtering selects which candidates to commit together, balancing parallelism and generation quality. When two positions support each other's predicted tokens, resolving one first may help disambiguate the other. This motivates using the pairwise score as a lightweight proxy for potential conflicts.
These decisions address two different aspects of parallel decoding: how far the next block should extend, and which tokens within it should be committed together. Together, they aim to balance generation quality and decoding speed as predictions evolve. Both use predictive distributions already available during decoding, without retraining or additional model forward passes. Sections 4.1–4.3 of the revised manuscript explain this intuition before introducing the scoring rules and distinguish DepCap from existing adaptive methods.
(2) About the gains over existing adaptive decoding methods.
Our objective is to improve the quality–speed trade-off, rather than maximize either generation quality or throughput in isolation. Table 1 evaluates this trade-off across three backbones and four benchmarks using the same DepCap configuration. DepCap achieves higher observed accuracy and throughput simultaneously in 8 of the 12 settings compared with AdaBlock and 6 compared with Swordsman. These improvements span all three backbones and both mathematical reasoning and code generation, rather than being confined to a single model–task setting. Together, these results support DepCap’s competitive quality–speed trade-offs across settings. Section 5.2 of the revised manuscript makes this cross-setting analysis explicit.
(3) About cache compatibility and Figure 2.
Caching provides additional acceleration for DepCap while retaining its adaptive decoding policy. In Figure 2 of the submitted version, Dual Cache increases DepCap’s throughput. Fast-dLLM is faster with Dual Cache in that comparison, while DepCap is more accurate.
We expand the cache evaluation to two models and two tasks, comparing DepCap, Fast-dLLM, and AdaBlock with No Cache, Prefix Cache, and Dual Cache (Table a). Both Prefix Cache and Dual Cache increase DepCap’s throughput in all four model–task settings. Prefix Cache provides 2.39–3.08× acceleration on GSM8K and 1.31–1.39× on HumanEval. The table reports accuracy alongside throughput for each cache mode.
Table a. Expanded cache comparison. New evaluations on LLaDA-8B-Instruct and Dream-v0-base-7B using NVIDIA A100-SXM4-80GB GPUs (rather than the A100-PCIe-40GB GPUs used in the original paper). Each cell reports accuracy (%) / TPS on the full benchmark. HumanEval accuracy is pass@1.
In the submitted Figure 2, “Throughput” denotes TPS, and the left-panel bars and markers show accuracy and TPS, respectively. We replace that figure with Table 2 in the revised manuscript, explicitly reporting accuracy, TPS, and cache speedups relative to each method's No Cache result. Section 5.2 discusses the expanded comparison.