acceptodds
Under review as a conference paper at ICLR 2027

Beyond Global Thresholds: Token-Level Calibrated Adaptive Parallel Decoding for Diffusion Language Models

Abstract

Diffusion language models (DLMs) are a promising alternative to autoregressive models because they can generate multiple tokens in parallel. However, they are still slow in practice since they need many denoising steps. Recent methods speed up inference by using a fixed confidence threshold to decide which tokens can be decoded in parallel. These methods assume that confidence has the same meaning for all tokens, which often does not hold. In this work, we show that different tokens have different reliability even under the same confidence level, which reveals a systematic token-level miscalibration problem in confidence-aware decoding. We also find that confidence is related to final correctness, but the relationship varies across tokens. Based on these observations, we propose Token-Level Calibrated Adaptive Parallel Decoding (TC-APD), which estimates a separate threshold for each token from offline calibration data. During inference, we use these thresholds to guide adaptive parallel decoding without adding extra model components. Experiments on mathematical reasoning and code generation tasks show that our method improves the speed-quality trade-off over fixed-threshold baselines, while keeping generation quality. Overall, we demonstrate the effectiveness and potential of calibrating confidence thresholds at the token level for accelerating diffusion language models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
diffusion language model, DLM, acceleration

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4QWrOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies confidence-based adaptive parallel decoding for diffusion language models. Existing methods commonly commit all positions whose top-1 confidence exceeds a shared global threshold. The authors show that this policy overlooks systematic token-level differences: two predicted tokens with similar confidence can have substantially different probabilities of matching the token eventually produced by conservative decoding. Based on this observation, the paper proposes Token-Level Calibrated Adaptive Parallel Decoding (TC-APD). Offline denoising traces are used to collect, for each predicted token vv, pairs of confidence and self-final-consistency labels. A token-specific threshold τv\tau_v is then chosen as the lowest threshold whose empirical final-consistency rate reaches a target ρ\rho, subject to minimum-support constraints. During inference, a prediction of token vv is committed whenever its confidence exceeds τv\tau_v. Unsupported tokens fall back to a default global threshold. Experiments on LLaDA-8B-Instruct and Dream-7B-Base, using GSM8K, MATH500, and HumanEval, show that TC-APD generally improves the accuracy-efficiency trade-off over fixed-threshold APD. For example, on LLaDA with a maximum threshold of 0.900.90, it reduces average NFE from 94.7 to 68.6 while changing average accuracy from 47.7 to 48.0. The paper also reports wall-clock gains and an integration with SOAR.

Critical strengths
  • The paper isolates a plausible and empirically well-supported failure mode of global confidence thresholds.
  • TC-APD has negligible online overhead: inference requires only a token-ID lookup and introduces no extra model calls.
  • The comparison against fixed-threshold APD is conducted over multiple threshold settings rather than at only one operating point.
  • On LLaDA at τmax⁡=0.90\tau_{\max}=0.90, TC-APD improves average NFE from 94.7 to 68.6 without reducing average accuracy.
  • The Dream results suggest that the benefit is not limited to one backbone.
  • Wall-clock measurements show that the reduction in NFE produces practical throughput gains.
  • The method can be incorporated into an existing decoder such as SOAR with a small modification.
  • The appendix candidly reports that accelerated decoding often follows a different trajectory from standard decoding.
Critical weaknesses
  • Possible test-informed calibration selection. The paper compares calibration sources on the same benchmarks used for final evaluation and then chooses the source with the best observed trade-off. This can bias the reported results, even when calibration examples do not overlap with test examples.
  • Mismatch between calibration and deployment trajectories. Self-final consistency is estimated under standard decoding, but early commitment changes later model states. The low sequence agreement reported in Appendix D confirms that this distribution shift is substantial.
  • No out-of-sample calibration assessment. The paper does not report whether predictions accepted at a nominal target of ρ=0.995\rho=0.995 actually achieve a similar consistency rate on independent traces, especially during accelerated decoding.
  • Potential threshold-selection bias. Selecting the minimum τ\tau that satisfies Rv(τ)≥ρR_v(\tau)\geq\rho on the same samples used to calculate RvR_v can produce optimistic estimates. The paper does not use confidence bounds, isotonic calibration, or a separate threshold-selection split.
  • Limited support at a stringent target. With Amin⁡=100A_{\min}=100, an empirical target of 0.9950.995 may be supported by only 100 apparently successful observations. This provides weak statistical evidence for such a high reliability level.
  • Limited experimental scope. Results cover two models, three benchmarks, a fixed block size, and a maximum generation length of 256. It remains unclear how thresholds behave for longer generation, other languages, or less structured tasks.
  • Non-negligible offline cost. A single-domain calibration requires about 3.5 hours on four A100 GPUs, and recalibration may be necessary after changing the model, tokenizer, or decoding setup.
Questions for authors
  1. How exactly are candidate values of τ\tau enumerated, and is monotonicity of Rv(τ)R_v(\tau) enforced? Since empirical conditional accuracy need not be monotonic at finite sample sizes, how sensitive are the selected thresholds to sampling noise?
  2. Have you measured the realized consistency rate of accepted tokens on an independent calibration test set? More importantly, what is that rate on the trajectories generated by TC-APD itself, rather than on standard-decoding traces?
  3. Were the choices of Minerva-Math calibration for LLaDA and mixed-domain calibration for Dream made before evaluating GSM8K, MATH500, and HumanEval? If not, could you report results using a calibration-source selection protocol based entirely on separate validation data?
  4. Why is Amin⁡=100A_{\min}=100 sufficient for a target of ρ=0.995\rho=0.995? Could you report results using lower confidence bounds, such as a binomial confidence bound, instead of raw empirical consistency?
  5. How much of the improvement remains when fixed APD and TC-APD are compared at tightly matched NFE or wall-clock budgets, with paired uncertainty estimates for both task accuracy and efficiency?
  6. Which changes require recalibration? In particular, do the thresholds transfer across block sizes, generation lengths, prompting templates, quantization settings, or fine-tuned variants of the same backbone?
  7. Can the authors report the amortization point for the offline calibration cost, measured in the approximate number of generated sequences required before TC-APD provides a net compute saving?
9 comments
AuthorsOpenReview ↗

Response to Reviewer 4QWr [1/6]

We thank the reviewer 4QWr for the constructive feedback and positive assessment of TC-APD. We address the concerns with additional experiments and analyses on calibration robustness, threshold reliability, cross-configuration transfer, and calibration-cost amortization. We would like to clarify that some additional experiments were conducted under different GPU hardwares, which may explain minor variations in baseline results. However, all methods within each table were evaluated under consistent settings to ensure fair comparisons. We will clarify this in the revised paper.

W1 & Q3 — Calibration source selection and cross-domain evaluation

Weakness: Possible test-informed calibration selection. The paper compares calibration sources on the same benchmarks used for final evaluation and then chooses the source with the best observed trade-off. This can bias the reported results, even when calibration examples do not overlap with test examples.

Question: Were the choices of Minerva-Math calibration for LLaDA and mixed-domain calibration for Dream made before evaluating GSM8K, MATH500, and HumanEval? If not, could you report results using a calibration-source selection protocol based entirely on separate validation data?

We thank the reviewer for highlighting the importance of separating calibration-source selection from final evaluation. Following the reviewer's suggestion, we first provide a comprehensive cross source evaluation using GSM8K, Minerva-Math, ACE-Code, and mixed domain calibration data, while evaluating each resulting table across the same three benchmarks.

Calibration sourceGSM8K Acc./NFEMATH500 Acc./NFEHumanEval Acc./NFEAvg. Acc./NFESpeedup
Fixed threshold 0.9072.9 / 77.532.4 / 106.037.8 / 100.747.70 / 94.732.70×
GSM8K72.3 / 60.633.8 / 88.836.6 / 77.347.57 / 75.573.39×
Minerva-Math72.1 / 56.433.4 / 82.538.4 / 66.947.97 / 68.603.73×
ACE-Code72.2 / 63.333.8 / 90.238.4 / 67.048.13 / 73.503.48×
Mixed domain72.3 / 60.433.4 / 85.938.4 / 68.848.03 / 71.703.57×

The results show that all four calibration sources achieve comparable average accuracy while consistently reducing NFE compared with fixed-threshold APD. In particular, Minerva-Math achieves the lowest average NFE and the highest speedup among the four calibration sources. We therefore recommend it as the primary calibration setting for LLaDA. For Dream, we apply the same validation-based selection principle.

To avoid any potential ambiguity regarding test-informed calibration-source selection, we use GSM8K, MATH500, and HumanEval as validation benchmarks for selecting the calibration source. We then fix Minerva-Math as the calibration source and evaluate TC-APD on five additional held-out benchmarks covering instruction following, summarization, knowledge reasoning, multilingual mathematical reasoning, and open-ended long-form generation. The same threshold table is reused across all five benchmarks without task-specific recalibration. As shown in the table below, TC-APD maintains comparable task performance while reducing NFE by 6–25% across all five benchmarks. These results suggest that a calibration table selected using the validation benchmarks can generalize to additional unseen tasks.

BenchmarkFixed threshold 0.90TC-APD (max threshold 0.90)
IFEval (Prompt Strict Acc. / NFE)57.49% / 137.4657.86% / 124.62
XSum (ROUGE-L / NFE)21.83 / 46.7521.99 / 35.81
ARC-Challenge (Acc. / NFE)84.64% / 13.5784.56% / 10.12
MGSM-Chinese (Exact Match / NFE)67.60% / 82.4368.00% / 73.63
WritingPrompts, len 1024 (Rep-4 ↓ / NFE)29.01 / 725.4129.07 / 681.30
Average Score57.8958.10
Average NFE70.0561.05 (−12.9%)
Average Speedup3.65×4.19× (+14.8%)

Average Score, Average NFE, and Average Speedup are computed over the four benchmarks that use a 256-token generation budget. WritingPrompts uses 1,024 tokens and its Rep-4 is a degeneration statistic rather than an accuracy, so it is reported separately.

AuthorsOpenReview ↗

Response to Reviewer 4QWr [2/6]

W2 & W3 & Q2 — Independent consistency validation and trajectory shift

Weakness (W2): Mismatch between calibration and deployment trajectories. Self-final consistency is estimated under standard decoding, but early commitment changes later model states.

Weakness (W3): No out-of-sample calibration assessment. The paper does not report whether predictions accepted at a nominal target of ρ=0.995 actually achieve a similar consistency rate on independent traces, especially during accelerated decoding.

Question (Q2): Have you measured the realized consistency rate of accepted tokens on an independent calibration test set? More importantly, what is that rate on the trajectories generated by TC-APD itself, rather than on standard-decoding traces?

We conduct two additional experiments to address these concerns, separately examining the generalization of threshold estimation to unseen Standard decoding traces and its behavior during TC-APD inference.

(1) Independent validation on Standard-decoding traces.

We split the original Math calibration data into 4,000 traces for threshold estimation and 1,000 disjoint traces for independent validation. The token-specific thresholds are constructed using only the 4,000 training traces and evaluated on the held-out traces.

On the independent validation set, accepted token predictions achieve 99.443% observed final-token agreement. When including tokens that fall back to the default threshold of 0.90, the overall agreement is 99.447%. However, the agreement varies with token frequency, as shown below:

Accepted occurrences per token in validation setNumber of tokenObserved final-token agreement
1–94794.09%
10–2412595.14%
25–4919097.04%
50–9927298.45%
≥10059999.47%

These results show that threshold estimates generalize well in aggregate, while less frequently observed tokens exhibit greater uncertainty. We therefore clarify that ρ=0.995 is an empirical calibration target rather than a formal statistical guarantee, and that the reliability of individual token estimates depends on their observation frequency.

(2) Trajectory shift during TC-APD decoding.

To investigate the impact of early commitment, we run 500 paired GSM8K trajectories with per-iteration logging. We use the final tokens produced by Standard decoding at each position as a common reference and examine both token-level trajectory agreement and task-level correctness.

We apply the same TC-APD threshold criteria to intermediate predictions from both decoding methods and measure their agreement with the corresponding standard final tokens. Among threshold-accepted candidates, the agreement is 99.06% during standard decoding, but drops to 62.42% during TC-APD decoding. Furthermore, agreement decreases from 94.0% in the first block to 48.6% in the seventh block, reflecting the accumulated divergence caused by early commitment. We use Standard final tokens as the reference because comparing committed tokens with TC-APD's own final output would trivially yield 100% agreement, as committed tokens cannot be revised. These results confirm that early commitment substantially changes subsequent decoding trajectories, and that the original consistency estimates do not directly transfer to accelerated decoding states.

Despite the substantial token-level divergence, its impact on task-level correctness is much smaller. On the full 1,319-example GSM8K test set (Table A8), Standard decoding and TC-APD produce the same final answer on 85.7% of examples and agree on correctness on 90.3%. Correctness changes are bidirectional: 69 examples change from correct to incorrect and 59 from incorrect to correct, a net difference of 10 examples (0.8 pp). These results suggest that token-level trajectory divergence does not translate into a comparable task-level degradation, highlighting the distinction between reproducing Standard decoding trajectories and preserving downstream task performance.

AuthorsOpenReview ↗

Response to Reviewer 4QWr [3/6]

W4 & Q1 — Threshold selection and sampling robustness

Weakness: Potential threshold-selection bias. Selecting the minimum τ that satisfies R_v(τ) ≥ ρ on the same samples used to calculate R_v can produce optimistic estimates. The paper does not use confidence bounds, isotonic calibration, or a separate threshold-selection split.

Question: How exactly are candidate values of τ enumerated, and is monotonicity of R_v(τ) enforced? Since empirical conditional accuracy need not be monotonic at finite sample sizes, how sensitive are the selected thresholds to sampling noise?

We clarify the threshold-selection procedure and further investigate its sensitivity to calibration-data sampling.

(1) Threshold-selection protocol.

Candidate thresholds are enumerated on a fixed grid from 0.01 to 1.00 with a step size of 0.005 (199 candidates). A token is eligible if it has at least 200 candidate observations. Among eligible tokens, we select the lowest threshold τ for which the empirical consistency R_v(τ) reaches ρ = 0.995 and the accepted support is at least 100. Tokens without a valid threshold fall back to the global threshold (e.g. 0.90). We do not enforce monotonicity of R_v(τ) through isotonic regression or a monotone envelope, since empirical consistency may fluctuate across thresholds under finite sampling.

(2) Sensitivity to calibration sampling.

To quantify sampling sensitivity, for each calibration size N in {100, 200, 500, 1000, 2000}, we independently sample three subsets from the same pool of 5,000 Math Standard-decoding trajectories using seeds 2027, 2028, and 2029. We then independently reconstruct the token-specific threshold table for each subset. The 5,000 trajectory setting uses the complete pool and is included only as a full-data reference; its three nominal seed settings are identical by construction.

Calibration sizeValid thresholds (seeds 2027 / 2028 / 2029)Intersection / unionMembership Jaccard ↑Mean absolute threshold difference ↓Pairwise within 0.05 ↑
100141 / 127 / 14290 / 19147.12%0.11026.2%
200235 / 221 / 230153 / 31448.73%0.11927.8%
500410 / 408 / 439301 / 55054.73%0.10432.5%
1,000619 / 620 / 614470 / 79159.42%0.08841.8%
2,000910 / 916 / 911730 / 1,10266.24%0.07351.9%
5,000†1,453 / 1,453 / 1,4531,453 / 1,453100.00%0.000100.0%

† At size 5,000, each setting contains the complete calibration pool. The three tables are therefore identical by construction and are not independent sampling replicates.

The results show that token-level threshold selection is sensitive to calibration sampling at small data sizes. As the calibration size increases from 100 to 2,000, membership Jaccard improves from 47.12% to 66.24%, the mean pairwise threshold difference decreases from 0.110 to 0.073, and the fraction of shared thresholds agreeing within 0.05 increases from 26.2% to 51.9%. Thus, larger calibration sets generally improve both threshold-table coverage and value stability, although finite-sample variability remains at the level of individual tokens.

(3) Independent validation.

In addition, as described in our response to W3&Q2, we evaluate the resulting threshold table on 1,000 independent Standard-decoding traces, achieving 99.443% observed final-token agreement. This provides an out-of-sample assessment of the selected thresholds, complementing the sampling-robustness analysis above.

Together, these results clarify our threshold-selection procedure and characterize its finite-sample sensitivity. We acknowledge that ρ = 0.995 is an empirical selection target rather than a formal statistical guarantee, and that individual token thresholds may remain uncertain when calibration support is limited.

Reviewer_iRo1OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Token-Level Calibrated Adaptive Parallel Decoding (TC-APD) to accelerate diffusion language model inference. The authors observe that different tokens exhibit substantially different final-consistency rates even at similar confidence levels, revealing token-dependent variation in the confidence–stability relationship. Based on this observation, TC-APD estimates token-specific confidence thresholds from offline denoising traces. Experiments on LLaDA-8B-Instruct and Dream-7B-Base across mathematical reasoning and code generation benchmarks show that TC-APD improves inference throughput while maintaining comparable task accuracy.

Critical strengths

The paper reveals that different tokens exhibit substantially different final-consistency rates even at similar confidence levels. This observation is interesting and provides a clear motivation for moving beyond global confidence thresholds in adaptive parallel decoding. The proposed method is simple yet effective, achieving consistent improvements in inference efficiency.

Critical weaknesses
  1. Limited analysis of calibration data size. The paper does not investigate how calibration set size affects performance. Evaluating different data sizes would help assess the robustness of the method.

  2. Limited out-of-domain evaluation. The experiments mainly focus on math and code tasks, without evaluating chat tasks. A more comprehensive out-of-domain analysis across math, code, and chat, would better demonstrate the generalizability of the method.

Questions for authors

It would be valuable to investigate how calibration set size affects the performance and robustness of TC-APD. Additionally, a more comprehensive out-of-domain evaluation covering math, code, and chat tasks would help demonstrate the generalizability of the method. If these additional results further validate the robustness and effectiveness of TC-APD, I would be willing to increase my score.

4 comments
AuthorsOpenReview ↗

Response to Reviewer iRo1 [1/2]

We thank the reviewer iRo1 for the constructive feedback and positive assessment of TC-APD, and appreciate the willingness to reconsider the score based on additional evidence. We address both concerns with new experiments on calibration data size and out-of-domain generalization. Minor baseline variations may arise from different GPU hardwares, but all methods within each table were evaluated under consistent hardwares to ensure fairness. We will clarify this in the revised paper.

W1 & Q1: Experiments on calibration data size

Weakness: Limited analysis of calibration data size. The paper does not investigate how calibration set size affects performance.

Question: How does calibration-set size affect the performance and robustness of TC-APD?

Our initial experiments used 5,000 trajectories as an empirically chosen setting, rather than a theoretically established requirement, to provide sufficient samples for token-level threshold estimation. Following the reviewer's suggestion, we further investigate the calibration data requirements by evaluating TC-APD with calibration sizes ranging from 100 to 5,000 trajectories. For each size below 5,000, we use three independently sampled subsets and report the mean and sample standard deviation. The 5,000 trajectory setting uses the full calibration pool, so no standard deviation is reported.

Calibration sizeGSM8K Acc (%)MATH500 Acc (%)HumanEval Acc (%)Avg Acc (%)Avg NFE
Standard73.1631.4037.2047.25256.00
Fixed threshold 0.9073.0931.8037.2047.3694.19
10073.24 ± 0.1331.00 ± 1.2537.20 ± 1.0647.14 ± 0.7971.65 ± 1.16
20072.88 ± 0.4931.33 ± 0.8335.98 ± 0.6146.73 ± 0.4771.41 ± 1.65
50073.14 ± 0.9131.33 ± 1.3336.99 ± 0.9347.15 ± 1.0071.24 ± 0.72
1,00072.50 ± 0.6331.87 ± 0.6437.20 ± 1.2247.19 ± 0.3970.07 ± 0.39
2,00072.76 ± 0.1832.00 ± 0.4036.99 ± 1.2747.25 ± 0.5769.75 ± 0.56
5,00073.2432.2037.8047.7569.14

Notably, even with only 100 calibration trajectories (0.29 A100 GPU-hours), TC-APD achieves comparable task performance to fixed-threshold APD while reducing inference NFE. Some variability is observed at this small calibration size, particularly on MATH500 and HumanEval, due to differences in the sampled calibration trajectories. Nevertheless, average accuracy remains broadly comparable across calibration sizes, without a clear monotonic trend. Larger calibration sets generally provide lower NFE, suggesting that additional calibration data primarily improves decoding efficiency rather than task accuracy.

To better understand this behavior, we further examine token coverage under the 500 and 5,000 trajectory settings:

Metric500 trajectories5,000 trajectories
Number of valid token types4101,453
Token occurrence coverage92.93%98.22%

The coverage analysis shows that relatively small calibration sets already capture most high-frequency candidate tokens, although individual token thresholds remain sensitive to calibration data. Across independent subsets, larger calibration sets improve token coverage and decoding efficiency, while aggregate performance remains broadly stable. Overall, additional calibration data primarily benefits token-level estimation and decoding efficiency rather than task accuracy.

AuthorsOpenReview ↗

Response to Reviewer iRo1 [2/2]

W2 & Q2: Out-of-domain and cross-lingual generalization

Weakness: Limited out-of-domain evaluation. The experiments mainly focus on math and code tasks, without evaluating chat tasks.

Question: Would broader evaluation across math, code, and chat tasks better demonstrate the generalizability of TC-APD?

To further assess the generalization of TC-APD, we evaluate it on four additional benchmarks covering diverse chat-related tasks, using Minerva-Math or ACE-Code calibration data without task-specific recalibration:

  1. IFEval (Instruction Following, max length 256): Instruction-level strict accuracy, prompt-level strict accuracy, and NFE.
  2. ARC-Challenge (Knowledge Reasoning, max length 256): Accuracy and NFE.
  3. XSum (Summarization, max length 256): ROUGE-1, ROUGE-2, and NFE, evaluated on a fixed subset of 500 examples.
  4. MGSM-Chinese (Cross-lingual Reasoning, max length 256): Exact Match and NFE.
Calibration dataCalibration sizeIFEval Inst. Acc ↑IFEval Prompt Acc ↑IFEval NFE ↓ARC-C Acc ↑ARC-C NFE ↓XSum R-1 ↑XSum R-2 ↑XSum NFE ↓MGSM-CN EM ↑MGSM-CN NFE ↓
—Standard66.79%57.49%256.0084.64%256.0029.958.80256.0067.20%256.00
—Fixed threshold 0.9066.79%57.49%137.4684.64%13.5729.938.7946.7567.60%82.43
ACE-Code5,00066.55%57.30%126.5784.39%9.7629.838.9636.1467.60%74.94
Minerva-Math10066.19%57.12%128.0866.04%10.3429.888.7937.0567.60%73.26
Minerva-Math20066.91%57.49%128.6683.79%10.2729.989.0736.6668.40%74.80
Minerva-Math50066.55%57.67%126.7484.73%10.0829.929.0235.9568.00%75.44
Minerva-Math1,00066.31%57.30%126.3384.04%9.9929.969.0035.8768.00%74.44
Minerva-Math2,00066.79%56.93%126.1384.56%10.1129.949.0236.1767.60%74.05
Minerva-Math5,00067.03%57.86%124.6284.56%10.1229.888.9835.8168.00%73.63

Using calibration tables constructed from either Minerva-Math or ACE-Code, TC-APD maintains comparable performance on IFEval and XSum across calibration sizes. On ARC-Challenge, comparable accuracy is achieved with at least 200 calibration trajectories, while the smallest setting shows a degradation. Importantly, TC-APD also transfers effectively to MGSM-Chinese without language-specific recalibration, maintaining comparable or better accuracy while substantially reducing NFE. These results suggest that token-level calibration can generalize reasonably well across several tasks and languages, without requiring task-specific recalibration.

Overall, these experiments suggest that TC-APD can achieve practical decoding gains with relatively small calibration sets. The results also indicate that a single calibration table can transfer across several evaluated tasks without task-specific recalibration. We have incorporated these findings into the revised paper to provide a more comprehensive assessment of TC-APD's calibration data requirement and cross-domain applicability. If the reviewer has any further questions or concerns, we would be happy to provide additional clarification and address them.

Thank you for the detailed response and additional experiments,and I have raised my score accordingly.

Thank you very much for your follow-up and for reconsidering your assessment based on our additional experiments. We are glad that the new results helped address your concerns, and sincerely appreciate your recognition of our work and constructive feedback.

Sincerely,

The Authors

Reviewer_jgETOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Conventional adaptive parallel decoding for diffusion language models (DLMs) applies a single global confidence threshold to decide which tokens to commit in parallel. This rests on the flawed assumption that equal confidence yields equal prediction stability for all tokens.

The paper verifies that tokens show large differences in final‑consistency rates even at identical confidence. It proposes TC‑APD (Token‑Level Calibrated Adaptive Parallel Decoding). In an offline stage, token‑specific confidence thresholds are derived from DLM denoising traces, based on each token’s empirical consistency against final decoding outputs. At inference time, TC‑APD simply looks up per‑token thresholds to make commit decisions. It introduces no extra training, neural modules or additional model forward passes.

Critical strengths
  1. Empirically grounded observation: Validates vocabulary‑wide token‑dependent confidence‑stability mismatch, pointing out a core flaw of global‑threshold APD.

  2. Lightweight plug‑and‑play design: Only offline threshold‑table construction; inference is pure lookup, zero extra compute overhead, easy to integrate into existing DLM decoders.

Critical weaknesses
  1. Offline calibration dependency: Must collect large‑scale denoising traces beforehand; calibration carries non‑trivial one‑off GPU cost. Threshold tables are model‑specific, cannot zero‑shot transfer across different DLM backbones.

  2. Calibration‑data sensitivity: Performance depends heavily on the distribution of offline traces; mixed‑domain calibration does not always bring improvements.

  3. Trajectory change risk: Token‑level thresholds enable earlier commits, which alters intermediate decoding paths. Though aggregate accuracy stays similar, individual sample outputs may shift relative to standard full denoising.

Questions for authors
  1. Can we reduce or eliminate the heavy offline trace‑collection burden, for example estimating token thresholds without running full denoising trajectories?

  2. How to build calibration tables that robustly generalise across diverse domains without per‑domain re‑calibration?

  3. Could TC‑APD be combined with online dynamic threshold techniques to further mitigate rare‑token fallback limitations?

  4. What is the trade‑off between aggressive early token‑commit and generation robustness for long‑context / creative text generation?

6 comments
AuthorsOpenReview ↗

Response to Reviewer jgET [1/4]

We thank Reviewer jgET for the thoughtful and constructive feedback. We address the reviewer's concerns and questions point by point below, with additional experiments and analyses where appropriate. We would like to clarify that some additional experiments were conducted under different GPU hardwares, which may explain minor variations in baseline results. However, all methods within each table were evaluated under consistent settings to ensure fair comparisons. We will clarify this in the revised paper.

W1: Offline calibration cost and model-specific thresholds

Weakness: Offline calibration dependency: Must collect large-scale denoising traces beforehand; calibration carries non-trivial one-off GPU cost. Threshold tables are model-specific and cannot zero-shot transfer across different DLM backbones.

Regarding calibration cost, our results suggest that TC-APD can achieve practical decoding gains with relatively small calibration sets. In the original paper, we empirically chose 5,000 trajectories to provide sufficient token observations for threshold estimation, rather than treating this as a required calibration size. To further investigate the actual calibration requirements, we conduct additional experiments using 100 and 500 trajectories, with three independently sampled subsets for each setting. We report the mean value across runs.

Calibration settingGSM8K Acc.MATH500 Acc.HumanEval Acc.Avg. Acc.Avg. NFECalibration Cost (A100 GPU-hours)
Fixed threshold 0.9073.0931.8037.2047.3694.19--
TC-APD (100 trajectories)73.2431.0037.2047.1471.650.29
TC-APD (500 trajectories)73.1431.3336.9947.1571.241.45
TC-APD (5,000 trajectories)73.2432.2037.8047.7569.1414.5

Notably, only 100 calibration trajectories (0.29 A100 GPU-hours) are sufficient to achieve comparable average accuracy to fixed-threshold APD while substantially reducing NFE. Larger calibration sets generally provide further improvements in decoding efficiency. We therefore retain 5,000 trajectories as our primary setting to prioritize inference efficiency, rather than as a strict requirement of TC-APD.

Regarding model-specific thresholds, we acknowledge that the calibrated table depends on the target backbone. To assess transferability, we apply the LLaDA-8B table to LLaDA2.0-mini without recalibration. This reduces NFE but causes a 1.5-percentage-point average accuracy drop across GSM8K and MATH500, suggesting some transferability within the same model family, although backbone-specific calibration better preserves accuracy. Similar adaptation is used by OSDT, which calibrates thresholds using an initial sequence [1], and Learn2PD, which trains a lightweight filter on model-generated traces [2]. Unlike training-based approaches, TC-APD requires only one calibration per backbone, with the resulting table reusable across tasks.

Q1: Estimation without full denoising

Question: Can we reduce or eliminate the heavy offline trace‑collection burden, for example estimating token thresholds without running full denoising trajectories?

Although our results in W1 demonstrate that using only 100 calibration trajectories achieves comparable accuracy to fixed-threshold APD while substantially reducing NFE, we further investigate whether calibration costs can be reduced by using partial decoding trajectories.

Following the reviewer's suggestion, we experiment with two strategies that construct token-specific threshold tables from local decoding segments rather than complete denoising trajectories. Specifically, we use the first 48 steps of 500 trajectories and the first 24 steps of 1,000 trajectories, keeping the total calibration NFE approximately matched to that of 100 complete trajectories.

Calibration sourceCalibration NFECoverageGSM8K Acc. / NFEMATH500 Acc. / NFEHumanEval Acc. / NFEAvg. Acc.Avg. NFE
100 full trajectories25,60084.94%73.24 / 59.7131.00 / 82.3837.20 / 72.8747.1471.65
500 trajectories × 48 steps24,00080.89%74.07 / 66.0630.40 / 89.4535.37 / 73.0846.6176.20
1,000 trajectories × 24 steps24,00082.40%72.71 / 60.7631.20 / 84.8436.59 / 72.0946.8372.56

Under similar calibration budgets, partial trajectories achieve slightly lower average accuracy and higher NFE than complete trajectories. We hypothesize that partial trajectories do not fully capture how token probabilities change throughout the denoising process, resulting in less reliable threshold estimates. Therefore, we recommend allocating the calibration budget to fewer complete denoising trajectories rather than collecting more trajectories truncated to their early steps.

AuthorsOpenReview ↗

Response to Reviewer jgET [2/4]

W2 & Q2: Calibration-data sensitivity and cross-domain robustness

Weakness: Calibration-data sensitivity: Performance depends heavily on the distribution of offline traces; mixed-domain calibration does not always bring improvements.

Question: How to build calibration tables that robustly generalise across diverse domains without per-domain re-calibration?

In Appendix A of the submission, we provide initial cross-domain evaluation examining the generalization of TC-APD. Following the reviewer’s suggestion, we have extended these experiments with calibration tables constructed from GSM8K, Minerva-Math, ACE-Code, and their mixed-domain combination, and evaluated each of them across all domains.

Calibration sourceGSM8K Acc./NFEMATH500 Acc./NFEHumanEval Acc./NFEAvg. Acc./NFESpeedup
Fixed threshold 0.9072.9 / 77.532.4 / 106.037.8 / 100.747.70 / 94.732.70×
GSM8K72.3 / 60.633.8 / 88.836.6 / 77.347.57 / 75.573.39×
Minerva-Math72.1 / 56.433.4 / 82.538.4 / 66.947.97 / 68.603.73×
ACE-Code72.2 / 63.333.8 / 90.238.4 / 67.048.13 / 73.503.48×
Mixed-domain72.3 / 60.433.4 / 85.938.4 / 68.848.03 / 71.703.57×

The results show that calibration source choice primarily affects decoding efficiency, while having little impact on average accuracy. Notably, ACE-Code calibration generalizes well to mathematical benchmarks, and Minerva-Math provides the best overall efficiency and is therefore retained as our suggested primary calibration source.

Furthermore, we evaluate the calibrated thresholds on four additional benchmarks without domain-specific recalibration:

  1. IFEval (Instruction Following, max length 256): Instruction-level strict accuracy, prompt-level strict accuracy, and NFE.
  2. ARC-Challenge (Knowledge Reasoning, max length 256): Accuracy and NFE.
  3. XSum (Summarization, max length 256): ROUGE-1, ROUGE-2, and NFE, evaluated on a fixed subset of 500 examples.
  4. MGSM-Chinese (Cross-lingual Mathematical Reasoning, max length 256): Exact Match and NFE.
Calibration dataCalibration sizeIFEval Inst. Acc ↑IFEval Prompt Acc ↑IFEval NFE ↓ARC-C Acc ↑ARC-C NFE ↓XSum R-1 ↑XSum R-2 ↑XSum NFE ↓MGSM-CN EM ↑MGSM-CN NFE ↓
—Standard66.7957.49256.0084.64256.0029.958.80256.0067.20256.00
—Fixed 0.9066.7957.49137.4684.6413.5729.938.7946.7567.6082.43
ACE-Code5,00066.5557.30126.5784.399.7629.838.9636.1467.6074.94
Minerva-Math50066.5557.67126.7484.7310.0829.929.0235.9568.0075.44
Minerva-Math5,00067.0357.86124.6284.5610.1229.888.9835.8168.0073.63

Using calibration tables from one domain without task-specific recalibration, TC-APD maintains comparable performance across instruction following, knowledge reasoning, and summarization, while consistently reducing NFE. It also transfers effectively to Chinese mathematical reasoning, achieving comparable or better accuracy with substantially fewer forward evaluations. To improve the robustness of the calibration table, we prioritize covering high-frequency tokens, which account for most candidate-token occurrences during inference, while using the default threshold for low-frequency or unseen tokens. These results suggest that token-level calibration can generalize across several tasks and languages without requiring domain-specific recalibration.

AuthorsOpenReview ↗

Response to Reviewer jgET [3/4]

W3 & Q4: Trajectory change and the early-commit trade-off

Weakness: Trajectory change risk: Token-level thresholds enable earlier commits, which alter intermediate decoding paths. Although aggregate accuracy remains comparable, individual sample outputs may differ from those obtained with standard full denoising.

Question: What is the trade-off between aggressive early token commitment and generation robustness for long-context and creative text generation?

We agree that early token commitment can alter intermediate decoding trajectories and produce sample level outputs that differ from standard full denoising, even when aggregate task performance remains comparable. To further investigate this trade-off, we conduct additional experiments on two generation tasks:

  1. XSum (Summarization, max length 256): An abstractive summarization benchmark conditioned on news articles. We report ROUGE-1, ROUGE-2, ROUGE-L, and inference speedup, evaluated on a fixed subset of 500 examples.
  2. WritingPrompts (Creative Writing, max length 1,024): An open-ended story generation benchmark conditioned on creative writing prompts. We report the 4-gram repetition rate and inference speedup to characterize generation behavior and efficiency.

We fix the calibration data to the default setting and vary the maximum commitment threshold over {0.70, 0.80, 0.85, 0.90, 0.95}, comparing the resulting quality and efficiency against standard full denoising.

SettingXSum R-1 ↑XSum R-2 ↑XSum R-L ↑XSum Speedup ↑WritingPrompts Rep-4 ↓WritingPrompts Speedup ↑
Standard29.958.8021.841.00×28.711.00×
Fixed APD (0.90)29.938.7921.835.48×29.011.41×
TC-APD (0.95)29.888.9821.997.15×29.071.50×
TC-APD (0.90)29.888.9821.997.15×29.071.50×
TC-APD (0.85)29.928.9422.007.30×29.021.53×
TC-APD (0.80)29.988.9221.987.48×30.121.57×
TC-APD (0.70)29.768.7721.668.01×28.121.64×

Lower thresholds commit earlier, so 0.70 is the most aggressive setting here. On XSum, ROUGE is stable across the sweep, with only minor degradation at the most aggressive setting. On WritingPrompts, repetition does not degrade monotonically with the threshold: the most aggressive setting is in fact the least degenerate row, the worst point is an intermediate threshold, and the spread across the sweep is comparable to the gap between standard decoding and a fixed global threshold. These results suggest that early commitment does not impose a monotone quality-for-speed trade-off, and that its effect is task-dependent. Overall, TC-APD achieves substantial inference speedups with limited changes in the evaluated automatic metrics, which measure n-gram degeneration rather than human-judged writing quality.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.