- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies confidence-based adaptive parallel decoding for diffusion language models. Existing methods commonly commit all positions whose top-1 confidence exceeds a shared global threshold. The authors show that this policy overlooks systematic token-level differences: two predicted tokens with similar confidence can have substantially different probabilities of matching the token eventually produced by conservative decoding. Based on this observation, the paper proposes Token-Level Calibrated Adaptive Parallel Decoding (TC-APD). Offline denoising traces are used to collect, for each predicted token , pairs of confidence and self-final-consistency labels. A token-specific threshold is then chosen as the lowest threshold whose empirical final-consistency rate reaches a target , subject to minimum-support constraints. During inference, a prediction of token is committed whenever its confidence exceeds . Unsupported tokens fall back to a default global threshold. Experiments on LLaDA-8B-Instruct and Dream-7B-Base, using GSM8K, MATH500, and HumanEval, show that TC-APD generally improves the accuracy-efficiency trade-off over fixed-threshold APD. For example, on LLaDA with a maximum threshold of , it reduces average NFE from 94.7 to 68.6 while changing average accuracy from 47.7 to 48.0. The paper also reports wall-clock gains and an integration with SOAR.
Critical strengths
- The paper isolates a plausible and empirically well-supported failure mode of global confidence thresholds.
- TC-APD has negligible online overhead: inference requires only a token-ID lookup and introduces no extra model calls.
- The comparison against fixed-threshold APD is conducted over multiple threshold settings rather than at only one operating point.
- On LLaDA at , TC-APD improves average NFE from 94.7 to 68.6 without reducing average accuracy.
- The Dream results suggest that the benefit is not limited to one backbone.
- Wall-clock measurements show that the reduction in NFE produces practical throughput gains.
- The method can be incorporated into an existing decoder such as SOAR with a small modification.
- The appendix candidly reports that accelerated decoding often follows a different trajectory from standard decoding.
Critical weaknesses
- Possible test-informed calibration selection. The paper compares calibration sources on the same benchmarks used for final evaluation and then chooses the source with the best observed trade-off. This can bias the reported results, even when calibration examples do not overlap with test examples.
- Mismatch between calibration and deployment trajectories. Self-final consistency is estimated under standard decoding, but early commitment changes later model states. The low sequence agreement reported in Appendix D confirms that this distribution shift is substantial.
- No out-of-sample calibration assessment. The paper does not report whether predictions accepted at a nominal target of actually achieve a similar consistency rate on independent traces, especially during accelerated decoding.
- Potential threshold-selection bias. Selecting the minimum that satisfies on the same samples used to calculate can produce optimistic estimates. The paper does not use confidence bounds, isotonic calibration, or a separate threshold-selection split.
- Limited support at a stringent target. With , an empirical target of may be supported by only 100 apparently successful observations. This provides weak statistical evidence for such a high reliability level.
- Limited experimental scope. Results cover two models, three benchmarks, a fixed block size, and a maximum generation length of 256. It remains unclear how thresholds behave for longer generation, other languages, or less structured tasks.
- Non-negligible offline cost. A single-domain calibration requires about 3.5 hours on four A100 GPUs, and recalibration may be necessary after changing the model, tokenizer, or decoding setup.
Questions for authors
- How exactly are candidate values of enumerated, and is monotonicity of enforced? Since empirical conditional accuracy need not be monotonic at finite sample sizes, how sensitive are the selected thresholds to sampling noise?
- Have you measured the realized consistency rate of accepted tokens on an independent calibration test set? More importantly, what is that rate on the trajectories generated by TC-APD itself, rather than on standard-decoding traces?
- Were the choices of Minerva-Math calibration for LLaDA and mixed-domain calibration for Dream made before evaluating GSM8K, MATH500, and HumanEval? If not, could you report results using a calibration-source selection protocol based entirely on separate validation data?
- Why is sufficient for a target of ? Could you report results using lower confidence bounds, such as a binomial confidence bound, instead of raw empirical consistency?
- How much of the improvement remains when fixed APD and TC-APD are compared at tightly matched NFE or wall-clock budgets, with paired uncertainty estimates for both task accuracy and efficiency?
- Which changes require recalibration? In particular, do the thresholds transfer across block sizes, generation lengths, prompting templates, quantization settings, or fine-tuned variants of the same backbone?
- Can the authors report the amortization point for the offline calibration cost, measured in the approximate number of generated sequences required before TC-APD provides a net compute saving?
9 comments
Response to Reviewer 4QWr [2/6]
W2 & W3 & Q2 — Independent consistency validation and trajectory shift
Weakness (W2): Mismatch between calibration and deployment trajectories. Self-final consistency is estimated under standard decoding, but early commitment changes later model states.
Weakness (W3): No out-of-sample calibration assessment. The paper does not report whether predictions accepted at a nominal target of ρ=0.995 actually achieve a similar consistency rate on independent traces, especially during accelerated decoding.
Question (Q2): Have you measured the realized consistency rate of accepted tokens on an independent calibration test set? More importantly, what is that rate on the trajectories generated by TC-APD itself, rather than on standard-decoding traces?
We conduct two additional experiments to address these concerns, separately examining the generalization of threshold estimation to unseen Standard decoding traces and its behavior during TC-APD inference.
(1) Independent validation on Standard-decoding traces.
We split the original Math calibration data into 4,000 traces for threshold estimation and 1,000 disjoint traces for independent validation. The token-specific thresholds are constructed using only the 4,000 training traces and evaluated on the held-out traces.
On the independent validation set, accepted token predictions achieve 99.443% observed final-token agreement. When including tokens that fall back to the default threshold of 0.90, the overall agreement is 99.447%. However, the agreement varies with token frequency, as shown below:
| Accepted occurrences per token in validation set | Number of token | Observed final-token agreement |
|---|---|---|
| 1–9 | 47 | 94.09% |
| 10–24 | 125 | 95.14% |
| 25–49 | 190 | 97.04% |
| 50–99 | 272 | 98.45% |
| ≥100 | 599 | 99.47% |
These results show that threshold estimates generalize well in aggregate, while less frequently observed tokens exhibit greater uncertainty. We therefore clarify that ρ=0.995 is an empirical calibration target rather than a formal statistical guarantee, and that the reliability of individual token estimates depends on their observation frequency.
(2) Trajectory shift during TC-APD decoding.
To investigate the impact of early commitment, we run 500 paired GSM8K trajectories with per-iteration logging. We use the final tokens produced by Standard decoding at each position as a common reference and examine both token-level trajectory agreement and task-level correctness.
We apply the same TC-APD threshold criteria to intermediate predictions from both decoding methods and measure their agreement with the corresponding standard final tokens. Among threshold-accepted candidates, the agreement is 99.06% during standard decoding, but drops to 62.42% during TC-APD decoding. Furthermore, agreement decreases from 94.0% in the first block to 48.6% in the seventh block, reflecting the accumulated divergence caused by early commitment. We use Standard final tokens as the reference because comparing committed tokens with TC-APD's own final output would trivially yield 100% agreement, as committed tokens cannot be revised. These results confirm that early commitment substantially changes subsequent decoding trajectories, and that the original consistency estimates do not directly transfer to accelerated decoding states.
Despite the substantial token-level divergence, its impact on task-level correctness is much smaller. On the full 1,319-example GSM8K test set (Table A8), Standard decoding and TC-APD produce the same final answer on 85.7% of examples and agree on correctness on 90.3%. Correctness changes are bidirectional: 69 examples change from correct to incorrect and 59 from incorrect to correct, a net difference of 10 examples (0.8 pp). These results suggest that token-level trajectory divergence does not translate into a comparable task-level degradation, highlighting the distinction between reproducing Standard decoding trajectories and preserving downstream task performance.
Response to Reviewer 4QWr [3/6]
W4 & Q1 — Threshold selection and sampling robustness
Weakness: Potential threshold-selection bias. Selecting the minimum τ that satisfies R_v(τ) ≥ ρ on the same samples used to calculate R_v can produce optimistic estimates. The paper does not use confidence bounds, isotonic calibration, or a separate threshold-selection split.
Question: How exactly are candidate values of τ enumerated, and is monotonicity of R_v(τ) enforced? Since empirical conditional accuracy need not be monotonic at finite sample sizes, how sensitive are the selected thresholds to sampling noise?
We clarify the threshold-selection procedure and further investigate its sensitivity to calibration-data sampling.
(1) Threshold-selection protocol.
Candidate thresholds are enumerated on a fixed grid from 0.01 to 1.00 with a step size of 0.005 (199 candidates). A token is eligible if it has at least 200 candidate observations. Among eligible tokens, we select the lowest threshold τ for which the empirical consistency R_v(τ) reaches ρ = 0.995 and the accepted support is at least 100. Tokens without a valid threshold fall back to the global threshold (e.g. 0.90). We do not enforce monotonicity of R_v(τ) through isotonic regression or a monotone envelope, since empirical consistency may fluctuate across thresholds under finite sampling.
(2) Sensitivity to calibration sampling.
To quantify sampling sensitivity, for each calibration size N in {100, 200, 500, 1000, 2000}, we independently sample three subsets from the same pool of 5,000 Math Standard-decoding trajectories using seeds 2027, 2028, and 2029. We then independently reconstruct the token-specific threshold table for each subset. The 5,000 trajectory setting uses the complete pool and is included only as a full-data reference; its three nominal seed settings are identical by construction.
| Calibration size | Valid thresholds (seeds 2027 / 2028 / 2029) | Intersection / union | Membership Jaccard ↑ | Mean absolute threshold difference ↓ | Pairwise within 0.05 ↑ |
|---|---|---|---|---|---|
| 100 | 141 / 127 / 142 | 90 / 191 | 47.12% | 0.110 | 26.2% |
| 200 | 235 / 221 / 230 | 153 / 314 | 48.73% | 0.119 | 27.8% |
| 500 | 410 / 408 / 439 | 301 / 550 | 54.73% | 0.104 | 32.5% |
| 1,000 | 619 / 620 / 614 | 470 / 791 | 59.42% | 0.088 | 41.8% |
| 2,000 | 910 / 916 / 911 | 730 / 1,102 | 66.24% | 0.073 | 51.9% |
| 5,000† | 1,453 / 1,453 / 1,453 | 1,453 / 1,453 | 100.00% | 0.000 | 100.0% |
† At size 5,000, each setting contains the complete calibration pool. The three tables are therefore identical by construction and are not independent sampling replicates.
The results show that token-level threshold selection is sensitive to calibration sampling at small data sizes. As the calibration size increases from 100 to 2,000, membership Jaccard improves from 47.12% to 66.24%, the mean pairwise threshold difference decreases from 0.110 to 0.073, and the fraction of shared thresholds agreeing within 0.05 increases from 26.2% to 51.9%. Thus, larger calibration sets generally improve both threshold-table coverage and value stability, although finite-sample variability remains at the level of individual tokens.
(3) Independent validation.
In addition, as described in our response to W3&Q2, we evaluate the resulting threshold table on 1,000 independent Standard-decoding traces, achieving 99.443% observed final-token agreement. This provides an out-of-sample assessment of the selected thresholds, complementing the sampling-robustness analysis above.
Together, these results clarify our threshold-selection procedure and characterize its finite-sample sensitivity. We acknowledge that ρ = 0.995 is an empirical selection target rather than a formal statistical guarantee, and that individual token thresholds may remain uncertain when calibration support is limited.
Response to Reviewer 4QWr [1/6]
We thank the reviewer 4QWr for the constructive feedback and positive assessment of TC-APD. We address the concerns with additional experiments and analyses on calibration robustness, threshold reliability, cross-configuration transfer, and calibration-cost amortization. We would like to clarify that some additional experiments were conducted under different GPU hardwares, which may explain minor variations in baseline results. However, all methods within each table were evaluated under consistent settings to ensure fair comparisons. We will clarify this in the revised paper.
W1 & Q3 — Calibration source selection and cross-domain evaluation
We thank the reviewer for highlighting the importance of separating calibration-source selection from final evaluation. Following the reviewer's suggestion, we first provide a comprehensive cross source evaluation using GSM8K, Minerva-Math, ACE-Code, and mixed domain calibration data, while evaluating each resulting table across the same three benchmarks.
The results show that all four calibration sources achieve comparable average accuracy while consistently reducing NFE compared with fixed-threshold APD. In particular, Minerva-Math achieves the lowest average NFE and the highest speedup among the four calibration sources. We therefore recommend it as the primary calibration setting for LLaDA. For Dream, we apply the same validation-based selection principle.
To avoid any potential ambiguity regarding test-informed calibration-source selection, we use GSM8K, MATH500, and HumanEval as validation benchmarks for selecting the calibration source. We then fix Minerva-Math as the calibration source and evaluate TC-APD on five additional held-out benchmarks covering instruction following, summarization, knowledge reasoning, multilingual mathematical reasoning, and open-ended long-form generation. The same threshold table is reused across all five benchmarks without task-specific recalibration. As shown in the table below, TC-APD maintains comparable task performance while reducing NFE by 6–25% across all five benchmarks. These results suggest that a calibration table selected using the validation benchmarks can generalize to additional unseen tasks.
Average Score, Average NFE, and Average Speedup are computed over the four benchmarks that use a 256-token generation budget. WritingPrompts uses 1,024 tokens and its Rep-4 is a degeneration statistic rather than an accuracy, so it is reported separately.