- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies post-training quantization of diffusion LLMs under Fast-dLLM's cached parallel decoding, and shows that quantization interacts with the confidence-based decoding policy. The authors identify two effects: quantization errors are concentrated in forced commitments, and quantization can reduce confidence and harming decoding parallelism. Based on these observations, they propose DECAL, which reweighs calibration positions considering commitment risk and downstream impact. Experiments on LLaDA-8B-Instruct and Dream-7B show higher W4A4 accuracy recovery over prior calibration methods and reduced parallelism overhead.
Critical strengths
-
The core insight of this paper is clear. The paper identifies two practical dLLM-specific issues caused by quantization under parallel decoding, i.e., errors concentrated in forced commitments and reduced decoding parallelism. The proposed method is designed accordingly, and the experiments show accuracy improvements over the calibration baselines.
-
The experimental analysis and temperature study are sound. The same-state full-precision shadow comparisons in Section 3 help isolate quantization-induced changes in token predictions and commitment decisions from different decoding trajectories, while the temperature study in Section 5.4 provides empirical support for the confidence-deflation explanation of the parallelism tax.
Critical weaknesses
-
For masked diffusion LLMs, the current high-performing models are native block-diffusion model, e.g., LLaDA2.0 [1]. However, while the method seems to be effective, the paper's experiments use two full-sequence diffusion models in 2025, and with block-wise decoding schedule of Fast-dLLM only. I didn't find the method conflicts with block-diffusion models but experiments covering this aspect should be presented.
-
In Section 5.3’s ablation, the flip-only and reach-only variants also perform well and are close to/sometimes better than full DECAL, leaving the benefit of combining the two factors less convincing.
Reference:
[1] Tiwei Bie et al., LLaDA2.0: Scaling Up Diffusion Language Models to 100B, arXiv 2025.
Questions for authors
-
Token and placement flips are both induced by the same quantized confidences, while Eq. (2) models their margin with a zero-mean Gaussian assumption and combines them under an independence assumption. How accurate are these assumptions in practice?
-
How was the concentration parameter $(c)$ selected for each model? Is the performance sensitive to this choice?
8 comments
Response to Reviewer Y31z (2/4)
W2. The single-factor variants perform close to the full prior
We agree that Table 3 does not separate the product from the stronger single factor, and we have softened this claim in Section 5.3. A new control shows what the ablation does establish: the gain comes from where the calibration budget is placed, not from non-uniform weighting as such. We shuffled DECAL’s prior over the 256 window positions and calibrated with the protocol of Table 3: the weights keep their mean, spread and range but lose their alignment with the decoder.
Table R3. Table 3 with the shuffled control added (W4A4, dual cache, mean of two calibration runs).
| Prior | LLaDA GSM8K | LLaDA MATH | LLaDA Avg. | Dream GSM8K | Dream MATH | Dream Avg. |
|---|---|---|---|---|---|---|
| Uniform | 74.91 | 28.22 | 51.57 | 66.00 | 23.11 | 44.56 |
| Shuffled DECAL prior (new) | 74.68 | 28.47 | 51.58 | 66.46 | 24.79 | 45.63 |
| Flip probability only | 76.04 | 29.32 | 52.68 | 67.78 | 24.81 | 46.30 |
| Reach only | 76.17 | 29.11 | 52.64 | 65.97 | 25.89 | 45.93 |
| DECAL (product) | 76.65 | 28.85 | 52.75 | 67.74 | 24.90 | 46.32 |
On LLaDA, the shuffled prior is no better than uniform (51.58 against 51.57; Table R3), while every decoder-aligned prior gains 1.1 to 1.2 points. FAIR-Calib’s prior, also non-uniform but unrelated to ours (rank correlation −0.20), likewise adds only 0.3 points on these tasks (Table 1). On Dream, the shuffled prior is the weakest non-uniform variant. Placement, not non-uniformity, carries the gain.
The single factors look alike on LLaDA because they nearly coincide there (rank correlation 0.85 between them, and at least 0.95 with the product): fragile commitments sit early in the response, where reach is also highest. On Dream, a base model that writes past its answer, they diverge (rank correlation −0.33), and the better single factor depends on the task (flip probability on GSM8K, reach on MATH). We keep the product because it is the expected damage of an error (a flip that reaches the output), so it needs no per-model or per-task choice of factor, and it has the best average on both models.
Why this particular weighting. Each factor of the product is also supported on its own. The risk factor puts the right commitments first: ranked by the full-precision margin from which π is computed, a commitment whose token flips is rated riskier than one whose token does not in 98.7% of pairs on LLaDA and 93.9% on Dream (see our answer to Q1). The decisions it marks are also the ones that matter: when only the forced commitments of W4A4 (on Dream, also the unconditional write at the start of each block) take the full-precision choice, 30 of 33 (LLaDA) and 32 of 36 (Dream) GSM8K answers broken by quantization become correct again. The reach factor rests on a premise we verified, that the errors picked up over later reads are approximately independent (Appendix F), and it carries signal by itself: the reach-only prior beats uniform weighting by 1.1 (LLaDA) and 1.4 (Dream) average points (Table 3). The product thus weights each position by a quantity that is both measurable and decisive for the answer.
Response to Reviewer Y31z (3/4)
Q1. How accurate are the Gaussian and independence assumptions of Eq. (2)?
In short, both assumptions are close to the data, and neither changes the prior. Since Eq. (4) normalizes the prior, DECAL uses only the order of positions. We checked each part of Eq. (2) against the flips recorded for Figure 1b (W4A4 FlatQuant, first 400 GSM8K test questions; 102,400 commitments on LLaDA and 99,200 on Dream).
Smaller margins do flip more often. The margin is the lead of the top token over the runner-up at full precision, and π grows as it shrinks. Take any commitment whose token flips and any whose token does not: the one that flips has the smaller margin in 98.7% of such pairs on LLaDA and 93.9% on Dream, where a random ordering would give 50% (this is the AUROC, 0.987 and 0.939).
Quantization does not push in a fixed direction (zero mean). If quantization systematically favored the written token, near ties would rarely flip; if it favored the runner-up, they would flip most of the time. Without a preferred direction, a near tie is a coin toss, and this is what we observe: among commitments with a margin below 0.1, 47% flip on LLaDA and 51% on Dream.
The Gaussian curve matches the observed flip rates. Eq. (2) assumes that the flip rate falls with the margin along a Gaussian curve. With its single scale fitted to the data, this curve differs from the observed rates by only 0.1 percentage points on average on LLaDA and 0.8 on Dream. The default scale , which must be fixed before the quantizer is known, makes the predicted probabilities too high in absolute terms; DECAL is unaffected, because it uses only their order (see the last point).
Treating the two kinds of failure as independent is a close approximation. A forced commitment can fail by writing the wrong token or by choosing the wrong position. As the reviewer notes, both stem from the same confidences; they are slightly correlated (about 0.1) but rarely occur together, so treating them as independent barely changes how often a forced commitment fails: 37.8% predicted against 39.4% observed on LLaDA, and 39.8% against 39.2% on Dream.
The prior does not depend on these modeling choices. We rebuilt the prior with the Gaussian curves replaced by flip rates measured directly from the data, so that no assumed distribution enters. The rebuilt prior orders the 256 window positions almost exactly as ours: the rank correlation between the two is 0.994 on LLaDA and 0.997 on Dream, where 1 means an identical order.
Response to Reviewer Y31z (1/4)
We thank the reviewer for the positive assessment and the constructive suggestions, and for finding the core insight clear and our analyses sound. In brief: on the block-diffusion model Fast-dLLM v2 [1], forced commitments are as common as on LLaDA, quantization levies a parallelism tax, and DECAL improves W4A4 accuracy; the single factors look alike because they nearly coincide on LLaDA, and a new control shows that the gain comes from decoder-aligned placement; the assumptions of Eq. (2) are close to the data and do not change the prior; and the concentration parameter c only sets how concentrated the prior is, not the placement that carries the gain.
W1. Experiments on block-diffusion models
We added Fast-dLLM v2 [1] (v2 below), a native block-diffusion model adapted from Qwen2.5-Instruct [2], at 1.5B and 7B parameters. Among current block-diffusion models, it gives the cleanest test: its layers are dense like those of LLaDA and Dream, its 7B model is adapted, like Dream-7B, from Qwen2.5-7B (here its instruction-tuned version), and its decoder is the block-diffusion successor of Fast-dLLM, so block decoding is the main change from the paper’s setting. LLaDA2.0 [3], which the reviewer mentions, uses mixture-of-experts layers at 16B and 100B parameters and would change architecture and scale together with the decoding scheme.
Fast-dLLM v2 decodes blocks of 32 tokens from left to right with exact caching of completed blocks, and commits inside each block, in sub-blocks of 8 tokens, with the same threshold rule as Fast-dLLM (τ = 0.9). DECAL is applied unchanged, with its prior collected under v2’s own decoder.
Forced commitments and the parallelism tax carry over
Forced commitments are as common as in the paper: under W4A4, 10% to 15% of the commitments in the returned output are forced, close to the 16.5% on LLaDA (Section 3.2). Quantization lowers the number of tokens written per forward pass and raises the share of forced commitments (Table R1); the resulting tax, 7.6% on v2-1.5B and 14.0% on v2-7B, is well above FlatQuant’s 0.8% and 2.8% on LLaDA and Dream (Figure 1c).
Table R1. Parallelism tax on Fast-dLLM v2 (200 GSM8K prompts, τ = 0.9, W4A4 FlatQuant). Forward passes are counted per returned token, so answer length does not enter.
Accuracy on v2-7B
Table R2. W4A4 accuracy (%) on Fast-dLLM v2-7B (τ = 0.9; mean ± std over three calibration runs).
DECAL improves on FlatQuant by 1.1 points on GSM8K and 0.9 on MATH (Table R2), closing about a fifth of the gap to full precision, in line with its gains in Table 1.