acceptodds
Under review as a conference paper at ICLR 2027

Not All Tokens Are Equal: Rethinking Quantization for Diffusion Language Models

Abstract

Quantization is a standard approach for large language models' deployment. Most existing quantization methods for large language models are designed for autoregressive models and the quantization objective is based on layer-wise mean square reconstruction error. In this paper, we show that such objective is mismatched to diffusion language models. Unlike autoregressive models, diffusion language models distinguish between visible tokens, which provide conditioning context, and masked tokens, which are the actual prediction targets. This structural asymmetry induces a fundamentally different quantization objective. Our analysis shows that minimizing the standard layer-wise mean square reconstruction error can yield a loose bound for quantization error in diffusion language models. Motivated by the above analysis, we explicitly disentangle visible and masked tokens as two groups and propose MV-Quant, which minimizes the worst-group reconstruction error. We further prove that, for an n-layer Transformer, MV-Quant yields a tighter quantization error bound than the standard MSE objective. Extensive experiments across multiple diffusion language models and multiple benchmarks validate the effectiveness of our proposed MV-Quant.

Primary area:
foundation or frontier models, including LLMs
Keywords:
diffusion language model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_PCofOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies post-training quantization for diffusion language models (DLMs). The authors argue that DLMs process two structurally different token groups, i.e., visible tokens as conditioning context and masked tokens as prediction targets, so the standard layer-wise MSE reconstruction objective that treats all tokens equally is mismatched with the true quantization goal. Through a first-order error propagation analysis, they bound the final masked-token error by an input-dependent weighted combination of the group-wise reconstruction errors of the Q, K, V and O projections. They then propose MV-Quant, which replaces the MSE objective in AWQ with a worst-group (min-max) objective, and prove that it yields a tighter error bound under the same error tolerance. Experiments on LLaDA-1.5, Fast-dLLM v2 and SDAR over seven benchmarks show improved average accuracy over RTN, GPTQ and AWQ.

Critical strengths
  1. The paper points out that DLMs contain two token groups with different roles, i.e., visible tokens and masked tokens, which are treated equally by the standard MSE objective, and turns this observation into a simple calibration objective. Since MV-Quant only replaces the reconstruction objective of AWQ, it is easy to implement and introduces no extra inference cost.
  2. The paper provides an error propagation analysis with detailed proofs in the appendix, and evaluates the method on three DLMs and seven benchmarks covering math, code, knowledge and instruction following. MV-Quant achieves the best average accuracy among the quantized models in all reported settings.
Critical weaknesses
  1. Sec. 3 states that β differs significantly across Q, K, V and O and shifts dynamically across sequences, which is the main motivation for the worst-group objective. Could the authors provide empirical measurements or visualizations of α and β on real models to support this claim?
  2. The analysis and the objective (Theorem 3.1 and Eq. (9)) only involve WQW_Q, WKW_K, WVW_V and WOW_O, and the transformer layer in Sec. 2.3 does not include the FFN. How are the FFN layers quantized in MV-Quant, with the standard MSE objective or with the proposed one?
  3. Some implementation details are missing. The paper only states that pileval is used for calibration. How are the masked and visible tokens constructed during calibration (e.g., what masking ratio is used), how many calibration samples are used, and what group size is adopted? The evaluation setup only mentions “standard evaluation protocols” without giving the generation length, denoising steps or number of shots.
  4. There is no ablation study. For example, how does the proposed max objective compare with a weighted sum of the masked-token and visible-token reconstruction errors under different weights? How sensitive are the results to the masking ratio used in calibration? In addition, Eq. (9) is not normalized by the number of tokens in each group; would such normalization affect the results?
  5. The average improvement over AWQ on LLaDA-1.5 8B is only +0.3 (Table 3), much smaller than that on Fast-dLLM v2 (+1.5) and SDAR (+1.6). Since LLaDA is the only model trained from scratch among the three, could the authors explain the reason for this difference?
  6. Table 5 only compares FP and MV-Quant. Since MV-Quant only replaces the objective of AWQ, an AWQ row and the calibration time of both methods should be reported.
  7. Since MV-Quant is presented as a general quantization objective, can it also be combined with other PTQ methods such as GPTQ, and does it bring similar improvements?
Questions for authors

Please refer to the weakness part.

5 comments

W1: Sec. 3 states that β\beta differs significantly across Q,K,VQ, K, V and OO and shifts dynamically across sequences, which is the main motivation for the worst-group objective. Could the authors provide empirical measurements or visualizations of α\alpha and β\beta on real models to support this claim?

In practice, computing the exact theoretical αi(l)\alpha^{(l)}_i and βi(l)\beta^{(l)}_i in Theorem 3.1 is expensive in both memory and time, as AlA_l and BlB_l require chaining full cross-layer Jacobians T(n)⋯T(l+1)T^{(n)} \cdots T^{(l+1)} across all subsequent layers:

  • Memory overhead: For H∈RT×dH \in \mathbb{R}^{T \times d}, each layer Jacobian T(l)T^{(l)} is a (Td)×(Td)(Td) \times (Td) matrix. With T=2048T = 2048 and d=4096d = 4096, materializing even a single layer Jacobian requires (Td)2≈7.0×1013(Td)^2 \approx 7.0 \times 10^{13} entries (≈128 TiB\approx \mathbf{128\text{ TiB}} in FP16; and still ≈6.1 TiB\approx \mathbf{6.1\text{ TiB}} per layer even at T=512,d=3584T = 512, d = 3584).
  • Computational time: Avoiding matrix materialization via exact coordinate-wise autograd requires ∣M∣d\lvert M \rvert d backward passes per sample (≈4.2×106\approx 4.2 \times 10^6 backward passes at ∣M∣=1024,d=4096\lvert M \rvert = 1024, d = 4096, taking >50>50 GPU hours for a single sequence and hundreds of GPU days over 128 calibration samples), while multiplying (Td)×(Td)(Td) \times (Td) Jacobians scales as O(nT3d3)\mathcal{O}(n T^3 d^3).

This cost is also a reason why we adopt the parameter-free worst-group objective (Eq. 9) as a tractable surrogate for Eq. (7). However, to directly address your concern, we randomly select two layers (Layer 1 and Layer 14) on Fast-dLLM v2 7B and compute their coefficients αi(l)\alpha^{(l)}_i and βi(l)\beta^{(l)}_i. As shown in the table below, the results support the claims in Sec. 3: αi(l)\alpha^{(l)}_i and βi(l)\beta^{(l)}_i vary noticeably across {WQ,WK,WV,WO}\{W_Q, W_K, W_V, W_O\}, exhibit clear group imbalance, and shift substantially across different input sequences.

LayerProj.αi(l)\alpha^{(l)}_i (Mean ±\pm Std)αi(l)\alpha^{(l)}_i [Min, Max]βi(l)\beta^{(l)}_i (Mean ±\pm Std)βi(l)\beta^{(l)}_i [Min, Max]
Layer 1WQW_Q36.12±18.3136.12 \pm 18.31[14.56,95.10][14.56, 95.10]0.2974±0.14790.2974 \pm 0.1479[0.0255,0.6336][0.0255, 0.6336]
Layer 1WKW_K111.37±59.69111.37 \pm 59.69[22.09,243.75][22.09, 243.75]0.4330±0.20480.4330 \pm 0.2048[0.1041,0.8549][0.1041, 0.8549]
Layer 1WVW_V385.99±180.23385.99 \pm 180.23[128.69,849.67][128.69, 849.67]0.2844±0.15440.2844 \pm 0.1544[0.0204,0.6668][0.0204, 0.6668]
Layer 1WOW_O139.89±83.51139.89 \pm 83.51[26.80,422.92][26.80, 422.92]0.4219±0.20110.4219 \pm 0.2011[0.0650,0.8087][0.0650, 0.8087]
Layer 14WQW_Q7.01±3.477.01 \pm 3.47[2.81,18.46][2.81, 18.46]0.5007±0.19060.5007 \pm 0.1906[0.0915,0.8435][0.0915, 0.8435]
Layer 14WKW_K15.86±7.7215.86 \pm 7.72[6.33,41.75][6.33, 41.75]0.5124±0.19710.5124 \pm 0.1971[0.0865,0.8386][0.0865, 0.8386]
Layer 14WVW_V23.52±11.8023.52 \pm 11.80[9.25,58.05][9.25, 58.05]0.4884±0.17730.4884 \pm 0.1773[0.0748,0.7940][0.0748, 0.7940]
Layer 14WOW_O21.44±12.5821.44 \pm 12.58[6.90,60.61][6.90, 60.61]0.5066±0.18540.5066 \pm 0.1854[0.0710,0.8346][0.0710, 0.8346]

W2: The analysis and the objective (Theorem 3.1 and Eq. (9)) only involve WQ,WK,WVW_Q, W_K, W_V and WOW_O, and the transformer layer in Sec. 2.3 does not include the FFN. How are the FFN layers quantized in MV-Quant, with the standard MSE objective or with the proposed one?

For the FFN/MLP layers, we also apply the proposed MV-Quant objective rather than the standard uniform MSE loss. Although the FFN itself operates token-wise without direct token-to-token interactions, the quantization error introduced by an FFN layer becomes part of the hidden-state error and is subsequently propagated through the following Transformer layers via the propagation matrices T(l)T^{(l)} in Eq. (5). After decomposing this error into masked- and visible-token components, the two components propagate through the distinct cross-layer operators AlA_l and BlB_l in Proposition 3.2, and thus contribute asymmetrically to the final masked-token error.

W3: Some implementation details are missing. The paper only states that pileval is used for calibration. How are the masked and visible tokens constructed during calibration (e.g., what masking ratio is used), how many calibration samples are used, and what group size is adopted? The evaluation setup only mentions "standard evaluation protocols" without giving the generation length, denoising steps or number of shots.

Thank you for pointing this out. We provide the exact implementation details below and have included a full hyperparameter table in Sec. 5.1 and the Appendix:

  • Quantization Configuration: Tables 1–3 use W4A16 (4-bit asymmetric weight-only quantization with zero-point, group size = 128, and 16-bit FP16 activations), while Table 4 uses W3A16 (group size = 128). Following AWQ, both per-channel scaling ratios and weight clipping thresholds are searched over a 20-point uniform grid.
  • Calibration Data & Mask Construction: Following AWQ, we sample 128 text sequences from pileval. We independently sample t∼U(0,1)t \sim \mathcal{U}(0, 1) and set the masking probability pmask(t)=tp_{\mathrm{mask}}(t) = t (expected masking ratio 0.50.5). Each position is independently masked with [MASK] via Bernoulli(pmask(t))\mathrm{Bernoulli}(p_{\mathrm{mask}}(t)).
  • Evaluation Setup: All baselines and MV-Quant strictly follow the official evaluation protocol of each backbone. Under dynamic parallel decoding with a confidence threshold of 0.90.9, the number of denoising steps is adaptive rather than fixed. Specifically: For Fast-dLLM v2 7B, MMLU and GPQA are evaluated under 5-shot likelihood, GSM8K, MATH, and IFEval under 0-shot generation with a maximum generation length of 2,048 tokens, and HumanEval and MBPP under 0-shot greedy decoding with a maximum length of 768 tokens. For SDAR 8B, we set a maximum generation length of 4,096 tokens, adopting 5-shot prompting for MMLU and 0-shot prompting for all other tasks. For LLaDA-1.5 8B, we set the generation length and block size to (3,3)(3, 3) for MMLU (5-shot), (256,16)(256, 16) for GSM8K (4-shot), GPQA (5-shot), and IFEval (0-shot), (1024,128)(1024, 128) for MATH (0-shot), and (512,32)(512, 32) for HumanEval (0-shot) and MBPP (3-shot).

W4: There is no ablation study. For example, how does the proposed max objective compare with a weighted sum of the masked-token and visible-token reconstruction errors under different weights? How sensitive are the results to the masking ratio used in calibration? In addition, Eq. (9) is not normalized by the number of tokens in each group; would such normalization affect the results?

First, regarding the max objective and weighted sum: comparing AWQ and MV-Quant in Tables 1–4 already directly compares the equal-weighted sum (0.5Lm+0.5Lv0.5 \mathcal{L}_m + 0.5 \mathcal{L}_v, Eq. 8) against our max⁡(Lm,Lv)\max(\mathcal{L}_m, \mathcal{L}_v) objective (Eq. 9) under identical settings.

For the βi(l)\beta^{(l)}_i-weighted sum, first, computing the exact weights βi(l)\beta^{(l)}_i is computationally expensive due to cross-layer Jacobians (see W1). Moreover, as shown by our empirical measurements in W1, βi(l)\beta^{(l)}_i differs across {Q,K,V,O}\{Q, K, V, O\}, varies across layers, and shifts dynamically across sequences within [0.02,0.85][0.02, 0.85], so no fixed static weight can align across all layers and inputs. This directly motivates our parameter-free min-max objective max⁡(Lm,Lv)\max(\mathcal{L}_m, \mathcal{L}_v) as a tractable surrogate.

Second, regarding the calibration masking ratio and token-count normalization: during DLM inference, all masking ratios appear across denoising stages. Following the standard training procedure of masked DLMs (e.g., LLaDA), we construct calibration inputs using the same random masking process: for each calibration sequence, we sample t∼U(0,1)t \sim \mathcal{U}(0, 1) and independently replace each token with [MASK] with probability tt, covering diverse masking ratios across the denoising process without relying on a fixed masking ratio. Since E[t]=0.5\mathbb{E}[t] = 0.5, the expected masking ratio over the calibration dataset is 0.5, so the aggregated numbers of masked and visible tokens are balanced. Therefore, neither group dominates Eq. (9) due to group size, and token-count normalization is unnecessary. Whichever group has a larger value in Eq. (9) reflects higher per-token reconstruction error.

W5: The average improvement over AWQ on LLaDA-1.5 8B is only +0.3 (Table 3), much smaller than that on Fast-dLLM v2 (+1.5) and SDAR (+1.6). Since LLaDA is the only model trained from scratch among the three, could the authors explain the reason for this difference?

Fast-dLLM v2 and SDAR are initialized from pre-trained autoregressive models, which process only clean visible tokens during pre-training, and are subsequently fine-tuned into diffusion models by introducing an additional [MASK] token. This induces a stronger distributional discrepancy between visible- and masked-token activations, causing uniform MSE (AWQ) to suffer larger group imbalance and yielding larger gains for MV-Quant at 4-bit. In contrast, LLaDA-1.5 is trained from scratch purely with masked diffusion, so its masked and visible representations co-evolve more smoothly. Since recent state-of-the-art DLMs predominantly adopt this AR-initialized fine-tuning paradigm (e.g., DiffusionGemma, Fast-dLLM v2, and SDAR), this further highlights the advantage of MV-Quant.

W6: Table 5 only compares FP and MV-Quant. Since MV-Quant only replaces the objective of AWQ, an AWQ row and the calibration time of both methods should be reported.

Compared with AWQ, MV-Quant only replaces the scalar MSE loss with the min-max loss, while keeping all other components, calibration data (128×512128 \times 512), grid search steps (2020), and optimization procedures unchanged. Consequently, MV-Quant introduces no calibration overhead, requiring virtually the same PTQ calibration time as AWQ. Both AWQ and MV-Quant take approximately 15 minutes for 7B/8B models).

During inference, because both AWQ and MV-Quant pack weights into the exact same 4-bit format (W4A16, group size 128) and execute the identical INT4 kernel, AWQ has the same model memory footprint and kernel-level inference throughput as MV-Quant in Table 5.

W7: Since MV-Quant is presented as a general quantization objective, can it also be combined with other PTQ methods such as GPTQ, and does it bring similar improvements?

Thank you for your suggestion. We integrate our MV objective into two distinct PTQ frameworks GPTQ and Quant-dLLM, while keeping all other quantization and evaluation settings unchanged. As shown in Table B below, Quant-dLLM + MV also improves the performance of LLaDA 8B under 2-bit quantization from 56.3 to 57.1. Moreover, as shown in Table A below, GPTQ + MV improves the average performance of GPTQ over the seven benchmarks from 60.1 to 60.5. Together with AWQ, these results confirm that MV generalizes across activation-aware AWQ, Hessian-based GPTQ, and extreme low-bit Quant-dLLM frameworks.

Table A: Combining the MV objective with GPTQ (4-bit) on Fast-dLLM v2 7B.

MethodGSM8KMathHumanEvalMBPPGPQAMMLUIFEvalAvg.
GPTQ80.552.963.060.727.966.469.560.1
GPTQ + MV81.253.063.061.028.367.170.160.5

Table B: Combining the MV objective with Quant-dLLM (2-bit) on LLaDA-8B-Base.

BenchmarkQuant-dLLM+MV
PIQA68.168.2
ARC-C36.437.5
ARC-E66.068.9
HellaSwag44.044.5
WinoGrande66.966.5
Average56.357.1
Reviewer_nCpBOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that standard layer-wise reconstruction objectives are poorly matched to diffusion language models because visible and masked tokens contribute differently to prediction error. It analyzes their separate error propagation and proposes MV-Quant, which replaces AWQ’s reconstruction objective with the maximum of the visible- and masked-group errors. Under specified reconstruction-error tolerances and a first-order analysis, the paper derives tighter error bounds. Experiments on three diffusion language models and seven benchmarks report four-bit average accuracy improvements of 1.5, 1.6, and 0.3 percentage points over AWQ, with an additional 2.0-point improvement for three-bit SDAR. Deployment measurements compare memory use and throughput against full precision.

Critical strengths

The paper offers a novel perspective by distinguishing the roles of masked and visible tokens during quantization calibration for diffusion language models. Its analysis motivates a group-aware reconstruction objective, providing a principled way to adapt quantization to the prediction structure of these models.

Critical weaknesses
  1. Limited empirical validation of the group-specific error analysis. The experiments do not directly validate the different effects of masked- and visible-token quantization errors on final predictions. Although the paper reports an aggregate reduction in quantization error and improvements in downstream performance, these results do not establish how much each token group contributes to prediction degradation. Reporting group-wise reconstruction errors, together with controlled experiments that isolate each group’s effect on predictions at masked positions, would provide stronger empirical support for the central motivation behind MV-Quant.
  2. Unclear influence of token-group size on the MV objective. The MV objective uses each group’s total reconstruction error without normalizing by its token count. Consequently, the larger group may dominate the objective because of its size rather than greater per-token error or sensitivity of the final predictions. It is unclear how this balance changes across mask ratios and denoising stages. Please report which group dominates under these conditions and whether prioritizing that group reduces final prediction error.
Questions for authors

Please refer to critical weaknesses.

3 comments

W1: Limited empirical validation of the group-specific error analysis. The experiments do not directly validate the different effects of masked- and visible-token quantization errors on final predictions. Although the paper reports an aggregate reduction in quantization error and improvements in downstream performance, these results do not establish how much each token group contributes to prediction degradation. Reporting group-wise reconstruction errors, together with controlled experiments that isolate each group’s effect on predictions at masked positions, would provide stronger empirical support for the central motivation behind MV-Quant.

Following the reviewer's suggestion, we conduct both group-wise reconstruction error measurements and controlled group-isolation experiments on Fast-dLLM v2 7B (W4A16):

  1. Group-wise Reconstruction Errors (Table A): Summing layer-wise reconstruction errors across all 196 linear layers shows a clear imbalance, where visible-group error (∑ϵv2=110.52×106\sum \epsilon_v^2 = 110.52 \times 10^6) substantially exceeds masked-group error (∑ϵm2=87.43×106\sum \epsilon_m^2 = 87.43 \times 10^6). MV-Quant simultaneously reduces both group errors and lowers the worst-group error.

  2. Controlled Group-Isolation on Final Masked Predictions (Table B): As shown in Table B below, we isolate each group's effect by applying quantized weights only to visible tokens (Visible-Only), only to masked tokens (Masked-Only), or to both (Both). In all three settings above, MV-Quant consistently achieves lower final quantization error than AWQ.

Moreover, Visible-Only induces a larger final error at masked positions than Masked-Only (3,757.543,757.54 vs. 3,361.313,361.31 under AWQ), directly confirming the asymmetric impact of the two groups on final predictions.

Table A: Aggregated group-wise reconstruction errors across all linear layers on Fast-dLLM v2 7B (W4A16).

Method∑ϵm2\sum \epsilon_m^2 (×106\times 10^6)∑ϵv2\sum \epsilon_v^2 (×106\times 10^6)∑max⁡(ϵm2,ϵv2)\sum \max(\epsilon_m^2, \epsilon_v^2) (×106\times 10^6)
AWQ (MSE)87.43110.52110.73
MV-Quant87.23110.16110.36

Table B: Controlled isolation of visible- and masked-token quantization errors on final masked-position predictions (Fast-dLLM v2 7B, W4A16).

SettingMethodFinal Quant Error (↓\downarrow)
Visible-OnlyAWQ (MSE)3,757.54
Visible-OnlyMV-Quant3,737.97
Masked-OnlyAWQ (MSE)3,361.31
Masked-OnlyMV-Quant3,346.79
BothAWQ (MSE)4,451.11
BothMV-Quant4,437.68

W2.1: Unclear influence of token-group size on the MV objective. The MV objective uses each group’s total reconstruction error without normalizing by its token count. Consequently, the larger group may dominate the objective because of its size rather than greater per-token error or sensitivity of the final predictions.

Following the standard training procedure of masked DLMs (e.g., LLaDA), we construct calibration inputs using the same random masking process: for each calibration sequence, we sample t∼U(0,1)t \sim \mathcal{U}(0, 1) and independently replace each token with [MASK] with probability tt, covering diverse masking ratios across the denoising process. Since E[t]=0.5\mathbb{E}[t] = 0.5, the expected masking ratio over the calibration dataset is 50%, so the aggregated numbers of masked and visible tokens are balanced.

W2.2: It is unclear how this balance changes across mask ratios and denoising stages. Please report which group dominates under these conditions and whether prioritizing that group reduces final prediction error.

We would like to clarify that per-denoising-stage group prioritization is not the optimization mechanism of MV-Quant, and all PTQ methods cannot prioritize different groups independently at individual denoising stages.

Specifically, the quantized weights W^\hat{W} are determined once during calibration and remain fixed across all denoising stages during inference. The weights cannot be adjusted dynamically per stage to favor different groups. Instead, our calibration jointly accounts for all denoising stages via t∼U(0,1)t \sim \mathcal{U}(0, 1), over which the visible group exhibits larger overall reconstruction error (Table A) and minimizing the worst-group error reduces the final prediction error (Table B).

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer,

Thank you again for your efforts and valuable comments in reviewing our paper. We have carefully addressed your comments and concerns in detail in the rebuttal and we hope you might find the response satisfactory. We are very much looking forward to hearing from you about any further feedback. We will be very happy to clarify further concerns (if any).

Best,

Authors.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer,

Thank you again for your efforts and valuable comments in reviewing our paper. We have carefully addressed your comments and concerns in detail in the rebuttal and we hope you might find the response satisfactory. We are very much looking forward to hearing from you about any further feedback. We will be very happy to clarify further concerns (if any).

Best,

Authors.

Reviewer_MkoTOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a quantization framework "MV-Quant" for dlm. Firstly, this paper identifies that the visible and masked token groups contribute differently to quantization error. Then this paper provides a detailed theoretical analysis and an upper bound on quantization error, which demonstrates that mv-quant provides a tighter upper bound than MSE loss. Experiments on three dlm backbones report improved accuracy over the evaluated baselines, along with memory savings and inference speedups using existing INT4 kernels.

Critical strengths
  1. This paper provides a detailed theoretical analysis for the proposed quantization upper bound and loss objective, which gives a better objective to guide AWQ quantization.
  2. Experiments show the proposed loss objective could help to reduce the accuracy degradation of AWQ under 4-bit and 3-bit quantization, demonstrating its effectiveness.
Critical weaknesses
  1. The proposed method has limited novelty. The key contribution of this paper lies in separating the reconstruction of masked tokens from that of visible tokens. However, this is the standard min-max method and fails to provide sufficiently strong insights. Although experiments on table1-4 demonstrate the effectiveness of the proposed loss objective (eq9), further ablation studies and discussion are required to clarify its contributions. For example, compare the proposed loss objective with the standard reconstruction error under different bit-width quantization, which would help to access whether its benefits are consistent across different bit widths.
  2. The eq60 and eq64 use the different conditions. For the same ϵ\epsilon, eq64 provides a tigher upper bound. So obtaining a smaller upper bound for MV-Quant under this condition dose not demonstrate that it will achieve lower quantization error in practice. More experiments are needed to demonstrate it.
  3. The generalization of the proposed framework is not discussed. Experiments only demonstrate the results based on AWQ. But the effectiveness on other quantization frameworks (e.g. gptq) is unknown.
  4. DLLMQuant uses a W4A4 setting in its original paper, but this paper does not clearly specify the quantization setting used by MV-Quant. Therefore, it is unclear whether the two methods are compared under the same quantization setting. And the code of DLLMQuant is not released and the results are from its paper, which may not be under the same evaluation setting.
Questions for authors
  1. Could you provide more details on the experimental setup, including the quantization setting. (e.g. W4A4 or W4A16?) This raises concerns about the fairness of the comparison.
  2. Could you apply your framework to gptq and show the results based on gptq, which would help to assess the generalizability of the proposed objective?
  3. Could you add 2-bit results and compare with Quant-dLLM under the same average weight-bit budget and activation precision? This would give a more direct comparison with a dlm-specific method under more aggressive quantization.
5 comments

W1.1: The proposed method has limited novelty. The key contribution of this paper lies in separating the reconstruction of masked tokens from that of visible tokens. However, this is the standard min-max method and fails to provide sufficiently strong insights.

We would like to clarify that we do not claim that the novelty lies in inventing the min-max loss. However, our contribution is revealing why the conventional MSE-based reconstruction objective, which is widely used in prior LLM/DLM quantization methods (e.g., AWQ, GPTQ, and Quant-dLLM), is misaligned with DLMs. Specifically, our theorem and experiments show that the relative importance βi(l)\beta^{(l)}_i between visible- and masked-token errors varies substantially across {Q,K,V,O}\{Q, K, V, O\}, layers, and input sequences. Consequently, no fixed token weighting can align across layers and inputs. Moreover, directly computing exact α\alpha and β\beta requires chaining (Td)×(Td)(Td) \times (Td) cross-layer Jacobians (e.g., ≈\approx 128 TiB in FP16 for a single layer at T=2048,d=4096T=2048, d=4096), making it impractical for PTQ. This directly motivates minimizing the worst-group error to guarantee a tighter quantization error bound without extra overhead.

W1.2 & Q3: Although experiments on table1-4 demonstrate the effectiveness of the proposed loss objective (eq9), further ablation studies and discussion are required to clarify its contributions. For example, compare the proposed loss objective with the standard reconstruction error under different bit-width quantization, which would help to access whether its benefits are consistent across different bit widths. Could you add 2-bit results and compare with Quant-dLLM under the same average weight-bit budget and activation precision? This would give a more direct comparison with a dlm-specific method under more aggressive quantization.

Our paper already includes 3-bit quantization results in Table 4, where MV-Quant outperforms the standard MSE-based objective by 2.0 percentage points on SDAR-8B.

To further validate its effectiveness, we conduct additional 2-bit quantization experiments on LLaDA 8B. Specifically, we replace the reconstruction objective in Quant-dLLM with our MV objective, keeping all other quantization and evaluation settings unchanged (including the same average 2-bit weight budget and FP16 activation precision). As shown below, MV improves performance on four out of five benchmarks, with gains of up to +2.9 points on ARC-E, and increases average accuracy from 56.3 to 57.1.

Together with our 3-bit and 4-bit results, these experiments demonstrate the consistent effectiveness of MV across different quantization bit widths.

BenchmarkQuant-dLLM+MV
PIQA68.168.2
ARC-C36.437.5
ARC-E66.068.9
HellaSwag44.044.5
WinoGrande66.966.5
Average56.357.1

W2: The eq60 and eq64 use the different conditions. For the same ϵ\epsilon, eq64 provides a tigher upper bound. So obtaining a smaller upper bound for MV-Quant under this condition dose not demonstrate that it will achieve lower quantization error in practice. More experiments are needed to demonstrate it.

We agree that a tighter theoretical upper bound alone does not automatically guarantee a smaller quantization error in practice. First, as reported in Sec. 5.2 (Lines 428–434), on Fast-dLLM v2 7B (4-bit), replacing the standard MSE loss in AWQ with MV-Quant under identical settings directly reduces the measured end-to-end quantization error from 74,752 to 73,216. Second, to directly verify the layer-wise conditions in Eqs. (60) and (64), we newly measured the group-wise reconstruction errors summed across all 196 linear layers on Fast-dLLM v2 7B (W4A16). Compared with the standard MSE loss in AWQ, MV-Quant not only lowers the worst-group error ∑max⁡(ϵm2,ϵv2)\sum \max(\epsilon_m^2, \epsilon_v^2) from 110.73×106{110.73 \times 10^6} to 110.36×106{110.36 \times 10^6}, but also reduces both ∑ϵm2\sum \epsilon_m^2 (87.43×106→87.23×10687.43 \times 10^6 \to {87.23 \times 10^6}) and ∑ϵv2\sum \epsilon_v^2 (110.52×106→110.16×106110.52 \times 10^6 \to {110.16 \times 10^6}). Together with the consistent downstream accuracy gains across 4-bit, 3-bit, and 2-bit settings, these results empirically confirm that MV-Quant achieves lower quantization error both layer-wise and end-to-end in practice.

W3 & Q2: The generalization of the proposed framework is not discussed. Experiments only demonstrate the results based on AWQ. But the effectiveness on other quantization frameworks (e.g. gptq) is unknown. Could you apply your framework to gptq and show the results based on gptq, which would help to assess the generalizability of the proposed objective?

Thank you for your suggestion. To verify its generalization beyond AWQ, we further integrate MV into two different quantization frameworks: GPTQ and Quant-dLLM, while keeping all other settings unchanged. As shown in the table in W1.2, Quant-dLLM + MV improves the average accuracy on LLaDA-8B-Base under 2-bit quantization from 56.3 to 57.1. Moreover, as shown in the table below on Fast-dLLM v2 7B (W4A16), GPTQ + MV consistently outperforms or matches GPTQ across all seven benchmarks, improving average accuracy from 60.1 to 60.5. Together with AWQ, these results confirm that MV generalizes across activation-aware, Hessian-based, and extreme low-bit quantization frameworks.

MethodGSM8KMathHumanEvalMBPPGPQAMMLUIFEvalAvg.
GPTQ80.552.963.060.727.966.469.560.1
GPTQ + MV81.253.063.061.028.367.170.160.5

W4 & Q1: Could you provide more details on the experimental setup, including the quantization setting. (e.g. W4A4 or W4A16?). DLLMQuant uses a W4A4 setting in its original paper, but this paper does not clearly specify the quantization setting used by MV-Quant. Therefore, it is unclear whether the two methods are compared under the same quantization setting. And the code of DLLMQuant is not released and the results are from its paper, which may not be under the same evaluation setting.

Thank you for pointing this out. MV-Quant and all reproduced baselines (RTN, GPTQ, and AWQ) in our paper follow the standard weight-only quantization setting: W4A16 in Tables 1–3 and W3A16 in Table 4 (asymmetric quantization with zero-point, group size = 128, FP16 activations, calibrated on 128 PileVal sequences of length 512 with random masking t∼U(0,1)t \sim \mathcal{U}(0, 1)). We have included these in our revised paper. Since DLLMQuant uses W4A4 and its code is not publicly available for evaluation under the same protocol, we agree that it is not directly comparable. We have removed the DLLMQuant row from Table 3 and explicitly specified the W4A16/W3A16 settings in the revised manuscript.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer,

Thank you again for your efforts and valuable comments in reviewing our paper. We have carefully addressed your comments and concerns in detail in the rebuttal and we hope you might find the response satisfactory. We are very much looking forward to hearing from you about any further feedback. We will be very happy to clarify further concerns (if any).

Best,

Authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.