- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Block-wise Token Transformation (BTT), a post-training quantization technique for diffusion language models (dLLMs). In contrast to existing transformation-based quantization approaches that mainly operate along the channel dimension, BTT exploits the token dimension by applying an orthogonal transformation to groups of simultaneously processed tokens before activation quantization and restoring the original token coordinates after the linear projection. Because linear projections share weights across tokens, the transformation and its inverse preserve the original linear function in the absence of quantization.
Critical strengths
- The token-dimension perspective is simple but conceptually interesting.
- The formulation is mathematically well motivated.
- The W4A4 improvements are nontrivial on several tasks.
Critical weaknesses
- The paper discusses QDLM, DLLMQuant, Quant-dLLM, STaR-Quant, and FAIR-Calib in Related Work, but the main experiments primarily build on generic LLM PTQ methods such as DuQuant, QuaRot, and SmoothQuant. Since the paper is specifically positioned as a method for diffusion language models, comparisons or combinations with recent dLLM-specific PTQ methods would considerably strengthen the empirical claim. Otherwise, it remains unclear whether BTT provides complementary gains over state-of-the-art techniques that already explicitly model masking states and denoising dynamics.
- The current evaluation is relatively narrow. Only two dLLM families are evaluated, both around the 7–8B scale, and downstream evaluation is restricted to four code/math benchmarks. It would be important to demonstrate whether the improvements generalize to additional dLLMs, model scales, and broader language-understanding or instruction-following benchmarks.
- The connection to diffusion-specific behavior is not yet convincingly established. The mathematical construction itself does not rely on diffusion modeling; it applies whenever multiple tokens are processed together by a shared linear projection.
- The cost of per-operator transformation search is not sufficiently characterized. Searching multiple seeds independently for a large number of linear operators may result in non-negligible calibration cost. The paper reports inference overhead but does not provide a detailed analysis of calibration/search time and memory consumption. This is particularly relevant for a PTQ method.
Questions for authors
Why are the QuaRot experiments performed without the identity fallback proposed in Section 3.4? How would QuaRot+BTT perform under exactly the same selection protocol as DuQuant+BTT?
3 comments
Thank you for the authors' explanations and additional experiments. However, I still believe that, as a work on dLLM quantization, the proposed method should be thoroughly compared with existing dLLM quantization methods, rather than demonstrating performance improvements over a single baseline. Furthermore, the rebuttal indicates that the actual inference speed of BTT-quantized models remains relatively slow after deployment, e.g., 61 min 21 s for LLaDA-8B-Instruct and 58 min for Dream-v0-Base-7B on a single NVIDIA A100 GPU. Based on these concerns, I maintain my original recommendation of Weak Reject.
Thank you for the follow-up. The 61 min 21 s for LLaDA and 58 min for Dream are the elapsed times of the one-time offline calibration jobs that searched all linear projections. These jobs include model loading, base quantization, calibration forward passes, and candidate scoring. Inference reuses the resulting seed configuration.
The actual inference measurements are in Table 6. On the same H800 fixed-work decoding benchmark, adding FFN-only BTT to DuQuant W4A4 changed latency from 1.548 to 1.617 s/prompt on LLaDA and from 0.950 to 1.008 s/prompt on Dream. The paired overheads were 4.44% and 6.12%, respectively. The current quantized backend is slower than the corresponding floating-point reference under this workload. These measurements quantify the incremental inference cost of BTT on that backend.
At W4A4, the paper evaluates BTT with both DuQuant and QuaRot. At W8A8, it evaluates BTT with DuQuant and SmoothQuant. These are within-quantizer comparisons that measure the effect of adding BTT under each evaluated configuration. We agree that a matched comparison with dLLM-specific PTQ methods would answer a separate empirical question that the current evaluation leaves open.
Thank you for the review. We address the QuaRot question first, then clarify the evaluation scope and offline costs.
QuaRot and identity fallback. The initial fallback-enabled QuaRot run stopped at the evaluation wrapper, so the FFN-only QuaRot rows in Table 1 report the completed forced-on runs. The fallback rule enables the best of 16 candidates only if it reduces local reconstruction error relative to identity. All scores are percentages.
In the DuQuant W4A4 FFN-only ablation, the selected transform beat identity at every projection (32/32 on LLaDA and 28/28 on Dream). The fallback-enabled policy therefore applied BTT at every FFN output projection, yielding the same per-layer transforms and downstream scores as the forced-on policy. With QuaRot, fallback enabled 0/32 and 1/28 projections, respectively. LLaDA reproduced baseline scores, and Dream stayed within 0.22 percentage points of baseline on all four tasks. Forced-on QuaRot had higher four-task means on both models and raised LLaDA HumanEval pass@1 from 32.93% to 39.02%. On Dream, the forced-on variant gave higher HumanEval and GSM8K point estimates than fallback. These results motivate making fallback optional for FFN-only BTT in the evaluated settings. The two quantizers use the same identity decision rule. A comparison under identical calibration protocols remains open.
1. Comparison with dLLM-specific PTQ. We built BTT on QDLM’s open-source dLLM implementations of DuQuant and QuaRot. BTT transforms token activations around a shared linear projection and reuses the baseline’s quantized weights. This interface can also be used in other dLLM pipelines that jointly process multiple token positions. Within each group, we held decoding and evaluation settings fixed. In Table 1, FFN-only BTT improved mean W4A4 scores by 1.50–3.13 percentage points across the four groups. On Dream, MBPP pass@1 rose from 43.80% to 49.40% with DuQuant, and HumanEval pass@1 rose from 31.10% to 37.80% with QuaRot.
We could not locate public author-provided implementations of DLLMQuant, STaR-Quant, or FAIR-Calib for a matched evaluation. Quant-dLLM provides public code for approximately 2-bit weight-only PTQ. BTT could also be integrated with dLLM-specific weight–activation quantization methods. Whether these combinations yield additional gains remains an open empirical question.
2. Evaluation coverage. We evaluated LLaDA-8B-Instruct and Dream-v0-Base-7B, two families in the 7–8B range also studied by DLLMQuant and STaR-Quant. HumanEval and MBPP cover Python code generation. GSM8K and MATH cover mathematical reasoning. Further evaluation on larger dLLMs, including MoE models, and on language-understanding and instruction-following tasks would test how broadly the observed gains transfer.
3. Connection to diffusion decoding. We agree that the function-preserving identity is general. BTT exploits token activations made available during diffusion decoding. In standard KV-cached autoregressive decoding, a projection receives one new activation row per sequence per step. A nontrivial within-sequence transform would therefore require additional rows. Masked diffusion decoding supplies multiple positions from the same sequence in a single denoising pass, potentially including both masked and already-unmasked positions. BTT exploits this property by mixing their activations before quantization and restoring their token coordinates after the shared projection within the same pass.
4. Offline calibration and search cost. With block size 64 and 16 random seed candidates per projection, the logged W4A4+BTT offline runs took 61 min 21 s for LLaDA-8B-Instruct and 58 min for Dream-v0-Base-7B on a single NVIDIA A100 GPU per model. The projection scope of these runs is pending verification. Inference reuses the selected transforms. On an A100 80GB, peak allocated GPU memory during BTT search was 16.23 GiB for LLaDA and 15.46 GiB for Dream. Base DuQuant W4A4 calibration used 128 WikiText-2 sequences of 2,048 tokens each. On an A100 80GB, peak allocated memory was 20.50 GiB for LLaDA and 17.87 GiB for Dream. Peak reserved memory was 20.82 GiB and 18.87 GiB, respectively.