- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
DualQuant utilises a dual scaling mechanism derived from SINQ, but optimises dual scaling factors by minimising the mean squared error of weight reconstruction rather than minimising the activation-weight product error, thereby enabling competitive activation quantisation without the need for calibration data.
Critical strengths
- DualQuant achieves better perplexity and accuracy in most popular W4A4 percision formats compared to other SOTA solutions, while it is calibration-free.
- The observation that minimising the weights-reconstruction error can also achieve effective and accurate activation rescaling is interesting.
Critical weaknesses
- The observation on “minimising the weights-reconstruction error can also achieve effective and accurate activation rescaling” is empirical and lacks clear boundaries regarding its applicability. Although author provides evaluation results on different models, it is unclear when this approach might fail, or whether it is generally applicable.
Questions for authors
- It is good to see that Tables 11 and 12 present comparisons with FP16; please consider including further comparisons with other quantisation schemes in terms of end-to-end memory usage and throughput/latency.
- The phrase "calibration-free" might be ambiguous, the process of finding and could also be considered as calibration, please consider use more precise description.
- Please clarify the details of the baseline configuration. Although the author provides some information in Appendix F, there remain some concerns regarding the baseline comparison. For example, QuaRot should support quantisation across the entire computation path (including the KV cache, weights and activations), whilst Table 1 only illustrates the quantisation precision for weights and activations. Does the evaluation of QuaRot uses 4-bit precision KV cache quantisation, but compare it to your work without using KV cache quantisation? If so the comparision is unfair.
3 comments
Thank you to the authors for their response. The authors' response addresses some of my concerns but does not fully resolve my concerns regarding the applicability of the proposed method. Therefore, I maintain my rating of Weak Accept.
Follow-up Response to Reviewer Js8N: Guaranteed Weight Optimization, General Applicability, and When the Activation Benefit Is Largest
We would like to thank the reviewer for the follow-up. We understand the remaining concern as: where does DualQuant apply, and under which conditions does its weight-derived activation rescaling help? We address this in three parts as below:
-
The weight-side optimization is guaranteed. DualQuant solves an explicit optimization problem: it minimizes the weight-reconstruction MSE under the format's scale constraints. Every projected update is an exact block minimizer (Lemmas 1–2). On a fixed grid, the error decreases monotonically and the iterates terminate finitely at a coordinate-wise stationary point (Theorem 1(a)). On the adaptive grid used in our experiments, the same holds once the quantized weights stabilize, which can be checked at run time and occurs within about 15 iterations in practice (Theorem 1(b), Proposition 1, Figures 2–3).
-
DualQuant is generally applicable by construction. Its only inputs are a weight matrix and the target quantizer, which every linear layer provides. It therefore applies to any linear layer in any architecture, with no calibration data, no architecture-specific design, and no tuning beyond the iteration count. In particular, DualQuant is applicable wherever established PTQ methods such as GPTQ, SmoothQuant, QuaRot, ResQ, AdpQ, EasyQuant, and many more are applicable: like DualQuant, they act on the linear layers of the model, but most additionally require calibration data (GPTQ, SmoothQuant) or architecture-specific rotation fusion (QuaRot, ResQ), whereas DualQuant needs only the weights themselves. Unlike approaches that rely on activation statistics (SmoothQuant), proxy heuristics (SINQ's matrix imbalance), or rotations alone (QuaRot), DualQuant optimizes an explicit error criterion under the format constraints. It also cannot fail by diverging: the format is built into the optimization problem itself, so every scale is projected onto the admissible set (E8M0, UE4M3, or BF16) at every step and is representable by construction. This is precisely the failure mode behind the divergence of SINQ+ and QuaRot on NVFP4 (Tables 1, 2, 8), which DualQuant avoids by design across all formats, models, and bit-widths we evaluated.
-
When the activation-side benefit is largest. The same column scales β attenuate input channels with large-magnitude weight columns. This benefits activation quantization most when activation outliers are channel-aligned and persistent across tokens, which is the regime documented for LLM linear-layer inputs:
- 4-bit block formats with constrained scales (MXINT4, MXFP4, NVFP4), where the shared block scale cannot absorb channel outliers. Here DualQuant is the strongest calibration-free method and often surpasses calibrated SmoothQuant (Tables 1, 2, 8). These are precisely the native formats of current accelerators.
- Projections with channel-wise outliers, e.g.,
k_proj,v_proj,q_proj(+0.81, +0.65, +0.25 dB activation SQNR; Table 7). The gain is more modest where outliers are token-specific rather than channel-aligned, e.g.,down_projinputs with massive activations (+0.06 dB, Table 7), a case that no static per-channel scale, including SmoothQuant's, can target. Even there, β never reduced activation SQNR in any projection we measured (Table 7).
We would be glad to evaluate any specific scenario the reviewer has in mind, e.g., a particular architecture, format, or bit-width.
Response to Reviewer Js8N: Activation Rescaling, End-to-End Memory and Throughput Comparisons, and Experimental Clarifications
We thank the reviewer for the thoughtful questions. We address them below and highlight the corresponding additions to the revised paper.
W1: Applicability of weight-derived activation rescaling.
The activation-side benefit of DualQuant is a key empirical observation of our work, supported by broad evidence across models, formats, and tasks. DualQuant's scales play two complementary roles:
Weight side (optimization). The row and column scales minimize a weight-reconstruction MSE under format-specific scale constraints. Each projected closed-form update exactly minimizes its block with the quantized weights fixed. Our revised Theorem 1 guarantees finite termination for fixed grids and conditional convergence for adaptive grids once the quantized matrix stabilizes. Importantly, this optimization uses only model weights, with no activation statistics or calibration data.
Activation side (empirical). The same column scales β rescale layer inputs before activation quantization. Their benefit is not theoretically guaranteed, but we provide three complementary forms of evidence:
(i) Isolated activation study (Appendix E, Table 7): Holding quantized weights fixed and varying only β, activation SQNR improves across all seven projection types of Llama-3.1-8B, averaging +0.33 dB, with the largest improvements in
k_proj(+0.81 dB) andv_proj(+0.65 dB).(ii) Scale ablation (Table 4): Adding β to row-only scaling consistently reduces W+A perplexity across the evaluated models and formats.
(iii) End-to-end evaluation: We evaluate six Llama/Qwen models, four 4-bit formats, perplexity, five zero-shot tasks, and newly added reasoning/code-generation benchmarks, demonstrating broad empirical effectiveness.
We hypothesize that weight-derived column scales rescale activations in a way that benefits activation quantization. Validating such benefits empirically is common practice in PTQ, as in QuaRot (NeurIPS 2024) and ResQ (ICML 2025); following this practice, we evaluate DualQuant broadly: six models, four 4-bit formats plus 8-bit, perplexity, zero-shot, and reasoning/code tasks, and native INT4/NVFP4 deployment. Across these settings, DualQuant is the strongest calibration-free method in almost all cases and matches or surpasses calibrated SmoothQuant in most settings, all without any calibration data.
Q1: Memory and throughput vs. other quantization schemes.
Beyond the BF16 comparisons (Tables 11–12 of the original submission, now Tables 15–16), we extended packed INT4 inference to native NVFP4 W4A4 on RTX 5090 (extended Tables 3 and new table 6), and directly compare DualQuant against calibrated SmoothQuant and AWQ+ using the same packed FP4 GEMM backend.
Llama-3.1-8B: Native NVFP4 inference (RTX 5090, batch 16).
*Calibrated. DualQuant achieves better PPL at matched efficiency, without calibration data.
Additional Qwen3-14B native deployment (rebuttal-only), packed NVFP4 W4A4 on RTX 5090. Model-weight and runtime-overhead memory.
Qwen3-14B: End-to-end GPU memory and throughput vs. BF16. (Memory includes workload activations and 16-bit KV cache where applicable):
Overall, DualQuant scales to Qwen3-14B with up to 3.07× speedup and 2.70× memory reduction over BF16, while the activation quantization, including fused β, uses only 1.9–4.2% of GPU time.
Q2: Definition of “calibration-free.”
In DualQuant, calibration-free means calibration data-free: both α and β are determined exclusively from model weights, without input samples, forward passes, or activation statistics. Although we optimize these scales, the optimization requires no calibration data. We clarified this distinction in the revised Introduction.
Q3: QuaRot and KV-cache quantization.
In all experiments, only linear-layer weights and activations are quantized; KV caches and attention computations remain 16-bit. For QuaRot, we disable its optional KV-cache quantization and use the same W4A4 setting as DualQuant and other baselines. We now state this explicitly in Section 5.1 and Appendix F.