- Rating
- 3(+1)
- Confidence
- 3(−1)
- Informativeness
- 3(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this paper, the authors propose a unified Hessian-guided framework for LLMs PTQ named HZ-PTQ. HZ-PTQ introduces a novel approximation method to estimate the downstream impact of weight and activation quantization errors, and then develop a common optimization method based on this approximation.
Critical strengths
- Using output sensitivity to guide quantization optimization is theoretically meaningful.
- The output Kronecker approximation makes the downstream-sensitivity task computationally tractable.
- The method achieves consistent competitive performance on four model families and diverse configurations.
Critical weaknesses
- The derivations rely on several approximations, including rank-one Fisher factorization, independence between weight and activation quantization errors and negligible clipping effects. However, these assumptions may not hold in some cases. The paper does not sufficiently evaluate how accurately these approximations reflect the actual downstream loss.
- Eq11 derives a quantization-error objective that depends on both transformation-dependent quantization ranges and gradient sensitivities. However, Eq. (12) optimizes only the gradient terms. Therefore, the transformation is not necessarily optimal for the objective in Eq11. The paper lacks of justification or empirical validation of this simplification.
- Calibration cost is an important metric to evaluate a PTQ method. However, this paper lacks of reporting the calibration time, peak memory consumption and computational overhead compared with existing PTQ method.
Questions for authors
- Can the authors evaluate the approximation error in W1?
- Can the authors evaluate whether the transformations actually minimize the original objective in Eq11 in W2?
- Please provide the calibration cost of the proposed method compared with baselines in W3?
Response to Reviewer QtFL
We thank the reviewer for the precise questions, for finding our use of output sensitivity "theoretically meaningful" and the Output Kronecker approximation "computationally tractable", and for noting the "consistent competitive performance".
1. Approximation Validation
We extend the comparison in Figure 1 to more models and lower bit widths, and add the decomposition in Eq. 8, which assumes independent weight and activation errors (see Table A in our General Response). The factorized proxy (Eq. 6) tracks the full output-Fisher proxy (Eq. 5) closely, and the decomposition (Eq. 8) leaves the correlation with output KL unchanged. This holds across models (0.97–0.99) and down to W2A8 (0.71), where the commonly used MSE shows no correlation (−0.02). The decomposition of activation and weight quantization error is further motivated by existing work in the literature [1].
We recognize that our transform derivation neglects the effect of clipping, however, the additional results reported in Table D confirm that our approximate solution contributes to the reduction of the Fisher-weighted quantization error. We will add these additional references and results to the paper.
2. Transform Optimality
Eq. 12 optimizes only the gradient terms of Eq. 11, but those terms are invariant to orthogonal rotations. We therefore first find the transform M^ that minimizes Eq. 12, then apply a Hadamard rotation. Being orthogonal, it leaves the value of Eq. 12 unchanged while reducing the quantization ranges in Eq. 11. To check that this approximately minimizes the target objective, we compare the value of the factorized Fisher-weighted error under our transform, the Hadamard transform (QuaRot) and the CAT [1] transform (Table D). Our transform attains the lowest average Fisher-weighted error. We will add these results to the paper.
Table D: Average layer-output error under each transform on Llama 3.2 1B Instruct with W4A4 rounding after the transform.
The values are reported in decibels [dB] (10⋅log10 value) for readability. Note that while CAT attains the lowest L2 error., our transform reaches lower values for the Fisher-weighted proxy.
3. Calibration Time and Memory
The memory and runtime costs of our method are reported in Table B of our General Response. Our method takes 158 min and 32 GB on Llama 3.2 1B Instruct, versus 96 min and 32 GB for YAQA, 127 min and 25 GB for FlatQuant+GPTQ, and 74 min and 25 GB for CAT+GPTQ. We will include them in the paper. Our method extends YAQA's [2] Kronecker-factored curvature to activation quantization, transform selection and range setting. It therefore inherits the overhead of estimating the Kronecker-factored curvature, here with respect to the layer outputs, which takes time and compute that are essentially equivalent to YAQA. Transform and range selection are performed layer by layer and have low peak memory. Concurrent and complementary work, [3], speeds up two-sided LDLQ and proposes a lower-memory procedure for computing the Kronecker-factored Hessians. These improvements apply directly to our method and would likely reduce this overhead.
References
[1] M. Federici, B. van Breugel, P. N. Whatmough, M. Nagel. "Dissecting Quantization Error: A Concentration-Alignment Perspective". ICML 2026.
[2] A. Tseng, Z. Sun, C. De Sa. "Model-Preserving Adaptive Rounding". ICML 2026.
[3] J. Birnick, R. Saab. "FastKron: Efficient Quantization with Kronecker-Factored Hessians". arXiv:2608.06291, 2026.