- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper answers the question whether exact FP equivalence is necesary when learning rotation transformation matrix throughout the quantization process. The paper propose a QAT framework that generate a fixed equivalence-preserving Hadamard matrix at initialization, then use global quantization-aware distillation (QAD). The update are optimized directly into the merged weight instead of updating the rotation matrix to preserve the equivalence. Experiment for W4A4 setting show improvements across many architecture over rotation only optimization.
Critical strengths
-
Consistent improvement: the proposed method improve performance across 4 architectire, using perplexity and downstream zero-shot accuracy.
-
The motivation is intuitive and makes sense. The experiments about direct-weight optimization and rotation only are meaningful.
Critical weaknesses
-
Limited novelty. The proposed algorithm basically combines fixed Hadamard rotation preprocessing with normal QAF framework using STE and KL loss. Moreover, while the motivation make sense, existing QAT methods already optimize full-precision weights without functional equivalence (e.g. LLM-QAT, RCP). The two proposition are fairly simple, so theoretical contribution isn't strong.
-
The experiment results are not unexpected, because direct-weight optimization has more degree of freedom for optimization, while the rotation-only optimization has constraints.
-
Lack of competisons with existing QAT baselines. The paper only compare with same baseline but using rotation-only optimization.
Questions for authors
Please answer the concerns in weakness section
3 comments
Follow up
Dear Reviewer R5FH,
As the discussion period is approaching its end, we would like to kindly follow up and ask whether our rebuttal has addressed your concerns. We have provided additional experimental results and clarifications in response to your comments and would greatly appreciate your feedback.
Best regards, Authors
Thank you for the rebuttal. The new experiments are helpful, solve some of my concerns.
But my novelty concern remains. The gap seem a bit too narrow for me, and it is unclear whether the gains would persist against more flexible transformations, such as affine transformations that preserve the model’s full-precision outputs. While existing QAT methods are difficult to compare across settings, the authors could still test an existing QAT method with and without the proposed initialization under the same protocol. This would show whether the initialization also helps beyond the paper’s own QAD recipe.
I am keeping my original rating.
Response to Reviewer R5FH
We thank the reviewer for recognizing the improvements across four architectures and the value of the direct-weight versus rotation-only comparison. We address these three concerns in turn.
W1 / Q1(a): Novelty is limited, and existing QAT already updates full-precision weights without equivalence.
Answer: Latent-weight QAT and fused-weight adaptation predate this work. The narrower question here is whether, after the same equivalence-preserving initialization, retaining exact equivalence helps or restricts the finite-budget endpoint.
The submission does not claim that direct fused-weight adaptation is new or first. LLM-QAT/BitDistiller/EfficientQAT perform latent-weight QAT, and RCP fuses rotation into the weights while jointly optimizing them with learnable clipping and non-uniform quantization parameters. Our distinction is to compare, from the same equivalence-preserving starting representation, a parameterization that maintains exact equivalence against one that releases it. The propositions formalize why releasing the constraint enlarges the admissible search space and why strict improvement is possible; the experiments test whether that distinction matters under a finite optimization budget.
W2 / Q1(b): The direct-weight arm has more degrees of freedom, so the improvement is expected.
Answer: Releasing exact equivalence does add optimization degrees of freedom. That is precisely the design variable we test: do those degrees of freedom produce a reproducible finite-budget benefit from the same rotation-derived initialization, or does greater freedom merely appear beneficial by construction?
We do not claim parameter-count matching or universal superiority. We measure the finite-budget gain, reproducibility across seeds, and whether the drift exhibits error compensation rather than arbitrary movement.
The interaction interval is a sequence-paired 95% bootstrap interval over the same held-out C4 set. The seed study covers LLaMA-2-7B and TinyLlama-1.1B only; no LLaMA-3 seed-variance claim is made. It uses forward KL, 800/32 C4 train/validation sequences, 500 updates, batch 64, 32,000 rank-samples, W4A4/G128/nohad evaluation, initializer seed 0, three predeclared training/data-order seeds, and the final checkpoint for both arms.
The table reports rows with all three seeds completed. TinyLlama uses a similar budget and evaluator family but a different model and setup; it supports the observed direction, not model-scale generalization. The seeds vary training and data-order randomness only, so initializer variance is not estimated.
W3 / Q1(c): Existing QAT baselines are missing.
Answer: External QAT methods provide important context, but they cannot serve as protocol-matched controls for the specific intervention studied here. We therefore distinguish method positioning from empirical comparison and test the equivalence constraint using a rotation-constrained control with the same initializer.
RCP targets W2/3A4KV4 with online rotations and a learnable non-uniform quantizer, unlike our W4A4/G128/nohad setting with a fixed uniform quantizer and matched C4/update budget. LLM-QAT/EfficientQAT likewise differ in data, bit configuration, trainable objects, and compute. Because we do not have protocol-compatible local reproductions of these external methods, we do not present paper-reported numbers as same-setting baselines; any contextual comparison is source-separated with model/data/bits/quantizer/graph/compute differences explicit. We do not propose direct-weight QAT as a new algorithm, nor claim that additional degrees of freedom always improve performance. Our claim is the narrower finite-budget consequence of maintaining exact equivalence after the same rotation-derived initialization.