- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MirrorQuant, a PTQ framework that parameterizes learnable rotations as a fixed orthogonal base followed by a short chain of Householder reflections. This reduces learnable state from quadratic to linear complexity while preserving exact orthogonality. For the non-foldable MLP rotation R 4 , MirrorQuant combines an FHT with learned reflections to enable efficient online execution. The paper also introduces projection-guided learning and progressive calibration. Experiments report consistent improvements over DartQuant across multiple model families and low-bit settings.
Critical strengths
- The paper addresses an deployment bottleneck of dense learned rotations.
- The Householder parameterization is simple, theoretically well motivated, and exactly orthogonal.
- The distinction between foldable and online rotations is clearly presented.
- Projection-guided learning has a reasonable motivation and is supported by ablations.
Critical weaknesses
-
Missing comparison with Kronecker-structured online transformations. A FlatQuant-style Kronecker transform is a direct alternative for R4 . It may offer greater expressiveness through learnable affine transformations and can be implemented using small, Tensor-Core-friendly GEMMs fused with activation quantization.
-
Outdated and insufficient evaluation for W4A4. The evaluation mainly uses perplexity and legacy short-form zero-shot benchmarks such as ARC, HellaSwag, PIQA, LAMBADA, and MMLU. These tasks do not measure error accumulation during long chain-of-thought generation. Recent studies [1][2] show that W4A4/W4A4KV4 may preserve conventional QA accuracy while substantially degrading mathematical, scientific, and coding reasoning. [1] Quantization Hurts Reasoning? [2] PTQ Baselines for Reasoning LLMs on Ascend
-
Lack of reasoning-model evaluation. The paper evaluates base Llama and Qwen models but not modern reasoning models such as DeepSeek-R1-Distill or QwQ. Calibration on short WikiText-2 sequences may also fail to represent long-CoT activation distributions. Therefore, the current results do not demonstrate that MirrorQuant preserves reasoning ability under aggressive activation quantization.
Questions for authors
- How does MirrorQuant compare with a parameter-matched FlatQuant-style Kronecker transform when only R 4 is replaced and all other settings are fixed?
- Are the Householder/FHT operations fused with activation quantization? How would their latency compare with a fused Kronecker GEMM kernel?
- Can the authors evaluate W4A4KV4 on reasoning models like DeepSeek-R1-Distill or QwQ using MATH-500, AIME, GPQA-Diamond, and LiveCodeBench?
7 comments
Response to Reviewer 4Jc8 (2)
W2. Outdated and insufficient evaluation for W4A4 about mathematical, scientific, and coding reasoning tasks. W3. Lack of reasoning-model evaluation. Q3. Evaluation on reasoning models like DeepSeek-R1-Distill or QwQ using MATH-500, AIME, GPQA-Diamond, and LiveCodeBench.
Thank you for recommending these studies. We agree that the original evaluation on base models and short-form tasks left long-CoT reasoning insufficiently tested. Quantization Hurts Reasoning? shows that low-bit reasoning performance varies across models and tasks, and the Ascend study reports long-generation failures in several W4A4KV4 configurations. These findings motivated us to add direct generation-based evaluation on a reasoning model. We will cite both studies and clarify the scope of the original evaluation in the revision.
Reference: Quantization Hurts Reasoning?, COLM 2025 · PTQ Baselines for Reasoning LLMs on Ascend, arXiv
We retained the short WikiText-2 calibration and evaluated DeepSeek-R1-Distill-Qwen-7B on the four requested benchmarks. We use W4A4KV16 to isolate the effects of 4-bit weights and activations. MirrorQuant's rotation learning addresses a complementary part of the quantization pipeline to dedicated KV-cache methods such as PM-KVQ and TurboQuant. Keeping the cache at 16 bits provides a direct test of weight–activation quantization after short calibration.
Reference: PM-KVQ, ICLR 2026 · TurboQuant, ICLR 2026
We followed the evaluation setup of the Ascend study and ran BF16, FlatQuant, and MirrorQuant locally on Ascend 910C under the same protocol. Scores are in percent, with the better W4A4KV16 result bolded. is the MirrorQuant score minus the FlatQuant score in percentage points. Evaluation protocol
| Method | MATH-500 ↑ | AIME-120 ↑ | GPQA-Diamond ↑ | LiveCodeBench ↑ |
|---|---|---|---|---|
| DeepSeek-R1-Distill-Qwen-7B | 93.20 | 48.33 | 48.48 | 33.58 |
| FlatQuant | 92.00 | 37.50 | 43.43 | 28.73 |
| MirrorQuant | 92.80 | 38.33 | 46.46 | 29.48 |
| +0.80 | +0.83 | +3.03 | +0.75 |
Under W4A4KV16, MirrorQuant improves on FlatQuant across all four benchmarks, by 0.80 pp on MATH-500, 0.83 pp on AIME-120, 3.03 pp on GPQA-Diamond, and 0.75 pp on LiveCodeBench. After short-sequence calibration, MirrorQuant stays within 0.40 pp of BF16 on MATH-500. The gaps to BF16 are 2.02 pp on GPQA-Diamond and 4.10 pp on LiveCodeBench, with the largest gap of 10.00 pp on AIME-120.
We will discuss the two reasoning studies in Related Work and report the W4A4 reasoning experiments in Appendix.
We hope the above responses have adequately addressed your concerns. Please feel free to contact us if you have any further questions. If not, we would greatly appreciate your consideration for a higher score.
Thank You for Your Constructive Review — Discussion Open Until November 18
Dear Reviewer 4Jc8,
We sincerely appreciate your detailed review and the related studies you recommended. Your suggestions helped us identify where the evaluation could be strengthened. We have shared the additional results and explanations in our point-by-point responses and hope they address your concerns.
If any questions remain about our responses, we sincerely hope to have the opportunity to continue the discussion with you before it closes on November 18. We would be very happy to provide further analysis, run additional experiments, or explain in more detail any aspect you would like to discuss.
Thank you again for your thoughtful review and constructive feedback!
Best regards,
Authors of Submission 2369
Response to Reviewer 4Jc8 (1)
Thank you for the careful review and for recognizing the deployment bottleneck at the online MLP interface, the exact orthogonality of the parameterization, and the motivation for projection guidance. Your suggestions on fused Kronecker alternatives and reasoning-model evaluation helped us extend the experiments in both directions.
Thank you for suggesting fusion with activation quantization. Following this suggestion, we implemented fused execution of MirrorQuant's FHT/Householder R4 and per-token A4 quantization. We agree that FlatQuant's Kronecker-factorized affine transform is a relevant structured alternative and compared it using a fused Kronecker GEMM+A4 kernel.
Reference: FlatQuant, ICML 2025
For all three models, we hold R1/R2 and the quantization settings fixed, replace only the online R4 with a FlatQuant-style Kronecker transform matched to MirrorQuant’s R4 in trainable parameter count, and retrain it during calibration on the corresponding model. Both alternatives use their fused implementations under W4A4KV4. The Δ column compares accuracy with the Kronecker baseline within each model. E2E denotes end-to-end latency, and Op. denotes the R4+A4 operator path, excluding the down-projection GEMM. Bold values mark the better result within each model, including ties.
MirrorQuant improves accuracy by 0.13–0.31 pp across all three models, with end-to-end prefill and decode latency differing by less than 1% from Kronecker in every comparison. Operator-level results depend on the model. Kronecker has lower prefill latency on Llama-2-7B, while MirrorQuant has lower prefill and decode latency on Llama-2-13B and Llama-3-8B. These fused comparisons show a small accuracy benefit for MirrorQuant at closely matched end-to-end latency.
We will report the fused Kronecker comparison and describe the fusion and timing protocol in Appendix.