acceptodds
Under review as a conference paper at ICLR 2027

Few Bits, One Law: Toward W2A4KV2

Abstract

Extreme low-bit LLM compression is most challenging when weights, activations, and KV caches are quantized together: their distributions differ, and quantization errors interact throughout the network. We introduce CanonQ, a unified quantization-aware training framework that addresses these challenges by separating source canonicalization from task-aware adaptation. Fixed rotations and energy normalization map heterogeneous tensor sources to canonical coordinates, enabling frozen Gaussian-reference codebooks to be reused across layers and models. Joint training then adapts the network to the coupled errors of weight, activation, and cache quantization within a common scalar/vector interface. We bound frozen-codebook transfer error and local task loss, and derive an exact normalization-aware straight-through Jacobian that links quantization distortion to gradient bias. The strongest gains arise under joint W2A4KV2 compression: across LLaMA3-1B/3B/8B, CanonQ-Omni achieves up to 14.28× lower WikiText-2 perplexity and up to 57.9% higher mean zero-shot accuracy than prior state-of-the-art and representative quantization baselines. The benefits extend to Qwen3-1.7B, code generation, and mathematical reasoning: on instruction-tuned MobileLLM-Pro-1B at W2A16KV16, CanonQ achieves relative improvements of 41.7% in HumanEval pass@1 and 39.1% in GSM8K exact match over the strongest evaluated quantization baseline.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Model Quantization, Low-Bit Quantization, Codebook Quantization, Quantization-Aware Training

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_kccgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces CanonQ, a QAT framework for compressing weights, activations, and KV cache (2-4 bits). It applies a Hadamard–Diagonal–Permutation transform plus normalization to map to each tensor block into canonical coordinates that approximate a standard Gaussian. Then, each block is quantized with a frozen scalar or vector codebook. The codebook only depends on bitwidth and d (dimensionality hyperparameter), so it is reused across layers and models. QAT adapts the network to the combined quantization errors. Two variants are compared: CanonQ-W (new transform applied to weights only) and CanonQ-Omni (applied to all 4 tensor types).

Experiments are reported on Llama3-1B (primarily), as well as Llama3-3B/8B, Qwen3-1.7B, and MobileLLM-Pro-Instruct-1B. The main evaluation is performed on wikitext-2 and 8 standard zero-shot tasks. It is briefly extented to code generation and reasoning tasks (for a weight-quantization case only).

Under aggressive quantization (W2A4KV2), CanonQ-Omni shows consistent and remarkable perplexity/accuracy gains compared to the reported quantized baselines.

Critical strengths
  • the proposed quantization scheme uses a simple and reusable design: same codec construction applies to all tensor type (W/A/KV), and the resulting frozen tables are reused across layers and models. This design favors practical applicability of the technique.
  • CanonQ-Omni is shown to be most successful in the hardest regime when activations and KV cache are quantized. The QAT baselines degrade dramatically while CanonQ-Omni retains reasonable performance (not achieving iso-accuracy but a remarkable improvement nonetheless)
Critical weaknesses
  • novelty is limited: rotating, normalizing, using Gaussian-optimal codebooks is close to other works such as QuIP# [1], TurboQuant [2], HIGGS [3]. The primary contribution is applying known techniques together to all four tensor types (W, A, KV) in combination with QAT
  • the CanonQ-W variant is a useful intermediate step between the ParetoQ baseline and CanonQ-Omni, showing the impact of the proposed approach applied only to weights. The results is that CanonQ-W beats ParetoQ only under some quantization settings, localizing the most relevant gains to activations and KV (keeping in mind that all runs a single seed, an additional limitation of this work). However, the W-to-Omni step introduces several changes together: rotation, norm-based scaling, an extra quantization level, Gaussian-optimal level placement. Intermediate QAT baselines, especially rotation + uniform quantizers on activations/KV, would have helped isolating the contribution of rotations (potentially very impactful) from the Gaussian-optimal codebook
  • arguably the main barrier to QAT adoption is its compute budget. The paper has no kernels, throughput, or memory measurements, so the impact of on-the-fly rotation, normalization and nearest-codeword lookup is not assessed. At least, wall-clock training time or GPU-hours per model size / method should have been reported, along with CanonQ per-step overhead relative to ParetoQ.

[1] Tseng, Albert, et al. "Quip#: Even better llm quantization with hadamard incoherence and lattice codebooks." Proceedings of machine learning research 235 (2024): 48630.

[2] Zandieh, Amir, et al. "Turboquant: Online vector quantization with near-optimal distortion rate." International Conference on Learning Representations. Vol. 2026. 2026.

[3] Malinovskii, Vladimir, et al. "HIGGS: Pushing the limits of large language model quantization via the linearity theorem." Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Questions for authors

As per Critical Weaknesses, can the authors please comment on:

  1. isolating contribution of rotations from codebook on activations / KV
  2. training time, training steps, quality as a function training steps, CanonQ overheads
2 comments
AuthorsOpenReview ↗

Novelty, Training Efficiency, and Convergence

We thank the reviewer for their careful assessment. We are encouraged by their recognition of CanonQ's reusable W/A/KV design and its gains under joint low-bit quantization. We address novelty, component contributions, and training cost below.

1. Novelty and supporting evidence

To our knowledge, CanonQ is the first QAT framework to jointly quantize weights, activations, and KV caches using a shared, frozen Gaussian-reference codebook design. Each codebook is designed once per bitwidth and tuple dimension and reused across layers and models. QuIP# and HIGGS focus on weight compression, while TurboQuant addresses online vector quantization, including KV caches.

The challenge is making fixed codebooks work across different tensor distributions that change during training. Weight updates alter activations and caches, and their quantization errors interact. CanonQ uses HDP and energy normalization to make the reference codebooks reusable, then jointly trains the network with all W/A/KV quantizers active. Theorems 1–3 connect distribution mismatch, reconstruction error, task loss, and gradient error (Secs. 3–4).

The W2A4KV2 results demonstrate the benefit of this joint design:

ModelConfigurationPPL ↓8-task mean ↑
LLaMA3-1BVQLLM-QAT-d476.5142.32
LLaMA3-1BCanonQ-W-d164.4042.49
LLaMA3-1BCanonQ-Omni-d115.4253.56
LLaMA3-3BVQLLM-QAT-d452.8242.07
LLaMA3-3BCanonQ-Omni-d211.1660.04

Keeping the canonical weight quantizer fixed, extending CanonQ-W to CanonQ-Omni improves 1B accuracy by 11.07 points and reduces PPL from 64.40 to 15.42 (Sec. 5.2, Table 1).

2. Stronger baselines and controlled component ablations

At matched 240K updates (31.5B tokens), CanonQ achieves 15.42 PPL / 53.56 mean accuracy, versus 27.57 / 48.04 for ParetoQ + QuaRot-style A/KV quantization: a 5.52-point accuracy gain.

To separate the component contributions, the following 120K-update controls keep the canonical weight codec, training settings, and alphabet cardinality fixed (16 A4 levels and 4 KV2 levels):

A/KV levelsScaleHDPPPL ↓8-task mean ↑
UniformAbsmaxOff22.8745.08
UniformAbsmaxOn23.3349.16
UniformEnergyOn17.1151.21
GaussianAbsmaxOn19.0250.82
GaussianEnergyOff42.3340.79
GaussianEnergyOn16.2752.28

Gaussian codebooks contribute beyond rotation and normalization. With HDP and energy fixed, Gaussian levels improve PPL from 17.11 to 16.27 and accuracy from 51.21 to 52.28. Under absmax and HDP, they also improve both metrics: 23.33 → 19.02 PPL and 49.16 → 50.82 accuracy. The complete configuration achieves the best results across all six controls (Sec. 5.4, Table 3; Appendix C.2).

Separately, three 240K CanonQ runs on 1B achieve 53.61 ± 0.67 mean zero-shot accuracy (mean ± sample SD, n = 3; Appendix C.2).

3. Training cost and convergence

On eight A100-80GB GPUs, with 131,072 tokens per update, the measured W2A4KV2 training costs are:

ConfigurationSeconds / updatePeak GB / GPU
ParetoQ + uniform A/KV1.163260.97
ParetoQ + QuaRot-style A2.563850.20
CanonQ-Omni2.074947.71

Measurements use 600 updates, excluding the first 100 for warmup. CanonQ uses 19.1% less time per update and 5.0% less peak memory than the rotated baseline; memory is also 21.7% lower than the uniform baseline (Appendix C.8).

Quality at matched training checkpoints is:

UpdatesParetoQ + QuaRot PPL ↓ParetoQ + QuaRot Acc. ↑CanonQ PPL ↓CanonQ Acc. ↑
15K37.1244.0117.8650.11
60K30.5946.3017.1851.60
120K27.3147.7316.2752.28
240K27.5748.0415.4253.56

CanonQ at 15K updates already exceeds the rotated baseline at 240K on both metrics, using one-sixteenth as many updates. It also leads at every matched checkpoint (Appendix C.1).

AuthorsOpenReview ↗

Follow-up on our rebuttal

Thank you again for your review. As the discussion period draws to a close, we would appreciate your feedback on whether the additional results in our rebuttal address your concerns.

Reviewer_6ZCaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the degradation of LLMs under joint extreme low-bit quantization of weights, activations, and KV cache, and proposes a unified QAT framework, CanonQ. Fixed HDP orthogonal transforms and energy normalization map W/A/K/V into approximately Gaussian canonical coordinates; a frozen reference codebook is reused across layers and models, and joint QAT adapts the network to the interacting quantization errors across sources. The authors further establish theoretical bounds linking codebook transfer error, local task loss, and STE gradient bias. Experiments cover LLaMA3-1B/3B/8B, Qwen3-1.7B, and MobileLLM-Pro-Instruct-1B. At W2A4KV2, CanonQ-Omni achieves up to 14.28× lower WikiText-2 perplexity and a 57.9% relative improvement in mean zero-shot accuracy over ParetoQ(Table 1).

Critical strengths
  1. CanonQ handles W/A/KV quantization errors through a unified canonicalized representation and joint QAT, achieving substantial gains under extreme low-bit joint quantization and demonstrating consistent effectiveness across model scales (Table 1)
  2. Section 4 provides a fairly complete theoretical analysis, and Figure 2 and Tables 7–8 further validate the effectiveness of the canonicalized representation, QAT adaptation, and the KV quantization path.
Critical weaknesses
  1. Existing ablations change multiple design factors simultaneously and do not yet adequately separate the independent contributions of HDP rotation, energy normalization, and the frozen codebook.
  2. The connection between theory and experiments remains insufficient, particularly for the joint-distribution deviation in Theorem 1 and the STE gradient bias in Theorem 3, which lack direct empirical validation.
Questions for authors
  1. Do the authors have ablation results that vary only the HDP rotation, normalization, or codebook design while keeping the quantization configuration and training conditions fixed? How can the independent contributions of these components be further distinguished?
  2. Theorem 1 relies on the Wasserstein deviation of the joint vector distribution, whereas the existing experiments mainly report scalar-marginal metrics. Do the authors have statistical results on joint-distribution deviation or quantization distortion that support codebook transfer analysis in the theory?
  3. Theorem 3 establishes an STE optimization guarantee for a smooth reference objective. Can the authors further clarify how this result relates to practical hard-quantized QAT, and whether there are experimental observations supporting the link between gradient bias and quantization distortion?
4 comments
AuthorsOpenReview ↗

Controlled Ablations and Empirical Support for the Theory

We thank the reviewer for the detailed assessment of our unified formulation, empirical gains, and theory. The additional experiments separate the effects of rotation, scaling, and quantization levels, and test the theoretical predictions.

1. Separating rotation, scale, and level placement

The following LLaMA3-1B W2A4KV2 controls use 120K updates, the same canonical weight codec, and full alphabet cardinality: 16 A4 levels and 4 KV2 levels.

A/KV levelsScaleHDPPPL ↓8-task mean ↑
UniformAbsmaxOff22.8745.08
UniformAbsmaxOn23.3349.16
UniformEnergyOn17.1151.21
GaussianAbsmaxOn19.0250.82
GaussianEnergyOff42.3340.79
GaussianEnergyOn16.2752.28

With HDP and energy normalization fixed, Gaussian levels improve PPL from 17.11 to 16.27 and mean accuracy from 51.21 to 52.28 (+1.07 points). Under absmax and HDP, Gaussian levels also improve both metrics, adding 1.66 accuracy points. Energy normalization adds 2.05 points under uniform levels and HDP; adding HDP under Gaussian levels and energy reduces PPL from 42.33 to 16.27 and adds 11.49 points.

Multiple seeds. CanonQ achieves 53.61 ± 0.67 8-task mean zero-shot accuracy on LLaMA3-1B (mean ± sample SD, n = 3). A full QAT run requires 240K updates, approximately 31.5B tokens, and eight GPUs, making exhaustive replication across the full experimental grid costly. Controlled comparisons nevertheless match training budgets and hold the other quantizer components fixed.

2. Frozen-codebook transfer: Theorem 1

Theorem 1 allows the actual tensor distributions to differ from the Gaussian reference. We test the scalar case, d = 1, where each coordinate is a complete tuple. All 12 sampled W/A/K/V checks at 15K, 60K, and 240K satisfy the tested reconstruction-error bound. Two examples using the deployed BF16 codebooks at 240K are:

SourceExcess root distortionMismatch term, 2Δ₁
Weights0.05610.2317
Keys0.00120.2007

With matched held-out scale tuning, canonical coding also reduces weight RMS error relative to tuned uniform quantization:

CheckpointReduction in weight RMS error
15K0.56%
60K0.65%
240K0.92%

At 240K, the frozen codebook achieves 0.32 points higher mean accuracy and 0.16 lower PPL than evaluation-only refitting with model weights fixed, supporting reuse of the frozen codebook (Appendix C.3).

3. Practical QAT gradients: Theorem 3

We compare the hard-quantized model's STE gradients with the tested reference gradients at the same checkpoint, using 64 paired held-out batches per checkpoint:

CheckpointMean cosine ↑Minimum cosine ↑
15K0.99380.9877
60K0.99170.9887
240K0.99050.9783

A cosine of 1 denotes identical gradient directions. Mean cosine exceeds 0.99 at every checkpoint, and every tested batch has cosine at least 0.9783, supporting close directional agreement throughout the measured trajectory. Appendix C.4 defines the reference gradients and reports the numerical checks relating quantization distortion to local backward error in Eq. (13).

Reviewer_6ZCaOpenReview ↗

Thanks for your rebuttal

Thanks for adding the controlled runs. With the training budget matched on LLaMA3-1B W2A4KV2, it's easier to see what HDP, energy normalization, and A/KV level placement each contribute. The close agreement in gradient direction on held-out batches and the local backward-error checks also help explain how the reference-gradient analysis applies to hard-quantized QAT.

The scalar checks support the transfer analysis in Theorem 1. I'd still like to see the joint Wasserstein mismatch and transfer distortion for d > 1. The refit after training is a useful check on reusing the codebook at inference. With the weights held fixed during that comparison, I'm still unsure how much freezing the codebook during QAT contributes.

Combined with other reviewers' assessments, I will keep my rating at 2.

AuthorsOpenReview ↗

Follow-up on Vector Codebooks and Freezing During QAT

Thank you for the follow-up. We are glad the matched ablations clarified each component’s contribution, the gradient checks clarified how Theorem 3 relates to hard-quantized QAT, and the scalar results addressed your question about Theorem 1 at d=1d=1.

We will post an update tomorrow on both remaining questions.

  1. Vector codebook transfer. On the LLaMA3-3B dW=2d_W=2 checkpoint, we will measure the joint Wasserstein mismatch and compare Gaussian and uniform-product codebook distortion on the same weight tuples at the same bit budget.
  2. Freezing during QAT. Freezing preserves the codebook’s Gaussian-reference fit. From the same 15K LLaMA3-1B checkpoint, both runs will update model weights for 1,000 steps on identical batches. One will also update the codewords. We will compare PPL, 8-task mean accuracy, and Gaussian-reference distortion.

Regarding other reviews, we ask that the assessment focus on the specific technical arguments and revised evidence. A score alone does not identify a concern we can address. We have raised our concern about the basis for Reviewer 9TCx’s rating with the Area Chair. If another technical point affected your recommendation, we would welcome the opportunity to address it directly.

AuthorsOpenReview ↗

Follow-up on Vector Codebook Transfer and Frozen vs. Learnable QAT

Thank you for the follow-up. We ran two experiments to address your remaining questions.

1. Vector codebook transfer

On LLaMA3-3B W2A4KV2 with dW=2d_W=2, we measured the normalized joint Wasserstein-2 distance between the quantizer's two-dimensional weight tuples and the Gaussian reference. We evaluated both codebooks on the same tuples at the same bit budget. Values are mean ± SD across draws.

MeasurementResult
Normalized joint W2W_2, weight tuples vs. Gaussian0.1496 ± 0.0169
Normalized joint W2W_2, two independent Gaussian samples of matched size0.1392 ± 0.0094
Root distortion on weights, CanonQ0.3257 ± 0.0095
Root distortion on weights, uniform-product codebook0.3548 ± 0.0104

The measured joint distance was close to the Gaussian sampling baseline. On the actual weight tuples, CanonQ reduced root distortion by 8.20% relative to the uniform-product codebook.

2. Frozen vs. learnable codebooks during QAT

Both runs resumed from the same 15K LLaMA3-1B W2A4KV2 checkpoint and continued updating the network weights on identical batches. In the learnable run, we kept the same hard nearest-codeword assignment and cross-layer sharing. At each update, each selected codeword accumulated gradients from the language-model loss across its uses and was updated alongside the network weights. We report the follow-up 5K updates of the planned 15K continuation.

Global stepFrozen PPL ↓Learnable PPL ↓Frozen 8-task mean ↑Learnable 8-task mean ↑
15K (shared start)17.8617.8650.1150.11
16K17.8218.3449.3648.25
18K17.6518.4250.1048.21
20K17.8418.0951.3849.54

Freezing yielded lower WikiText-2 PPL and higher 8-task mean accuracy at all 3 checkpoints after branching. At 20K, freezing improved the eight-task mean by 1.84 points and lowered PPL by 0.25 under the tested update rule.

Benefits of frozen codebooks

  • Theoretical support. Theorem 1 (Sec. 4.1) bounds the change in a fixed codebook's root distortion between the Gaussian reference and model tensors in terms of their joint Wasserstein distance. Freezing preserves the reference codewords throughout QAT.
  • Training efficiency. Frozen codewords require no gradients, optimizer state, or parameter updates of their own.
  • Reuse and storage. For each bitwidth and tuple dimension, one table can be reused across layers and models without refitting. Storage for the tables stays constant as model size grows.

These results support our choice to freeze CanonQ's reference codebooks. We are extending the comparison to additional models. Designing more effective ways to learn shared codebooks during joint W/A/KV QAT is an interesting research question in its own right and warrants a separate study.

We hope these results address your remaining questions.

Reviewer_9TCxOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper proposes CanonQ, a quantization-aware training (QAT) framework for joint low-bit quantization of weights, activations and KV caches. The key idea is to separate source canonicalization from task adaptation.

Critical strengths

The paper separates source-specific transform and normalization from the a source-independent frozen codebook. This gives a single scalar/vector interface for W, A, K and V. A frozen table that depends only on (b, d) is simple and reusable.

The CanonQ-W vs. CanonQ-Omni comparison locates the gain in the activation and cache paths. The zero-update control shows the codec alone is not sufficient, since PTQ PPL is above 7000.

Critical weaknesses

The most recent method actually compared is ParetoQ (early 2025). The others (GPTQ, QuaRot, KIVI, RTN) are 2023–2024 work. Moreover, the main “ParetoQ” baseline at W2A4KV2 is a composite the authors built themselves: ParetoQ weights plus a naive absmax uniform A/KV quantizer. The headline claims (up to 14.28× lower PPL, 57.9% higher accuracy) are measured against this composite, not against a method designed for joint compression. Authors should includs more SoTA methods.

The paper's format is like AI-generated. Section 6 (Discussion and Limitations) takes main-text space but mostly restates Sec. 5. It would be better placed in the appendix or limitations checklist.

The paper’s motivation is deployment efficiency, yet all results are accuracy-only.

The main experiments use LLaMA3-1B/3B/8B (2024). Only W2A4 is reported on Qwen3-1.7B, without KV2.

Questions for authors

Refer to weaknesses.

3 comments
AuthorsOpenReview ↗

Updated Baselines, Inference Efficiency, and Generalization

We thank the reviewer for their time and careful review. We are encouraged by the recognition of CanonQ's reusable W/A/KV interface and the distinction between weight-only and joint quantization. We address the concerns below.

1. Joint baselines and 2026 comparisons

Our joint baseline combines ParetoQ weights with uniform A/KV quantization, active throughout QAT and evaluation. Adding QuaRot-style A4/KV2 quantization strengthens this baseline. On LLaMA3-1B W2A4KV2, with 240K updates and 31.5B training tokens:

ConfigurationWikiText-2 PPL ↓8-task mean ↑
ParetoQ + uniform A/KV90.5740.47
ParetoQ + QuaRot-style A27.5748.04
CanonQ-Omni-d115.4253.56

CanonQ retains a 5.52-point accuracy gain over the rotated baseline, with lower PPL. The appendix now reports the rotated baseline's complete per-task results (Appendix C.5). We also add the requested 2026 methods:

MethodW2A8KV2 PPL ↓W2A8KV2 Acc. ↑W2A4KV2 PPL ↓W2A4KV2 Acc. ↑
LC-QAT-ext20.6550.3131.9446.06
ParetoQ + GSRQ25.5849.5940.7142.62
ReQAT-ext23.1249.8229.6847.36
CanonQ-Omni-d115.7053.4515.4253.56

LC-QAT-ext retains LC-QAT's weight training and adds QuaRot-style A/KV quantizers; ParetoQ + GSRQ applies GSRQ to KV2; ReQAT-ext adapts its FP4 training recipe to the target W/A/KV precisions. CanonQ leads on both metrics at both precisions, including a 6.20-point W2A4KV2 gain over ReQAT-ext. Implementation details are provided in Appendix C.5.

2. Measured inference efficiency

On one A100-80GB, our W2 tensor-core kernel achieves the following speedups over same-run cuBLAS BF16 at M = 8:

Matrix width, N = K4096819216384
Kernel speedup2.08×2.71×3.36×

The compiled Hugging Face decode loop gives:

MetricLLaMA3-1BLLaMA3-3BLLaMA3-8B
Decode speedup over BF161.20×1.27×1.59×
BF16 peak memory (GB)2.76.816.6
CanonQ peak memory (GB)1.02.04.6
Peak-memory reduction63.0%70.6%72.3%

These W2 measurements use randomly initialized model shapes, batch 1, a 2048-token prompt, and 128 generated tokens. They include attention, online transforms, cache, embeddings, and the language-model head. Thus the deployed W2 codebook implementation delivers model-loop speedups and up to 72.3% lower peak memory (Figure 2; Appendix C.7).

3. Transfer beyond LLaMA

LLaMA is a widely used, representative benchmark family for LLM quantization. Our 1B/3B/8B study provides a controlled comparison across model scales. CanonQ's quantization rule makes no assumptions about network architecture or model size: it operates on tensor blocks, and the codebooks depend only on bitwidth and tuple dimension. The same design therefore applies across architectures and scales.

The original submission evaluates MobileLLM-Pro-Instruct-1B (released October 2025) on code generation and mathematical reasoning. We also report W2A4KV2 results on Qwen3 and MiniCPM5 in this rebuttal:

Qwen3-1.7B (released April 2025)

ConfigurationWikiText-2 PPL ↓8-task mean ↑
FP reference16.3358.10
ParetoQ-based joint baseline194.5337.81
CanonQ-Omni-d119.0248.99

MiniCPM5-1B-Base (released May 2026)

ConfigurationWikiText-2 PPL ↓8-task mean ↑
FP reference24.7853.99
LC-QAT-ext81.5541.71
CanonQ-Omni-d134.5148.85

CanonQ improves accuracy by 11.18 points over ParetoQ on Qwen3 and 7.14 points over LC-QAT-ext on MiniCPM5, with lower PPL in both comparisons. The MiniCPM5 comparison matches group sizes, training data, token budget, and evaluation harness, with all W/A/KV quantizers active.

Section 6 connects the theoretical and empirical findings. Writing assistance for clarity was disclosed in the submission.

Thank you for your response. After understanding your response, other reviews, and careful consideration, I keep my score.

AuthorsOpenReview ↗

Request for Clarification on Remaining Technical Concerns

We appreciate your response, but find it difficult to understand the basis for maintaining a score of 1 without identifying any remaining technical concerns.

We have directly addressed all your questions and provided the requested additional experiments. We are more interested in whether these concerns have been resolved than in the score itself.

A scientific evaluation should be grounded in technical arguments and experimental evidence. If you believe our responses remain insufficient, we ask you to identify the specific unresolved issues rather than simply reaffirming the original score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Manuscript Revisions

We thank the reviewers and the Area Chair for their time and constructive feedback on our paper. In response, we have revised the manuscript and made the following improvements:

  • Stronger baselines: comparisons with LC-QAT, GSRQ, ReQAT, and ParetoQ + QuaRot (Sec. 5.2, Table 1; Appendices C.1 and C.5).
  • Broader evaluation: W2A4KV2 results on Qwen3-1.7B and MiniCPM5-1B-Base (Sec. 5.2; Appendix C.6).
  • Controlled ablations and multiple seeds: separate analyses of rotation, normalization, and codebook levels, plus three-run statistics (Secs. 5.1 and 5.4, Table 3; Appendix C.2).
  • Theory–experiment connections: direct measurements of codebook transfer error and gradient alignment (Sec. 4.3; Appendices C.3–C.4).
  • Efficiency and convergence: kernel and end-to-end inference benchmarks, training time, memory usage, and training trajectories (Sec. 5.7, Figure 2; Appendices C.1 and C.7–C.8).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.