- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes REVQ, a framework for joint low-bit weight and activation quantization of Transformers and CNNs. The authors first analyze the effects of weight quantization, activation quantization, and cross-layer error propagation on model outputs, and then introduce a two-stage quantization framework. In the first stage, learnable groupwise orthogonal rotations parameterized through Lie algebra are used to adjust the quantization coordinate systems of weights and activations. The rotation parameters and model weights are jointly optimized to reduce local quantization errors. In the second stage, cross-layer low-rank residual compensation is introduced, using SVD or Haar wavelet-based initialization to construct low-rank factors that correct propagated quantization errors. Experiments cover ViT, Swin Transformer, CNNs, and BERT-Large on ImageNet, CIFAR-100, COCO, and Wikipedia masked language modeling. The results show that Rotation-QAT achieves competitive accuracy under W4A4 and W3A3 settings, while the additional low-rank compensation provides further improvements on some models.
Critical strengths
Strength 1: The paper provides a clear formulation of local quantization errors and cross-layer error propagation in joint low-bit weight and activation quantization, and addresses them through rotation learning and low-rank residual compensation, respectively. By parameterizing groupwise rotations through the Lie algebra of the special orthogonal group, the proposed method maintains orthogonality throughout training. This design has a well-defined mathematical formulation and can be integrated into existing QAT optimization procedures.
Strength 2: The paper presents extensive experiments across different vision architectures and tasks, including ViT-S/B/L, Swin-T/S/B, several CNNs, and BERT-Large. In particular, under W3A3 quantization on ImageNet, Rotation-QAT substantially improves the accuracy of certain models, increasing ViT-S Top-1 accuracy from 54.33% to 69.97%. The paper also includes ablations on weight-only, rotation-only, and joint training, as well as analyses of SVD/Haar residual initialization and compensation interface selection. These experiments help clarify the contributions of the individual components.
Critical weaknesses
Weakness 1: The methodological novelty is limited. SpinQuant [1], FlatQuant [2], ButterflyQuant [3], and ParoQuant [4] have already explored learnable or structured rotations, while LQER [5] and QERA [6] have studied low-rank quantization error compensation. REVQ mainly combines Lie-algebra-based groupwise rotations with cross-layer low-rank correction. The paper lacks direct comparisons demonstrating the independent advantages of these components over existing approaches.
Weakness 2: The use of the term Product Vector Quantization is not sufficiently justified. Section 2.2 states that REVQ uses group-wise scalar rounding rather than explicit multidimensional codebook search. Although scalar quantization levels can form an implicit Cartesian-product codebook, as in FSQ [8], REVQ does not introduce a new vector codebook or vector-level optimization mechanism. Its distinction from conventional rotated scalar quantization remains unclear.
Weakness 3: The contribution of low-rank residual compensation is small and inconsistent. Table 4 shows that rank-1 Haar compensation improves ImageNet W4A4 Top-1 accuracy by at most 0.11 percentage points and even reduces accuracy on ViT-L and Swin-S. Moreover, the main SOTA comparisons in Tables 3 and 9 use only the first-stage Rotation-QAT, leaving the necessity of the complete two-stage framework insufficiently demonstrated.
Weakness 4: The cross-layer error analysis relies on a first-order approximation that neglects higher-order interactions between quantization errors. The paper does not sufficiently verify whether these omitted terms are negligible under W4A4 or W3A3. Furthermore, Table 5 shows that residual compensation reduces cross-layer error while increasing local-layer error, raising questions about whether the proposed objective effectively minimizes total output distortion.
Weakness 5: The baseline comparisons do not fully account for differences in training budgets. Table 3 compares REVQ, which jointly trains rotations and weights through block-wise QAT, with several training-free PTQ methods and GPLQ [7]. Appendix A.3 reports 500 training steps per block on ImageNet, but the paper lacks matched GPU-hour comparisons. Thus, it is difficult to separate the benefits of the proposed rotation mechanism from those of additional optimization.
Weakness 6: The practical inference efficiency of REVQ remains unclear. The paper does not report actual GPU latency, throughput, peak memory, or effective BPW. Moreover, the deployment precision and computational overhead of the low-rank residual factors are not specified, and the cost of activation rotations is not evaluated. Therefore, the practical storage and execution advantages of the complete W4A4 framework remain insufficiently validated.
Questions for authors
Q1: Given the existing learnable rotation methods [1–4] and low-rank error compensation approaches [5, 6], could the authors clarify the independent technical novelty of REVQ? Could matched-QAT comparisons and component-wise ablations demonstrate advantages over these existing techniques?
Q2: REVQ uses group-wise scalar rounding rather than explicit vector codebook optimization. Could the authors clarify how its Product Vector Quantization differs from conventional rotated scalar quantization or FSQ [8], and provide matched-bit comparisons with standard VQ/PQ methods?
Q3: Table 4 shows limited and inconsistent gains from residual compensation, while Tables 3 and 9 report only Rotation-QAT. Could the authors provide complete two-stage results across all major tasks and compare SVD and Haar initialization under matched residual parameter budgets and different ranks?
Q4: Could the authors quantify the neglected higher-order error terms in Eq. (5), particularly under W3A3? Given the increased local-layer error in Table 5, could they compare the proposed cross-layer compensation objective with directly minimizing total output reconstruction error?
Q5: Could the authors compare REVQ with GPLQ [7] and other strong QAT baselines under matched training data, optimization steps, and GPU-hours? Could they also report the separate training costs of rotation-only, weight-only, Rotation-QAT, and complete REVQ?
Q6: Could the authors report effective BPW, actual model size, GPU latency, throughput, and peak memory, including rotation and residual compensation overhead? Could they also clarify the residual factors' deployment precision and provide results across different group sizes, residual ranks, and random seeds?
[1] Liu, Z., Zhao, C., Fedorov, I., Soran, B., Choudhary, D., Krishnamoorthi, R., Chandra, V., Tian, Y., and Blankevoort, T. SpinQuant: LLM Quantization with Learned Rotations. In International Conference on Learning Representations (ICLR), 2025.
[2] Sun, Y., Liu, R., Bai, H., Bao, H., Zhao, K., Li, Y., Hu, J., Yu, X., Hou, L., Yuan, C., Jiang, X., Liu, W., and Yao, J. FlatQuant: Flatness Matters for LLM Quantization. In Proceedings of the 42nd International Conference on Machine Learning (ICML), pp. 57587–57613, 2025.
[3] Xu, B., Dong, Z., Elachqar, O., and Shang, Y. ButterflyQuant: Ultra-low-bit LLM Quantization through Learnable Orthogonal Butterfly Transforms. arXiv preprint arXiv:2509.09679, 2025.
[4] Liang, Y., Chen, H., Zhang, Z., Han, S., and Liu, Z. ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference. In International Conference on Learning Representations (ICLR), 2026.
[5] Zhang, C., Cheng, J., Constantinides, G. A., and Zhao, Y. LQER: Low-Rank Quantization Error Reconstruction for LLMs. In Proceedings of the 41st International Conference on Machine Learning (ICML), pp. 58763–58779, 2024.
[6] Zhang, C., Wong, J. T. H., Xiao, C., Constantinides, G. A., and Zhao, Y. QERA: An Analytical Framework for Quantization Error Reconstruction. In International Conference on Learning Representations (ICLR), 2025.
[7] Liang, G., Liu, X., and Wu, J. GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers. In Advances in Neural Information Processing Systems (NeurIPS), 2025.
[8] Mentzer, F., Minnen, D., Agustsson, E., and Tschannen, M. Finite Scalar Quantization: VQ-VAE Made Simple. In International Conference on Learning Representations (ICLR), 2024.
4 comments
W2&Q2: Product quantization terminology
Response: As noted in our earlier replies, REVQ does not learn centroids or build a codebook. The bit-width and quantization levels are the same as those of uniform quantization. We agree this is closely related to FSQ [C6], but the difference lies in the geometry of the grid. FSQ uses a fixed, axis-aligned grid, which is the special case in our formulation. In REVQ, each g-dimensional sub-vector is quantized on a lattice whose orientation is a learnable orthogonal rotation. QAT lets the rotation be trained with quantization objective, so the lattice can align with correlated directions in the weight or activation distribution, which an axis-aligned grid cannot. The product structure is taken across the G groups of a layer, and the lattice orientation is the vector-level quantity that we optimize. Since our method does not involve codebooks or codeword search, we have not conducted a matched-bit comparison with VQ/PQ at this stage.
[C6] Mentzer F, Minnen D, Agustsson E, and Tschannen M. Finite Scalar Quantization: VQ-VAE Made Simple. ICLR 2024.
W3&Q3: Compensation gains across ranks and seeds
Response: Thank you for the concerns. We have added the W4A4 Stage2 results to Table 3 and 11 in the revised manuscript. We further compare Haar(rank=1) and SVD(rank=2) under closely matched parameter budgets across three random seeds. Resume training without compensation uses the same additional training budget. The results show that Haar and SVD improve mean Top-1 over Rotation-QAT by 0.093 and 0.123 percentage points. The results support a modest but consistent benefit from compensation in W4A4.
| Params | seed 42 | seed 84 | seed 168 | mean ± std | mean ± std | |
|---|---|---|---|---|---|---|
| Rotation-QAT | 23,808(+0.112%) | 80 | 79.98 | 80.05 | 80.010 ± 0.036 | — |
| resume training | 23,808(+0.112%) | 80.01 | 79.97 | 80.07 | 80.017 ± 0.050 | — |
| SVD r=2 | 58,368(+0.275%) | 80.08 | 80.06 | 80.17 | 80.103 ± 0.059 | +0.093 ± 0.023 |
| Haar r=1 | 54,528(+0.257%) | 80.11 | 80.08 | 80.21 | 80.133 ± 0.068 | +0.124 ± 0.033 |
In this revision, we have not added W3A3 Stage2 results to Table 3. For completeness, our existing W3A3 results on ViT-S are reported below:
| Method | Top-1 (%) | Top-5 (%) | Top-1(pp) |
|---|---|---|---|
| Rotation-QAT (Stage 1) | 69.97 | 90.04 | — |
| SVD (r=1) | 69.73 | 89.97 | −0.24 |
| Haar (r=1) | 69.90 | 90.03 | −0.07 |
Results show that current compensation does not improve accuracy in W3A3. A possible explanation is that second-order error terms in Eq.5 become more consequential at lower precision, whereas Eq.12 captures only the first-order contribution of cross-layer input discrepancies. We discuss this limitation in the next response.
W4&Q4: Error approximation and compensation objectives
Response: We have clarified the error decomposition in Eq. (5) in Section 3, and conducted additional analyses of approximation and compensation objective.
- Quantifying the neglected high-order error terms
Neglecting higher-order error products has precedent in prior work [C7]. To quantify the effect of retaining only first-order terms, Let and denote the first-order and high-order terms in Eq. (5): We have . Using the Frobenius norms, we reported the relative magnitude , the relative weight error , and their ratio . We also measured the signed relative change in squared reconstruction error:
| Metric | W4A4 | W3A3 |
|---|---|---|
| 9.2% | 21.3% | |
| 0.101 | 0.235 | |
| 0.91 | 0.91 | |
| −0.13% | −0.65% |
The normalized ratio is approximately 0.91 in both settings, consistent with a larger higher-order contribution as the relative weight error increases. Under W4A4, omitting changes the squared reconstruction error by only −0.13%, supporting the approximation in this setting. Under W3A3, rises to 21.3% and the magnitude of the squared-error discrepancy increases to 0.65%, indicating that the higher-order contribution is more pronounced at lower precision.
[C7] Lin, Talathi, and Annapureddy. Fixed Point Quantization of Deep Convolutional Networks. ICML 2016.
- Comparing compensation objectives
The opposite trends of the two errors during Stage 2 arise from their different optimization objectives, as specified by the loss function in Section 3.4. Both involve end-to-end task adaptation, but the local-layer objective focuses on output reconstruction at the current layer, whereas the cross-layer objective additionally uses the full-precision baseline input.
To directly test the compensation target with all other settings unchanged, we compare Stage-2A, which uses the original first-order cross-layer target in Eq.12, with Stage-2B, which targets the full output reconstruction error in Eq.5. Stage-1 denotes the Rotation-QAT baseline without compensation.
| W4A4 | ViT-S | ViT-B |
|---|---|---|
| Stage-1 | 80.00 | 84.27 |
| Stage-2A | 80.11 | 84.35 |
| Stage-2B | 80.20 | 84.40 |
The table shows that Stage-2B improves upon Stage-2A by 0.09 and 0.05 percentage points on ViT-S and ViT-B. These results support directly targeting the full output reconstruction error as a refinement of the original compensation objective.
W5&Q5: Training cost and peak memory
Response: We measured GPU-hours and peak memory for ViT-B under W4A4 with group size 32. All measurements were conducted on a single RTX 3090 (24GB), using an effective batch size of 16, CUDA 12.4, PyTorch 2.6.0, torchvision 0.21.0, and timm 1.0.19. The host system has two Xeon Silver 4314 CPUs (64 logical cores). REVQ used BF16 autocast, whereas GPLQ used FP16 AMP.
The weight-only, rotation-only, and Rotation-QAT variants were evaluated under matched total optimization steps. Their costs, together with those of residual compensation and complete REVQ, are reported below.
| Configuration | GPU-hours | Peak memory (GiB) |
|---|---|---|
| Weight-only | 1.06 | 2.0 |
| Rotation-only | 0.81 | 1.9 |
| Rotation-QAT | 1.42 | 2.1 |
| Haar compensation (r=1) | 1.31 | 1.9 |
| Complete REVQ | 2.73 | 2.2 |
| GPLQ Stage1 (1 epoch) | 4.20 | 3.9 |
As described in Section 3.4, complete REVQ sequentially performs Rotation-QAT and residual compensation. Compared with GPLQ Stage1, REVQ uses 35.0% fewer GPU-hours and 43.6% less peak GPU memory. This comparison characterizes the measured computational costs. A fully controlled comparison with other strong QAT baselines remains to be completed.
W6&Q6: Storage and computational efficiency
Response: According to the comments, we report storage accounting, inference overhead, and group-size results below.
- Effective BPW and model size
We report two metrics: (i) Q-BPW, which measures storage per weight in the quantized layers, and (ii) full-model bits/parameter, which accounts for the entire model. Under W4A4, our storage accounting includes 4-bit linear-layer weights, fp16 groupwise scales, fp16 residual compensation factors, online rotation for activation, and all non-quantized parameters.
The ViT-S results are reported below, and more detailed results are provided in the Table 14 Appendix A.9.
| Method | Q-BPW | Full-model bits/param | Model size | Compression vs. FP16 |
|---|---|---|---|---|
| FP16 | 16 | 16 | 44.10MB | 1.00$\times$ |
| REVQ(SVD,r=1) | 4.53 | 4.97 | 13.71MB | 3.22$\times$ |
| REVQ(Haar,r=1) | 4.554 | 4.99 | 13.77MB | 3.20$\times$ |
| GPLQ+QwT | 5.837 | 6.21 | 17.13MB | 2.58$\times$ |
REVQ achieves approximately 3.2$\times$ compression relative to FP16, including auxiliary storage. On ViT-S, rank-1 compensation increases the parameter count by 0.18% for SVD and 0.32% for Haar, compared with 8.0% for the full-rank QwT correction.
- GPU latency of inference components
To isolate the cost of the operations added to a standard linear layer, we timed each component separately on a single RTX 3090 (torch.compile, CUDA Graphs, median over repeated runs). Each baseline is an FP16 GEMM with the same shape as the corresponding linear layer, so the percentages are relative to this GEMM latency only. “Rot” denotes groupwise () block-diagonal activation rotation. “Dynamic A-scale” denotes per-token, per-group dynamic activation scale computation. Compensation uses an SVD rank-1 branch.
| Model | Batch size | fp16 (ms) | Rot(ms / %) | Rot+Dynamic A-scale(ms / %) | Compensation(ms/%) |
|---|---|---|---|---|---|
| ViT-S | 32 | 4.74 | 1.04 / 22.0% | 1.86 / 39.3% | 0.89 / 18.7% |
| ViT-B | 32 | 24.22 | 3.01 / 12.4% | 5.31 / 21.9% | 2.17 / 9.0% |
| Swin-S | 32 | 16.28 | 5.07 / 31.2% | 8.63 / 53.0% | 3.57 / 21.9% |
| Swin-B | 32 | 29.63 | 7.82 / 26.4% | 12.43 / 42.0% | 5.29 / 17.8% |
| ViT-S | 1 | 0.54 | 0.17 / 30.5% | 0.25 / 45.2% | 0.15 / 27.1% |
| ViT-B | 1 | 1.13 | 0.19 / 16.8% | 0.31 / 27.6% | 0.19 / 16.8% |
| Swin-S | 1 | 1.11 | 0.34 / 31.0% | 0.57 / 51.2% | 0.31 / 27.6% |
| Swin-B | 1 | 1.26 | 0.40 / 31.6% | 0.64 / 50.7% | 0.33 / 26.1% |
At batch size 32, rotation alone costs 12–31% of the GEMM time, and rotation with dynamic activation scales costs 22–53%. Compensation adds 9–22%. At batch size 1 the relative costs are 17–32%, 28–51%, and 17–28%. The overhead is not negligible in our current implementation. We attribute it mainly to the low arithmetic intensity of the elementwise and small matrix operations, which makes memory traffic the bottleneck. A fused kernel could reduce this overhead, and we leave its implementation to future work.
- Throughput
We measured inference throughput for ViT-S, ViT-B, and Swin-S on a single RTX 3090 (torch.compile, CUDA Graphs; img/s=batch size/ latency). Stage1 is Rotation-QAT without compensation, and Stage2 adds SVD rank-1 compensation. Percentage indicate throughput relative to FP16.
| Model | Batch Size | FP16 ms (img/s) | Stage1 ms (img/s, %) | Stage2 ms (img/s, %) |
|---|---|---|---|---|
| ViT-S | 1 | 1.22 (820) | 2.19 (457, 55.7%) | 2.56 (391, 47.7%) |
| 32 | 7.40 (4324) | 11.42 (2802, 64.8%) | 12.14 (2636, 61.0%) | |
| ViT-B | 1 | 1.69 (592) | 3.41 (293, 49.6%) | 3.56 (281, 47.5%) |
| 32 | 21.81 (1467) | 29.85 (1072, 73.1%) | 31.19 (1026, 69.9%) | |
| Swin-S | 1 | 2.08 (481) | 4.54 (220, 45.8%) | 4.81 (208, 43.2%) |
| 32 | 23.55 (1359) | 33.87 (945, 69.5%) | 35.62 (898, 66.1%) |
The results show that adding compensation reduces throughput relative to Stage1 by 4.3-5.9% at batch size 32 and 4.2-14.5% at batch size 1. The measurements above additionally quantify the costs of activation rotation and dynamic scaling. The evaluation characterizes fake-quantized implementation and does not use fused INT4 kernels.
-
GPU-hours and peak memory results are reported in our response to W5&Q5.
-
Group sizes
Using random seed 84, we evaluated three group sizes on ViT-S. The results show that smaller groups can further improve accuracy but require additional storage for groupwise rotations and scales, and may increase inference latency. This introduces an accuracy-efficiency trade-off.
| Group Size | Rotation-QAT | SVD r=2 | Haar r=1 |
|---|---|---|---|
| 16 | 80.392 | 80.41 | 80.426 |
| 32 | 79.98 | 80.06 | 80.08 |
| 64 | 79.34 | 79.374 | 79.444 |
- Comparisons across residual rank configurations and random seeds are reported in our response to W3&Q3.
Thank you for your valuable comments and below are our responses to the weaknesses and questions.
W1&Q1: Rotation formulation and cross-layer compensation
Response: We have revised Section 3 to clarify the quantization procedure of the learnable rotations and distinguish the roles of rotations and low-rank compensation. Specifically, the revised weight and activation quantizers are defined as follows:
Q_a(x,R)=s_a(xR)\left[\operatorname{clip}\left(\left\lfloor\frac{xR}{s_a(xR)}+z_a(xR)\right\rceil,0,2^{b_a}-1\right)-z_a(xR)\right]R^\top.$$ The quantized linear-layer output is $\hat{Y}=Q_a(\hat{X},R_a)Q_w(W,R_w)$. Each quantizer first transforms its input into a rotated coordinate system, applies uniform scalar quantization and dequantization, and then maps the result back to the original coordinate system. Below, we discuss the difference between the proposed method and related works in rotation and compensation, and make further experimental evaluations. 1. **Rotation formulation and matched comparisons**. Existing rotation-based methods, including SpinQuant [C1], suppress outliers while preserving the full-precision linear mapping through coordinated orthogonal transformations. In our formulation, rotation and inverse rotation are included within the quantizer, and this constructs a quantization coordinate system that aligns the singular vectors of the activation and weight data during training. Therefore, our method permits separate rotations $R_a$ and $R_w$ for activations and weights. To accelerate convergence and improve training efficiency, all experiments reported in the current manuscript use the tied case $R_a=R_w$. In principle, decoupling these rotations could further improve performance. Our method and FSQ [C6] mainly differ in the geometry of quantization grid. FSQ uses a fixed, axis-aligned grid, which is the special case $R=I$ in our formulation. To evaluate the rotation component, we conducted additional comparisons with widely used baselines, including SpinQuant. All experiments use the same per-group quantization settings and training budgets as our method. Without low-rank compensation, Rotation-QAT improves Top-1 over SpinQuant by 3.83, 3.34, 0.54, and 0.75 percentage points on ViT-S, ViT-B, Swin-S, and Swin-B. | Method | W-A | ViT-S | ViT-B | Swin-S | Swin-B | |---|---|---|---|---|---| | Full-Precision | 32-32 | 81.4 | 85.2 | 83.21 | 85.16 | | SpinQuant | 4-4 | 76.17 | 80.93 | 82.07 | 83.88 | | FIMA-Q | 4-4 | 78.024 | 82.908 | 81.228 | 83.368 | | GPLQ | 4-4 | 77.44 | 82.19 | 81.94 | 84.18 | | Ours | Rotation-QAT | 80 | 84.27 | 82.61 | 84.63 | | | Haar (rank=1) | 80.11 | 84.35 | 82.53 | 84.66 | 2. **Cross-layer compensation target and QAT integration** We agree that the idea to introducing low-rank compensation is not new. In this paper, our distinction is the cross-layer compensation target and its integration into QAT. LQER [C4] uses SVD to approximate the residual $W-Q(W)$, and its variant $L^2$QER applies activation-based scaling before the SVD. QERA [C5] gives a closed-form correction that minimizes the expected layer-output error under input second-moment statistics. [C1] Liu Z, Zhao C, Fedorov I, et al. Spinquant: Llm quantization with learned rotations. ICLR 2025. [C2] Wu Z, Wang S, Zhang J, et al. FIMA-Q: Post-training quantization for vision transformers by fisher information matrix approximation. CVPR 2025. [C3] Liang G, Liu X, Wu J. Gplq: A general, practical, and lightning qat method for vision transformers. NeurIPS 2026. [C4] Zhang C, Cheng J, Constantinides G. A, and Zhao Y. LQER: Low-Rank Quantization Error Reconstruction for LLMs. ICML 2024. [C5] Zhang C, Wong J. T. H, Xiao C, Constantinides G. A, and Zhao Y. QERA: An Analytical Framework for Quantization Error Reconstruction. ICLR 2025.