acceptodds
Under review as a conference paper at ICLR 2027

REVQ: Rotation-Enhanced Product Vector Quantization with Low-Rank Residual Compensation

Abstract

Joint low-bit quantization of weight and activation could substantially reduce the storage and computational costs of Transformer-based architectures and convolutional neural networks (CNNs). However, existing methods cannot simultaneously minimize the layer-wise local quantization errors caused by joint misalignment between weight and activation distributions and address accumulation of errors across layers. In this paper, we formulate the cross-layer and local-layer errors for joint weight and activation quantization, and consequently, propose a rotation-based two-stage framework named REVQ for low-bit quantization to simultaneously address the errors. Specifically, we develop learnable groupwise rotation transformations parametrized by the Lie algebra of the special orthogonal group SO(g) to learn the coordinate system of parameter (weight and activation) space for achieving product vector quantization and reducing local-layer quantization error. Moreover, we design cross-layer low-rank residual compensation to bridge rotation-based quantization in adjacent linear layers with learned low-rank compensation via Haar wavelets for minimizing cross-layer quantization error. Extensive experiments demonstrate that REVQ achieves state-of-the-art performance in various tasks of image classification on ImageNet and CIFAR-100, object detection and instance segmentation on COCO, and language modeling on Wikipedia using a wide range of Transformer-based architectures and CNNs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Model quantization, quantization-aware training, vector quantization, learnable rotations, low-rank residual compensation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3pNxOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes REVQ, a framework for joint low-bit weight and activation quantization of Transformers and CNNs. The authors first analyze the effects of weight quantization, activation quantization, and cross-layer error propagation on model outputs, and then introduce a two-stage quantization framework. In the first stage, learnable groupwise orthogonal rotations parameterized through Lie algebra are used to adjust the quantization coordinate systems of weights and activations. The rotation parameters and model weights are jointly optimized to reduce local quantization errors. In the second stage, cross-layer low-rank residual compensation is introduced, using SVD or Haar wavelet-based initialization to construct low-rank factors that correct propagated quantization errors. Experiments cover ViT, Swin Transformer, CNNs, and BERT-Large on ImageNet, CIFAR-100, COCO, and Wikipedia masked language modeling. The results show that Rotation-QAT achieves competitive accuracy under W4A4 and W3A3 settings, while the additional low-rank compensation provides further improvements on some models.

Critical strengths

Strength 1: The paper provides a clear formulation of local quantization errors and cross-layer error propagation in joint low-bit weight and activation quantization, and addresses them through rotation learning and low-rank residual compensation, respectively. By parameterizing groupwise rotations through the Lie algebra of the special orthogonal group, the proposed method maintains orthogonality throughout training. This design has a well-defined mathematical formulation and can be integrated into existing QAT optimization procedures.

Strength 2: The paper presents extensive experiments across different vision architectures and tasks, including ViT-S/B/L, Swin-T/S/B, several CNNs, and BERT-Large. In particular, under W3A3 quantization on ImageNet, Rotation-QAT substantially improves the accuracy of certain models, increasing ViT-S Top-1 accuracy from 54.33% to 69.97%. The paper also includes ablations on weight-only, rotation-only, and joint training, as well as analyses of SVD/Haar residual initialization and compensation interface selection. These experiments help clarify the contributions of the individual components.

Critical weaknesses

Weakness 1: The methodological novelty is limited. SpinQuant [1], FlatQuant [2], ButterflyQuant [3], and ParoQuant [4] have already explored learnable or structured rotations, while LQER [5] and QERA [6] have studied low-rank quantization error compensation. REVQ mainly combines Lie-algebra-based groupwise rotations with cross-layer low-rank correction. The paper lacks direct comparisons demonstrating the independent advantages of these components over existing approaches.

Weakness 2: The use of the term Product Vector Quantization is not sufficiently justified. Section 2.2 states that REVQ uses group-wise scalar rounding rather than explicit multidimensional codebook search. Although scalar quantization levels can form an implicit Cartesian-product codebook, as in FSQ [8], REVQ does not introduce a new vector codebook or vector-level optimization mechanism. Its distinction from conventional rotated scalar quantization remains unclear.

Weakness 3: The contribution of low-rank residual compensation is small and inconsistent. Table 4 shows that rank-1 Haar compensation improves ImageNet W4A4 Top-1 accuracy by at most 0.11 percentage points and even reduces accuracy on ViT-L and Swin-S. Moreover, the main SOTA comparisons in Tables 3 and 9 use only the first-stage Rotation-QAT, leaving the necessity of the complete two-stage framework insufficiently demonstrated.

Weakness 4: The cross-layer error analysis relies on a first-order approximation that neglects higher-order interactions between quantization errors. The paper does not sufficiently verify whether these omitted terms are negligible under W4A4 or W3A3. Furthermore, Table 5 shows that residual compensation reduces cross-layer error while increasing local-layer error, raising questions about whether the proposed objective effectively minimizes total output distortion.

Weakness 5: The baseline comparisons do not fully account for differences in training budgets. Table 3 compares REVQ, which jointly trains rotations and weights through block-wise QAT, with several training-free PTQ methods and GPLQ [7]. Appendix A.3 reports 500 training steps per block on ImageNet, but the paper lacks matched GPU-hour comparisons. Thus, it is difficult to separate the benefits of the proposed rotation mechanism from those of additional optimization.

Weakness 6: The practical inference efficiency of REVQ remains unclear. The paper does not report actual GPU latency, throughput, peak memory, or effective BPW. Moreover, the deployment precision and computational overhead of the low-rank residual factors are not specified, and the cost of activation rotations is not evaluated. Therefore, the practical storage and execution advantages of the complete W4A4 framework remain insufficiently validated.

Questions for authors

Q1: Given the existing learnable rotation methods [1–4] and low-rank error compensation approaches [5, 6], could the authors clarify the independent technical novelty of REVQ? Could matched-QAT comparisons and component-wise ablations demonstrate advantages over these existing techniques?

Q2: REVQ uses group-wise scalar rounding rather than explicit vector codebook optimization. Could the authors clarify how its Product Vector Quantization differs from conventional rotated scalar quantization or FSQ [8], and provide matched-bit comparisons with standard VQ/PQ methods?

Q3: Table 4 shows limited and inconsistent gains from residual compensation, while Tables 3 and 9 report only Rotation-QAT. Could the authors provide complete two-stage results across all major tasks and compare SVD and Haar initialization under matched residual parameter budgets and different ranks?

Q4: Could the authors quantify the neglected higher-order error terms in Eq. (5), particularly under W3A3? Given the increased local-layer error in Table 5, could they compare the proposed cross-layer compensation objective with directly minimizing total output reconstruction error?

Q5: Could the authors compare REVQ with GPLQ [7] and other strong QAT baselines under matched training data, optimization steps, and GPU-hours? Could they also report the separate training costs of rotation-only, weight-only, Rotation-QAT, and complete REVQ?

Q6: Could the authors report effective BPW, actual model size, GPU latency, throughput, and peak memory, including rotation and residual compensation overhead? Could they also clarify the residual factors' deployment precision and provide results across different group sizes, residual ranks, and random seeds?

[1] Liu, Z., Zhao, C., Fedorov, I., Soran, B., Choudhary, D., Krishnamoorthi, R., Chandra, V., Tian, Y., and Blankevoort, T. SpinQuant: LLM Quantization with Learned Rotations. In International Conference on Learning Representations (ICLR), 2025.

[2] Sun, Y., Liu, R., Bai, H., Bao, H., Zhao, K., Li, Y., Hu, J., Yu, X., Hou, L., Yuan, C., Jiang, X., Liu, W., and Yao, J. FlatQuant: Flatness Matters for LLM Quantization. In Proceedings of the 42nd International Conference on Machine Learning (ICML), pp. 57587–57613, 2025.

[3] Xu, B., Dong, Z., Elachqar, O., and Shang, Y. ButterflyQuant: Ultra-low-bit LLM Quantization through Learnable Orthogonal Butterfly Transforms. arXiv preprint arXiv:2509.09679, 2025.

[4] Liang, Y., Chen, H., Zhang, Z., Han, S., and Liu, Z. ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference. In International Conference on Learning Representations (ICLR), 2026.

[5] Zhang, C., Cheng, J., Constantinides, G. A., and Zhao, Y. LQER: Low-Rank Quantization Error Reconstruction for LLMs. In Proceedings of the 41st International Conference on Machine Learning (ICML), pp. 58763–58779, 2024.

[6] Zhang, C., Wong, J. T. H., Xiao, C., Constantinides, G. A., and Zhao, Y. QERA: An Analytical Framework for Quantization Error Reconstruction. In International Conference on Learning Representations (ICLR), 2025.

[7] Liang, G., Liu, X., and Wu, J. GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers. In Advances in Neural Information Processing Systems (NeurIPS), 2025.

[8] Mentzer, F., Minnen, D., Agustsson, E., and Tschannen, M. Finite Scalar Quantization: VQ-VAE Made Simple. In International Conference on Learning Representations (ICLR), 2024.

4 comments

Thank you for your valuable comments and below are our responses to the weaknesses and questions.

W1&Q1: Rotation formulation and cross-layer compensation

Response: We have revised Section 3 to clarify the quantization procedure of the learnable rotations and distinguish the roles of rotations and low-rank compensation. Specifically, the revised weight and activation quantizers are defined as follows:

Q_a(x,R)=s_a(xR)\left[\operatorname{clip}\left(\left\lfloor\frac{xR}{s_a(xR)}+z_a(xR)\right\rceil,0,2^{b_a}-1\right)-z_a(xR)\right]R^\top.$$ The quantized linear-layer output is $\hat{Y}=Q_a(\hat{X},R_a)Q_w(W,R_w)$. Each quantizer first transforms its input into a rotated coordinate system, applies uniform scalar quantization and dequantization, and then maps the result back to the original coordinate system. Below, we discuss the difference between the proposed method and related works in rotation and compensation, and make further experimental evaluations. 1. **Rotation formulation and matched comparisons**. Existing rotation-based methods, including SpinQuant [C1], suppress outliers while preserving the full-precision linear mapping through coordinated orthogonal transformations. In our formulation, rotation and inverse rotation are included within the quantizer, and this constructs a quantization coordinate system that aligns the singular vectors of the activation and weight data during training. Therefore, our method permits separate rotations $R_a$ and $R_w$ for activations and weights. To accelerate convergence and improve training efficiency, all experiments reported in the current manuscript use the tied case $R_a=R_w$. In principle, decoupling these rotations could further improve performance. Our method and FSQ [C6] mainly differ in the geometry of quantization grid. FSQ uses a fixed, axis-aligned grid, which is the special case $R=I$ in our formulation. To evaluate the rotation component, we conducted additional comparisons with widely used baselines, including SpinQuant. All experiments use the same per-group quantization settings and training budgets as our method. Without low-rank compensation, Rotation-QAT improves Top-1 over SpinQuant by 3.83, 3.34, 0.54, and 0.75 percentage points on ViT-S, ViT-B, Swin-S, and Swin-B. | Method | W-A | ViT-S | ViT-B | Swin-S | Swin-B | |---|---|---|---|---|---| | Full-Precision | 32-32 | 81.4 | 85.2 | 83.21 | 85.16 | | SpinQuant | 4-4 | 76.17 | 80.93 | 82.07 | 83.88 | | FIMA-Q | 4-4 | 78.024 | 82.908 | 81.228 | 83.368 | | GPLQ | 4-4 | 77.44 | 82.19 | 81.94 | 84.18 | | Ours | Rotation-QAT | 80 | 84.27 | 82.61 | 84.63 | | | Haar (rank=1) | 80.11 | 84.35 | 82.53 | 84.66 | 2. **Cross-layer compensation target and QAT integration** We agree that the idea to introducing low-rank compensation is not new. In this paper, our distinction is the cross-layer compensation target and its integration into QAT. LQER [C4] uses SVD to approximate the residual $W-Q(W)$, and its variant $L^2$QER applies activation-based scaling before the SVD. QERA [C5] gives a closed-form correction that minimizes the expected layer-output error under input second-moment statistics. [C1] Liu Z, Zhao C, Fedorov I, et al. Spinquant: Llm quantization with learned rotations. ICLR 2025. [C2] Wu Z, Wang S, Zhang J, et al. FIMA-Q: Post-training quantization for vision transformers by fisher information matrix approximation. CVPR 2025. [C3] Liang G, Liu X, Wu J. Gplq: A general, practical, and lightning qat method for vision transformers. NeurIPS 2026. [C4] Zhang C, Cheng J, Constantinides G. A, and Zhao Y. LQER: Low-Rank Quantization Error Reconstruction for LLMs. ICML 2024. [C5] Zhang C, Wong J. T. H, Xiao C, Constantinides G. A, and Zhao Y. QERA: An Analytical Framework for Quantization Error Reconstruction. ICLR 2025.

W2&Q2: Product quantization terminology

Response: As noted in our earlier replies, REVQ does not learn centroids or build a codebook. The bit-width and quantization levels are the same as those of uniform quantization. We agree this is closely related to FSQ [C6], but the difference lies in the geometry of the grid. FSQ uses a fixed, axis-aligned grid, which is the special case R=IR=I in our formulation. In REVQ, each g-dimensional sub-vector is quantized on a lattice whose orientation is a learnable orthogonal rotation. QAT lets the rotation be trained with quantization objective, so the lattice can align with correlated directions in the weight or activation distribution, which an axis-aligned grid cannot. The product structure is taken across the G groups of a layer, and the lattice orientation is the vector-level quantity that we optimize. Since our method does not involve codebooks or codeword search, we have not conducted a matched-bit comparison with VQ/PQ at this stage.

[C6] Mentzer F, Minnen D, Agustsson E, and Tschannen M. Finite Scalar Quantization: VQ-VAE Made Simple. ICLR 2024.

W3&Q3: Compensation gains across ranks and seeds

Response: Thank you for the concerns. We have added the W4A4 Stage2 results to Table 3 and 11 in the revised manuscript. We further compare Haar(rank=1) and SVD(rank=2) under closely matched parameter budgets across three random seeds. Resume training without compensation uses the same additional training budget. The results show that Haar and SVD improve mean Top-1 over Rotation-QAT by 0.093 and 0.123 percentage points. The results support a modest but consistent benefit from compensation in W4A4.

Δ\Delta Paramsseed 42seed 84seed 168mean ± stdΔ\Delta mean ± std
Rotation-QAT23,808(+0.112%)8079.9880.0580.010 ± 0.036—
resume training23,808(+0.112%)80.0179.9780.0780.017 ± 0.050—
SVD r=258,368(+0.275%)80.0880.0680.1780.103 ± 0.059+0.093 ± 0.023
Haar r=154,528(+0.257%)80.1180.0880.2180.133 ± 0.068+0.124 ± 0.033

In this revision, we have not added W3A3 Stage2 results to Table 3. For completeness, our existing W3A3 results on ViT-S are reported below:

MethodTop-1 (%)Top-5 (%)Δ\Delta Top-1(pp)
Rotation-QAT (Stage 1)69.9790.04—
SVD (r=1)69.7389.97−0.24
Haar (r=1)69.9090.03−0.07

Results show that current compensation does not improve accuracy in W3A3. A possible explanation is that second-order error terms in Eq.5 become more consequential at lower precision, whereas Eq.12 captures only the first-order contribution of cross-layer input discrepancies. We discuss this limitation in the next response.

W4&Q4: Error approximation and compensation objectives

Response: We have clarified the error decomposition in Eq. (5) in Section 3, and conducted additional analyses of approximation and compensation objective.

  1. Quantifying the neglected high-order error terms

Neglecting higher-order error products has precedent in prior work [C7]. To quantify the effect of retaining only first-order terms, Let FF and HH denote the first-order and high-order terms in Eq. (5): F=−EXW+EAW+XEW,H=−EXEW+EAEW.F=-E^XW+E^AW+XE^W,\qquad H=-E^XE^W+E^AE^W. We have Y^−Y=F+H\widehat{Y}-Y=F+H. Using the Frobenius norms, we reported the relative magnitude rH=∥H∥F/∥F∥Fr_H=\|H\|_F/\|F\|_F, the relative weight error rW=∥EW∥F/∥W∥Fr_W=\|E^W\|_F/\|W\|_F, and their ratio k=rH/rWk=r_H/r_W. We also measured the signed relative change in squared reconstruction error: δ=∥F∥F2−∥F+H∥F2∥F+H∥F2×100%.\delta=\frac{\|F\|_F^2-\|F+H\|_F^2}{\|F+H\|_F^2}\times100\%.

MetricW4A4W3A3
rHr_H9.2%21.3%
rWr_W0.1010.235
kk0.910.91
δ\delta−0.13%−0.65%

The normalized ratio kk is approximately 0.91 in both settings, consistent with a larger higher-order contribution as the relative weight error increases. Under W4A4, omitting HH changes the squared reconstruction error by only −0.13%, supporting the approximation in this setting. Under W3A3, rHr_H rises to 21.3% and the magnitude of the squared-error discrepancy increases to 0.65%, indicating that the higher-order contribution is more pronounced at lower precision.

[C7] Lin, Talathi, and Annapureddy. Fixed Point Quantization of Deep Convolutional Networks. ICML 2016.

  1. Comparing compensation objectives

The opposite trends of the two errors during Stage 2 arise from their different optimization objectives, as specified by the loss function in Section 3.4. Both involve end-to-end task adaptation, but the local-layer objective focuses on output reconstruction at the current layer, whereas the cross-layer objective additionally uses the full-precision baseline input.

To directly test the compensation target with all other settings unchanged, we compare Stage-2A, which uses the original first-order cross-layer target in Eq.12, with Stage-2B, which targets the full output reconstruction error in Eq.5. Stage-1 denotes the Rotation-QAT baseline without compensation.

W4A4ViT-SViT-B
Stage-180.0084.27
Stage-2A80.1184.35
Stage-2B80.2084.40

The table shows that Stage-2B improves upon Stage-2A by 0.09 and 0.05 percentage points on ViT-S and ViT-B. These results support directly targeting the full output reconstruction error as a refinement of the original compensation objective.

W5&Q5: Training cost and peak memory

Response: We measured GPU-hours and peak memory for ViT-B under W4A4 with group size 32. All measurements were conducted on a single RTX 3090 (24GB), using an effective batch size of 16, CUDA 12.4, PyTorch 2.6.0, torchvision 0.21.0, and timm 1.0.19. The host system has two Xeon Silver 4314 CPUs (64 logical cores). REVQ used BF16 autocast, whereas GPLQ used FP16 AMP.

The weight-only, rotation-only, and Rotation-QAT variants were evaluated under matched total optimization steps. Their costs, together with those of residual compensation and complete REVQ, are reported below.

ConfigurationGPU-hoursPeak memory (GiB)
Weight-only1.062.0
Rotation-only0.811.9
Rotation-QAT1.422.1
Haar compensation (r=1)1.311.9
Complete REVQ2.732.2
GPLQ Stage1 (1 epoch)4.203.9

As described in Section 3.4, complete REVQ sequentially performs Rotation-QAT and residual compensation. Compared with GPLQ Stage1, REVQ uses 35.0% fewer GPU-hours and 43.6% less peak GPU memory. This comparison characterizes the measured computational costs. A fully controlled comparison with other strong QAT baselines remains to be completed.

W6&Q6: Storage and computational efficiency

Response: According to the comments, we report storage accounting, inference overhead, and group-size results below.

  1. Effective BPW and model size

We report two metrics: (i) Q-BPW, which measures storage per weight in the quantized layers, and (ii) full-model bits/parameter, which accounts for the entire model. Under W4A4, our storage accounting includes 4-bit linear-layer weights, fp16 groupwise scales, fp16 residual compensation factors, online rotation for activation, and all non-quantized parameters.

The ViT-S results are reported below, and more detailed results are provided in the Table 14 Appendix A.9.

MethodQ-BPWFull-model bits/paramModel sizeCompression vs. FP16
FP16161644.10MB1.00$\times$
REVQ(SVD,r=1)4.534.9713.71MB3.22$\times$
REVQ(Haar,r=1)4.5544.9913.77MB3.20$\times$
GPLQ+QwT5.8376.2117.13MB2.58$\times$

REVQ achieves approximately 3.2$\times$ compression relative to FP16, including auxiliary storage. On ViT-S, rank-1 compensation increases the parameter count by 0.18% for SVD and 0.32% for Haar, compared with 8.0% for the full-rank QwT correction.

  1. GPU latency of inference components

To isolate the cost of the operations added to a standard linear layer, we timed each component separately on a single RTX 3090 (torch.compile, CUDA Graphs, median over repeated runs). Each baseline is an FP16 GEMM with the same shape as the corresponding linear layer, so the percentages are relative to this GEMM latency only. “Rot” denotes groupwise (32×3232\times 32) block-diagonal activation rotation. “Dynamic A-scale” denotes per-token, per-group dynamic activation scale computation. Compensation uses an SVD rank-1 branch.

ModelBatch sizefp16 (ms)Rot(ms / %)Rot+Dynamic A-scale(ms / %)Compensation(ms/%)
ViT-S324.741.04 / 22.0%1.86 / 39.3%0.89 / 18.7%
ViT-B3224.223.01 / 12.4%5.31 / 21.9%2.17 / 9.0%
Swin-S3216.285.07 / 31.2%8.63 / 53.0%3.57 / 21.9%
Swin-B3229.637.82 / 26.4%12.43 / 42.0%5.29 / 17.8%
ViT-S10.540.17 / 30.5%0.25 / 45.2%0.15 / 27.1%
ViT-B11.130.19 / 16.8%0.31 / 27.6%0.19 / 16.8%
Swin-S11.110.34 / 31.0%0.57 / 51.2%0.31 / 27.6%
Swin-B11.260.40 / 31.6%0.64 / 50.7%0.33 / 26.1%

At batch size 32, rotation alone costs 12–31% of the GEMM time, and rotation with dynamic activation scales costs 22–53%. Compensation adds 9–22%. At batch size 1 the relative costs are 17–32%, 28–51%, and 17–28%. The overhead is not negligible in our current implementation. We attribute it mainly to the low arithmetic intensity of the elementwise and small matrix operations, which makes memory traffic the bottleneck. A fused kernel could reduce this overhead, and we leave its implementation to future work.

  1. Throughput

We measured inference throughput for ViT-S, ViT-B, and Swin-S on a single RTX 3090 (torch.compile, CUDA Graphs; img/s=batch size/ latency). Stage1 is Rotation-QAT without compensation, and Stage2 adds SVD rank-1 compensation. Percentage indicate throughput relative to FP16.

ModelBatch SizeFP16 ms (img/s)Stage1 ms (img/s, %)Stage2 ms (img/s, %)
ViT-S11.22 (820)2.19 (457, 55.7%)2.56 (391, 47.7%)
327.40 (4324)11.42 (2802, 64.8%)12.14 (2636, 61.0%)
ViT-B11.69 (592)3.41 (293, 49.6%)3.56 (281, 47.5%)
3221.81 (1467)29.85 (1072, 73.1%)31.19 (1026, 69.9%)
Swin-S12.08 (481)4.54 (220, 45.8%)4.81 (208, 43.2%)
3223.55 (1359)33.87 (945, 69.5%)35.62 (898, 66.1%)

The results show that adding compensation reduces throughput relative to Stage1 by 4.3-5.9% at batch size 32 and 4.2-14.5% at batch size 1. The measurements above additionally quantify the costs of activation rotation and dynamic scaling. The evaluation characterizes fake-quantized implementation and does not use fused INT4 kernels.

  1. GPU-hours and peak memory results are reported in our response to W5&Q5.

  2. Group sizes

Using random seed 84, we evaluated three group sizes on ViT-S. The results show that smaller groups can further improve accuracy but require additional storage for groupwise rotations and scales, and may increase inference latency. This introduces an accuracy-efficiency trade-off.

Group SizeRotation-QATSVD r=2Haar r=1
1680.39280.4180.426
3279.9880.0680.08
6479.3479.37479.444
  1. Comparisons across residual rank configurations and random seeds are reported in our response to W3&Q3.
Reviewer_GRrTOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a rotation-based two-stage framework, REVQ, to separately address the quantization error of the current layer and the input error propagated from preceding layers for the weight and activation quantization in CNNs and Transformers. In the first stage, each projection layer learns a set of rotation transformations shared across groups. In the second stage, the weights and rotations are frozen, and low-rank compensation for minimizing cross-layer quantization error is introduced. Experimental results show that joint training of rotation and weight improves the performance of quantized models in multiple settings.

Critical strengths
  1. The paper provides comparisons with some QAT and PTQ methods and demonstrates that supporting rotation yields independent gains across certain architectures.

  2. The experiments cover a variety of models and tasks, with compensation providing clear benefits in some settings.

Critical weaknesses
  1. The rotation ablation or comparison is insufficent. SpinQuant has already implemented the learning of orthogonal rotations and explored parameterization methods based on matrix exponentials. Current ablation experiments fail to demonstrate that the "shared-group parameterization" scheme employed by REVQ outperforms fixed rotations or other learnable rotation schemes under identical conditions.

  2. Ratotation and vector quantization methods have already been well studied in LLM quantization method. It's hard to identify the contribution of the paper without the experiments on LLM quantization.

  3. Vector quantization or product quantization achieves less real speed-up and marginal accuracy gain (0.4-0.6 top1 acc gain in w4a4, according to Table 3) compared to those fixed-point quantization methods. The authors better provide real-time speed-up comparison between the presented methods and other QAT methods. In general, vector quantization or product quantization achieves less speed up than fixedpoint quantization.

  4. It's hard to attribute the source of the performance gains for Haar-based approach.In Appendix A.8, for a given nominal rank, the Haar-based approach employs more parameters than SVD, potentially reaching up to twice the total rank. On the ImageNet W4A4 benchmark, the observed gains are at most 0.11 percentage points.

  5. The abalation in Table 4 and Table 9 shows that the the low rank residual compensation provides little accuracy gain. And sometimes the the proposed low rank residual compensation achieves negative effect and lowers down the accuracy.

Questions for authors
  1. How does REVQ perform compared to fixed rotations or other learned rotation methods, given identical group sizes, quantization coverage, and training budgets?

  2. Does the gain brought by this compensation still exist when the parameter amount and optimization budget remain consistent? Among them, how much improvement has cross-layer supervision specifically contributed?

  3. What's the bit-width definition for the vector product quantization in this paper, is the codebook ( centroid) storage taken into the consideration?

  4. In the inference stage, does the low-rank compensation still exist? does it slow down the inference of deep neural network?

  5. The low-rank compensation has a similar form with the LoRA methods? does adding a LoRA to the quantized network improves the accuracy and outperforms the proposed low-rank compensation?

3 comments

Thank you for your valuable comments and below are our responses to the weaknesses and questions.

W1&Q1: Fair comparison with SpinQuant

Response: We have clarified the rotation formulation and compared the proposed method and SpinQuant in methodology and additional experiments under the same budget.

  1. Clarifying the rotation formulation.

We have revised Section 3 to better present the formulation of proposed learnable rotations in the revised manuscript. The revised quantization functions are defined as follows:

Q_a(x,R)=s_a(xR)\left[\operatorname{clip}\left(\left\lfloor\frac{xR}{s_a(xR)}+z_a(xR)\right\rceil,0,2^{b_a}-1\right)-z_a(xR)\right]R^\top.$$ The quantized output is $\hat{Y}=Q_a(\hat{X},R_a)Q_w(W,R_w)$. Our method uses separate orthogonal rotation $R_a$ and $R_w$ for activations and weights. This formulation constructs a quantization coordinate system that aligns the singular vectors of the activation and weight data during training. Each g-dimensional sub-vector is quantized on a lattice whose orientation is a learnable orthogonal rotation. QAT lets the rotation be trained with quantization objective, so the lattice can align with correlated directions in the weight or activation distribution. Note that we do not learn centroids or build a codebook. The bit-width and quantization levels are the same as those of uniform quantization. To accelerate convergence and improve training efficiency, all experiments reported in the current manuscript use the tied case $R_a=R_w$. The main difference of our work from FSQ [C6] lies in the geometry of quantization grid. FSQ uses a fixed, axis-aligned grid, which is the special case $R=I$ in our formulation. 2. **Additional experiments with matched quantization settings** We use the same quantization scheme (group-32 symmetric uniform quantization for weight, dynamic group-32 asymmetric uniform quantization for activation) and compare our method with SpinQuant[B1]. The table below shows that our method consistently outperforms the baselines across different architectures. | Method | W-A | ViT-S | ViT-B | Swin-S | Swin-B | |---|---|---|---|---|---| | Full-Precision | 32-32 | 81.4 | 85.2 | 83.21 | 85.16 | | SpinQuant | 4-4 | 76.17 | 80.93 | 82.07 | 83.88 | | FIMA-Q | 4-4 | 78.024 | 82.908 | 81.228 | 83.368 | | GPLQ | 4-4 | 77.44 | 82.19 | 81.94 | 84.18 | | Ours | Rotation-QAT | 80 | 84.27 | 82.61 | 84.63 | | | Haar (rank=1) | 80.11 | 84.35 | 82.53 | 84.66 | [B1] Liu Z, Zhao C, Fedorov I, et al. Spinquant: Llm quantization with learned rotations. ICLR 2025. [B2] Wu Z, Wang S, Zhang J, et al. FIMA-Q: Post-training quantization for vision transformers by fisher information matrix approximation. CVPR 2025. [B3] Liang G, Liu X, Wu J. Gplq: A general, practical, and lightning qat method for vision transformers. NeurIPS 2026. **W2: Evaluation QAT on LLMs** **Response**: Our work focuses on QAT for vision Transformer based architectures. According to the reviewer's concern, we further consider evaluating Transformers for language modeling. Due to substantial computational resources for QAT for LLMs, we conducted 2-bit weight-only QAT (W2A32) on GPT2-124M using OpenWebText. We applied the proposed groupwise learnable rotations and used the FP32 model as the teacher for knowledge distillation. Under the same training configuration, our method achieves lower perplexity than the full-precision baseline on both WikiText2 and C4. In terms of training cost, one full training epoch takes more than 10 hours on a single A800 GPU. | Bit width | Initialization | WikiText2 PPL | C4 PPL | Avg PPL | |---|---|---|---|---| | FP32 | — | 28.88 | 44.29 | 36.59 | | W2A32 | Pretrained | 26.82(↓2.06) | 41.30(↓2.99) | 34.06(↓2.53) |

W3: Inference latency compared with other QAT methods

Response: As clarified in our response to W1, our product vector quantization formulation does not require storing centroids or a codebook, so the codebook lookup overhead associated with conventional VQ/PQ does not apply to our implementation. The additional inference operations include online groupwise rotations and low-rank compensation. Our current implementation uses fake quantization, including quantization and dequantization, it does not demonstrate an inference speedup over FP16 or represent the performance of native INT4 kernels. Specifically, we measured the end-to-end inference latency of REVQ on ViT-S/B and Swin-S/B using a single RTX 3090, and compared it with the FP16 baseline, standard W4A4 fixed-point QAT, and GPLQ+QwT. (Stage2 uses SVD rank-1 compensation in this evaluation.)

ModelBatch SizeFP16 (ms)Fixed-point QAT (ms)GPLQ + QwT (ms)REVQ-Stage1 (ms)REVQ-Stage2 (ms)
ViT-S327.4010.98 (+3.58)11.66 (+4.26)11.42 (+4.02)12.14 (+4.74)
ViT-B3221.8128.46 (+6.65)30.28 (+8.47)29.85 (+8.04)31.19 (+9.38)
Swin-S3223.5534.18 (+10.63)35.85 (+12.30)33.87 (+10.32)35.62 (+12.07)
Swin-B3234.1647.24 (+13.08)49.48 (+15.32)48.23 (+14.07)50.33 (+16.17)
ViT-S11.222.23 (+1.01)2.30 (+1.08)2.19 (+0.97)2.56 (+1.34)
ViT-B11.693.37 (+1.68)3.56 (+1.87)3.41 (+1.72)3.56 (+1.87)
Swin-S12.084.39 (+2.31)4.65 (+2.57)4.54 (+2.46)4.81 (+2.73)
Swin-B12.365.31 (+2.95)5.62 (+3.26)5.46 (+3.10)5.78 (+3.42)

At batch size 32, REVQ achieves higher accuracy than GPLQ+QwT with modest latency differences, as reported in Table 3 of the revised manuscript. On Swin-S, it improves Top-1 by 0.58 percentage points while reducing latency by 0.6%. On ViT-S and ViT-B, it improves Top-1 by 2.61 and 2.15 pp with latency increases of 4.1% and 3.0%. These results characterize the accuracy–latency trade-off of our current implementation.

W4&W5: Haar performance and low-rank compensation gains

Response: Thank you for raising this concern. To distinguish the effect of compensation from additional parameters, we conducted further experiments on ViT-S with a group size of 32 across three random seeds. We compare Haar(rank=1) with SVD(rank=2) at comparable parameter counts. The results show that Haar uses fewer trainable parameters and achieves slightly higher accuracy in all three runs.

Δ\Delta Paramsseed 42seed 84seed 168mean ± stdΔ\Delta mean ± std
Rotation-QAT23,808(+0.112%)8079.9880.0580.010 ± 0.036—
resume training23,808(+0.112%)80.0179.9780.0780.017 ± 0.050—
SVD r=258,368(+0.275%)80.0880.0680.1780.103 ± 0.059+0.093 ± 0.023
Haar r=154,528(+0.257%)80.1180.0880.2180.133 ± 0.068+0.124 ± 0.033

Why compensation gains are modest. Low-rank residual compensation provides modest gains and can reduce accuracy in some configurations, as shown in Tables 4 and 9. There are two possible contributing factors: (i) compensation is optimized in a separate stage from the rotations, which may limit their joint adaptation; and (ii) the remaining scope for accuracy recovery is limited after Stage 1. We will consider exploring jointly optimizing rotations and compensation to improve the coordination.

Q2: Compensation gains under controlled budgets and cross-layer supervision

Response: We evaluate compensation gains under controlled budgets and cross-layer supervision below.

  1. Parameter and training-budget controls

As discussed in our response to W4&W5, we compare Haar(rank=1) and SVD(rank=2) at comparable parameter counts and include a continued-training baseline with the same additional training budget. SVD and Haar improve mean Top-1 over Rotation-QAT by approximately 0.09 and 0.12 percentage points, compared with 0.01 pp from continued training alone. Both compensation variants improve accuracy over Rotation-QAT across all three random seeds.

  1. Ablating cross-layer supervision

To isolate the contribution of explicit cross-layer supervision, we remove LcleanL_{clean} from Stage-2 loss (defined in Section 3.4), while keeping the parameter count and training configuration unchanged. The table below shows that adding LcleanL_{clean} can improve the mean Top-1 accuracy, yielding gains of 0.137 and 0.107 percentage points.

Stage-2 Lossseed 42seed 84seed 168mean ± stdΔ\Delta mean ± std
SVD r=2No LcleanL_{clean}80.0179.8380.0679.967±0.121—
SVD r=2—80.0880.0680.1780.103±0.0590.137±0.083
Haar r=1No LcleanL_{clean}80.0479.9280.1280.027±0.101—
Haar r=1—80.1180.0880.2180.133±0.0680.107±0.047

Q3: Bit-width definition and codebook storage

Response: As shown by the quantization functions provided in our response to W1&Q1, we apply uniform scalar quantization in a learned rotated coordinate system. The learned rotations determine the orientation of the quantization grid in the original coordinate system, while quantization within the rotated space remains coordinate-wise and uniform. For example, W4A4 denotes 4-bit quantization of each scalar weight and activation in the rotated space. A g-dimensional quantized sub-vector therefore requires 4g bits for its quantized values. The grid is implicitly defined by the uniform scalar quantizer and the learned rotation, so no explicit centroid codebook is stored.

We separately account for auxiliary storage, including FP16 groupwise scales, FP16 low-rank compensation factors, and the activation-side rotation matrices retained for inference. For ViT-S with SVD(r=1), the resulting quantization BPW is 4.53, and the full-model storage cost, including unquantized parameters, is 4.97 bits per parameter. This corresponds to 13.71 MB and 3.22× compression relative to the 44.10 MB FP16 baseline. Results for all four models are provided in the Table 14 in Appendix A.9.

Q4: Inference overhead of the compensation branch

Response: The low-rank compensation branch remains active during inference and adds a low-rank correction to the linear layer output, introducing additional computation. To quantify its practical impact, we compare the end-to-end latency of REVQ-Stage2 with REVQ-Stage1 (which is defined in Section 3.4) on a single RTX 3090. As reported in our response to W3, across ViT-S/B and Swin-S/B, Stage2 increases latency by 0.72-2.10ms (4.4-6.3%) at batch size 32, and 0.15-0.37ms (4.4-16.9%) at batch size 1. These results show that compensation incurs a measurable inference cost, and the full table is provided in the Table 16 in Appendix A.9.

Q5: Objectives and initialization compared with LoRA

Response: Thank you for your question. Both our compensation and LoRA use additive low-rank weight updates, and share a similar algebraic structure. The main differences lie in the initialization and optimization objectives. Standard LoRA is designed for task adaptation and learns low-rank updates through a downstream task loss, with one factor initialized randomly and the other to zero. Our compensation is initialized through closed-form regression using full-precision reference outputs, explicitly accounting for errors propagated from preceding quantized layers. During Stage2, the factors are further optimized with explicit cross-layer supervision. We have not conducted a direct comparison with a LoRA baseline. As discussed in our response to W4&W5, we plan to explore low-rank compensation that is more closely coordinated with the rotations learned in Stage 1.

Reviewer_7td3OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

REVQ is a two-stage QAT method for joint W4A4/W3A3 quantization of ViT, Swin, BERT, and CNNs. Stage 1 inserts a learnable block-diagonal 32×32 rotation (parametrized via the Lie algebra of SO(32)) before each linear layer and trains it jointly with the weights to reduce local quantization error. Stage 2 freezes everything and adds a rank-1 low-rank correction to each quantized weight, fitted to the error propagated from earlier layers and initialized by SVD or a Haar-wavelet split.

Critical strengths
  1. Broad evaluation: ViT/Swin/CNNs/BERT on ImageNet, CIFAR-100, COCO detection and segmentation, and MLM, all with one recipe and on a single RTX 3090.

  2. Clean error decomposition (Eq. 5) into local-layer and cross-layer terms, with ablations (Tables 5, 6) that measure each term directly rather than only accuracy.

Critical weaknesses
  1. Comparisons are unfair and omit SOTA. REVQ is a QAT method but Table 3 compares almost only against PTQ methods (PTQ4ViT, APQ-ViT, RepQ-ViT), and under a different quantizer: REVQ uses group-32 weight and dynamic group-32 activation quantization while the baselines use per-tensor/per-channel static quantizers. REVQ's own untrained W4A4 ViT-S already beats every baseline, so the gap is the quantizer, not the method.

  2. The second contribution has no measurable effect. Rank-1 residual compensation changes ImageNet Top-1 by −0.08 to +0.11 pp with no seeds or std; Haar vs SVD differ by ≤0.06 pp while Haar uses 1.5× parameters.

  3. Limited novelty. Learned rotations for W4A4 (SpinQuant, OSTQuant, FlatQuant, ButterflyQuant) and groupwise block-diagonal rotations (DuQuant, DuQuant++) exist; the Lie-algebra parametrization only swaps SpinQuant's Cayley retraction for the exponential map. Rotation plus low-rank residual for W4A4 exists in SVDQuant and ResQ, neither cited.

  4. Inflated claims. "W4A4 above FP" on CIFAR-100 results from extra task-loss fine-tuning on top of a 10-epoch FP head fine-tune, not from quantization. "Product vector quantization" is groupwise uniform scalar quantization with no codebook.

  5. No inference cost reported for the online 32×32 rotation per linear layer and the per-group dynamic activation scales.

Questions for authors

None

3 comments

Thank you for your comments and below are our responses to the weaknesses.

W1: Fair comparison with state-of-the-art methods

Response: To remove the effect of quantizer, we use the same quantization scheme (group-32 symmetric uniform quantization for weight, dynamic group-32 asymmetric uniform quantization for activation) and compare our method with recent representative methods such as FIMA-Q [A1] for PTQ, SpinQuant [A2] using learnable rotations, and GPLQ [A3] for QAT. The table below shows that our method consistently outperforms the baselines across different model architectures. This justifies the contribution of the proposed method to the performance gain.

MethodW-AViT-SViT-BSwin-SSwin-B
Full-Precision32-3281.4085.283.2185.16
FIMA-Q4-478.0282.9181.2283.37
SpinQuant4-476.1780.9382.0783.88
GPLQ4-477.4482.1981.9484.18
ProposedRotation-QAT80.0084.2782.6184.63
+Haar(rank=1)80.1184.3582.5384.66

[A1] Wu Z, Wang S, Zhang J, Chen J, Wang Y. FIMA-Q: Post-training quantization for vision Transformers by fisher information matrix approximation. CVPR 2025.

[A2] Liu Z et al. SpinQuant: LLM quantization with learned rotations. ICLR 2025.

[A3] Liang G, Liu X, Wu J. GPLQ: A general, practical, and lightning QAT method for vision Transformers. NeurIPS 2026.

W2: Measure effect of second contribution

Response: To distinguish the effect of compensation from additional parameters, we have added experiments across three random seeds (42, 84, and 168) to compare Haar (rank=1) with SVD (rank=2) at closely matched parameter counts, together with a continued-training baseline using the same additional training budget (model=vit-small, group-size=32).

Δ\DeltaParamsseed 42seed 84seed 168mean ±\pm stdΔ\Deltamean ±\pm std
Rotation-QAT23,808(+0.112%)8079.9880.0580.010 ±\pm 0.036—
resume training23,808(+0.112%)80.0179.9780.0780.017 ±\pm 0.050—
SVD r=258,368(+0.275%)80.0880.0680.1780.103 ±\pm 0.059+0.093 ± 0.023
Haar r=154,528(+0.257%)80.1180.0880.2180.133 ±\pm 0.068+0.124 ±\pm 0.033

The results show that Haar method achieves a modest benefit over SVD with comparable parameter counts.

W3: Novelty and positioning relative to prior rotation and low-rank methods

Response: We have revised Section 3 to clarify the formulation and its interpretation. The revised quantization functions for weight and activation quantization are defined as follows:

Q_a(x,R)=s_a(xR)\left[\mathrm{clip}\left(\left\lfloor\frac{xR}{s_a(xR)}+z_a(xR)\right\rceil,0,2^{b_a}-1\right)-z_a(xR)\right]R^{\top}.$$ The quantized output is: $$\hat{Y}=Q_a(\hat{X},R_a)Q_w(W,R_w).$$ Below we elaborate on the novelty in learnable rotations in our method and explain its distinction in compensation from existing methods such as SVDQuant [A4] and ResQ [A5]. 1. **Novelty in learnable rotations** Our method uses separate orthogonal rotation $R_a$ and $R_w$ for activations and weights. This formulation constructs a quantization coordinate system that aligns the singular vectors of the activation and weight data during training. By learning the rotation angles, it adapts this coordinate system to the data distribution. It requires no separate centroid learning or codebook construction, and the quantization bit-widths and centroids remain the same as those used in uniform quantization. To accelerate convergence and improve training efficiency, all experiments reported in the current manuscript use the tied case $R_a=R_w$. 2. **Exponential-map rotation parameterization** Furthermore, we use Lie algebra parameterization as a mathematical means to obtain rotations through the exponential map. We agree that it and the Cayley transform both parameterize rotation matrics, and we do not claim this as a contribution. The key distinction lies in what the rotation is used for. SpinQuant learns rotations via Cayley SGD to smooth data distribution for scalar quantization. We learn decoupled rotations $R_a$ and $R_w$ to define product vector quantization coordinates, and the rotation angles are learned to minimize the quantization error in original space. The performance comparison with SpinQuant shows consistent gains in W4A4 quantization on various ViT and Swin backbones. 3. **Distinction in compensation from SVDQuant and ResQ** We have clarified the relationship between our method and SVDQuant [A4] and ResQ [A5] in Setion 2.3. SVDQuant uses a low-rank branch to absorb dominant components of the smoothed weights, while ResQ preserves high-variance activation subspaces at higher precision and applies random rotations within the subspaces. Our compensation formulation explicitly accounts for accumulated errors propagated from preceding quantized layers. As shown in Eq.5, we separate the cross-layer error from the local-layer quantization errors, and use an output-residual objective to obtain a closed-form initialization of the low-rank compensation factors. These factors are then refined through training with explicit cross-layer supervision. [A4] Li M, Lin Y, Zhang Z, et al. Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models. ICLR 2025. [A5] Utkarsh Sa, Sayeh S, Kaushik R, Xin W. ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals. ICML 2025. **W4: Fair comparisons on CIFAR-100 and Quantization Formulation** **Response**: We have presented CIFAR-100 results under a matched training budget and clarify the main difference regarding the product quantization formulation 1. **CIFAR-100 results under a matched training budget** We have retrained the CIFAR-100 baselines under the same training budget and updated Table 2 in the revised manuscript. On Swin-T, REVQ still outperforms the full-precision baseline by 0.24 percentage points in Top-1. On ViT-B and ViT-L, REVQ achieves 92.24% and 94.20% Top-1, within 0.37 and 0.03 pp of their fp baselines. 2. **Clarifying the Product Quantization Formulation** As explained in our earlier responses, REVQ does not learn centroids or store a codebook. It uses the same bit-width and scalar quantization levels as uniform quantization. In REVQ, each g-dimensional sub-vector is quantized on a lattice whose orientation is a learnable orthogonal rotation. QAT optimizes the rotation jointly with the quantized model, allowing the lattice orientation to adapt to correlations in the weight or activation distribution. The product structure is taken across the G groups of a layer, and the lattice orientation is the vector-level quantity to be optimized. Regarding the related work FSQ [C6], the main difference lies in the geometry of quantization grid. FSQ uses a fixed, axis-aligned grid, which is the special case $R=I$ in our formulation.

W5: Inference overhead of online rotations and dynamic activation scaling

Response: We focus on improving accuracy and reducing storage requirements. Table 14 in the revised manuscript provides detailed storage accounting. For example, on ViT-S, our method with rank-1 Haar compensation achieves approximately 3.2$\times$ compression relative to FP16, compared with 2.58$\times$ for GPLQ+QwT.

According to the reviewer's concern, we measure the inference overhead of online rotations and dynamic activation scaling. We evaluate (i) online block-diagonal rotation with 32×32 blocks applied at each linear layer and (ii) rotation together with per-token, per-group dynamic activation scale computation, jointly compiled using torch.compile. We perform the evaluations on a single idle RTX 3090 using CUDA graphs, and report the median latency over repeated iterations. We used FP16 GEMMs with matching shapes and the same aggregation procedure as the reference.

ModelBatch SizeFP16 GEMM(ms)Rot(ms / %)Rot+dynamic A-scale(ms / %)
ViT-S324.741.04 / 22.0%1.86 / 39.3%
ViT-B3224.223.01 / 12.4%5.31 / 21.9%
Swin-S3216.285.07 / 31.2%8.63 / 53.0%
Swin-B3229.637.82 / 26.4%12.43 / 42.0%
ViT-S10.540.17 / 30.5%0.25 / 45.2%
ViT-B11.130.19 / 16.8%0.31 / 27.6%
Swin-S11.110.34 / 31.0%0.57 / 51.2%
Swin-B11.260.40 / 31.6%0.64 / 50.7%

At batch size of 32, rotation alone costs 12.4%-31.2% of the FP16 GEMM latency, while rotation combined with dynamic activation scale computation costs 21.9%-53.0%. At batch size of 1, corresponding costs are 16.8%-31.6% and 27.6%-51%. These results reveal non-negligible overhead in current implementation, with larger relative costs on the Swin models. The elementwise operations and small matrix multiplications involved have low arithmetic intensity, suggesting that memory traffic contributes to this overhead. A dedicated fused kernel could reduce intermediate memory accesses, and we leave its implementation and evaluation to future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.