- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes QuAR for unsupervised test-time adaptation of quantized vision transformers. It applies a per-channel affine correction at the activation-quantizer inputs in the last four transformer blocks, using source statistics and running target statistics. A global branch also corrects the CLS representation at the classifier input. Model weights and quantizer parameters remain frozen, and adaptation requires neither backpropagation nor additional forward passes. The authors report accuracy improvements across several bit-widths and distribution-shift benchmarks, with ablations on the recalibration scope and matching strength.
Critical strengths
QuAR outperforms the compared backpropagation-free adaptation methods across multiple bit-widths on ImageNet-C, and the advantage persists on the full test set. Results on other datasets and backbones further support the empirical value of the design.
Critical weaknesses
[W1] - The conceptual contribution is limited, and the mechanism analysis does not fully explain the accuracy gains
The core operation matches per-channel target activation means and scales to source statistics, which is closely related to existing activation-statistics alignment methods. The main distinctions are its placement at late quantizer inputs and the combination of partial matching with online estimation.
The authors argue that distribution shift contracts some channels, reducing the number of quantization codes they occupy, and that restoring the source scale recovers resolution. This is plausible, but the connection to classification accuracy remains incomplete. Given the reported small changes in clipping and quantization-error magnitude, bringing the code distribution closer to the source does not by itself explain why decisions improve.
The reduced classification margin under shift explains why quantized models become more vulnerable, but does not establish how QuAR repairs that vulnerability. Full matching restores more of the code distribution yet achieves lower accuracy, further showing that this diagnostic alone cannot explain performance. Evidence on how recalibration changes classification margins or harmful quantization-induced prediction flips would help separate general feature-alignment gains from gains specific to quantization. Because the method itself is straightforward, the incomplete explanation limits my assessment of its contribution.
[W2] - The accuracy-efficiency trade-off under actual quantized deployment has not been established
Resource-constrained edge deployment is a central motivation, but both the server and Jetson experiments use floating-point simulation of quantization rather than actual integer kernels. Running this implementation on Jetson demonstrates that it executes on an edge device; it does not establish that the same accuracy gains and relative overhead hold under real low-bit inference.
QuAR requires activation corrections to remain active during inference while target statistics are updated online. The reductions and transformations at multiple interfaces may add memory traffic and kernel-launch overhead, or interfere with existing operator fusion. Once integer kernels accelerate the backbone, these operations may account for a larger fraction of inference time. A single forward pass therefore does not establish low deployment overhead.
At least one real W4A4 or W8A8 implementation, with end-to-end accuracy and latency measurements, is needed to assess the claimed deployment benefits.
Questions for authors
-
How does recalibration change classification margins or harmful quantization-induced prediction flips?
-
Can you report accuracy and end-to-end latency under real W4A4 or W8A8 execution, including statistics updates and correction, against baselines on the same stack?
4 comments
Official Comment by Authors (2/2)
[Weakness 1-3] The conceptual contribution
W1-1 and W1-2 add decision-level evidence, harmful flips and margins across methods and . These results sharpen our contribution, a quantization-specific failure mode and a correction aimed at it directly. Under shift, activations leave the ranges the frozen quantizers were calibrated on, which costs the contracted channels resolution and lets the unchanged quantization error flip more decisions. Existing quantized TTA acts after these quantizers or upstream of them; QuAR corrects the activation at the input of each frozen quantizer in the late blocks. The per-channel affine form is shared with statistics alignment, but the target, a frozen quantizer's operating point, is not; what the paper adds is the evidence that its gain is quantization-specific and an analysis of where and how much to correct. The distinctions the reviewer names, namely placement, online estimation and partial matching, are the mechanism.
- Only the late quantizer inputs carry the quantization-specific gain. Moved after the quantizer or onto the residual stream, the same affine gains and points less at W6A6 (Table 38), and applied to all twelve blocks it falls below No Adapt (Table 5), because the quantized path amplifies an early correction's residual where the full-precision path does not (; Appendix G).
- Pooling the moments over the stream is what the post-nonlinearity inputs need. A block has four quantizer inputs, two after a LayerNorm and two after a nonlinearity (attention, GELU). The post-LayerNorm pair gains the same with pooled or batch moments; the post-nonlinearity pair adds points only with pooled moments, in line with one batch estimating their gain to times worse than at the post-LayerNorm pair (Tables 25 and 26, Appendix K).
- Partial matching is optimal because the estimated gain carries error. Matching fully applies the whole estimated gain and with it the whole estimation error, so accuracy peaks at a partial strength (W1-2).
[Weakness 2] Real quantized deployment
We now have the measurement the reviewer asks for (Table I). The ViT-B/16 is exported to a QDQ graph; accuracy is read on ONNX Runtime int8 and end-to-end latency on the Orin Nano with TensorRT INT8, whose engine reproduces the ONNX Runtime accuracies of the unadapted model and QuAR ( and against and ; Appendix U).
Table I. Real W8A8 on integer kernels (ImageNet-C, 15 corruptions; one run, seed 0). Baselines run their own code on the integer graphs; latency includes each method's host-side work.
| W8A8 on integer kernels | top-1 | latency, batch 64 | batch 16 |
|---|---|---|---|
| No Adapt | ms () | ms () | |
| T3A | ms () | ms () | |
| FOZO | ms () | ms () | |
| FOA | ms () | ms () | |
| ZOA | ms () | ms () | |
| NEO | ms () | ms () | |
| QuAR | ms () | ms () |
- The accuracy gain transfers. QuAR remains the most accurate method, its gain () tracks the fake-quant gain () and the baselines keep their accuracy order; every method sits slightly lower in absolute terms because ONNX Runtime's PTQ is plainer than PTQ4ViT's.
- The single-pass overhead transfers. QuAR runs at – the unadapted engine, against – for the two-pass methods (– in floating-point simulation).
- Fusion and memory traffic. The profile shows the GEMMs unchanged ( vs ms, batch 64), so the Q/DQ fusion is intact; the corrections alone are free ( ms without the moment reductions). What the method adds is the memory traffic of the online statistics, ms in the token-sum GEMMs and ms in the pointwise kernels that square and write their outputs, plus a ms host update; kernels account for of the added ms, so launch overhead is at most ms.
[Question 1] Margins and flips
We thank the reviewer for the question and added this measurement (Table G). Recalibration at the quantizer inputs removes a quarter of the harmful flips (% %) and returns the median logit margin to its full-precision value (). See our replies to Weakness 1-1 and 1-2 for details.
[Question 2] Real W8A8 execution against baselines on the same stack
We implemented W8A8 on a real integer stack, with the statistics update and correction included and every forward-only baseline on the same stack (Table I). QuAR is the most accurate (, above ZOA) at – against the two-pass baselines' –. See our reply to Weakness 2 for details.
Thank you for the additional experiments. The real W8A8 results address much of my deployment concern. The margin and prediction-flip analyses strengthen the mechanism explanation, but do not fully isolate quantization-specific gains from general feature alignment. Given the additional evidence, I will raise my score to 3 (weak accept).
We thank the reviewer for the careful reading and constructive feedback, as well as for raising the score. We would welcome any further questions during the discussion period.
Official Comment by Authors (1/2)
[Weakness 1-1] Margins, harmful flips, and what is quantization-specific
We thank the reviewer for pinpointing the decision-level evidence that links the analysis to accuracy, and we have now measured it. We agree that a restored code distribution does not by itself explain the decisions, so we measured them. Running the full-precision and the quantized network over the same images, each method applied to the quantized one, we count harmful flips (correct in full precision, wrong once quantized) and helpful ones (the reverse), with the median logit margin (top-1 minus top-2) of the adapted quantized network (Table G).
Table G. Harmful and helpful flips and median logit margin per method (W6A6, 15 corruptions, seed 0).
Recalibration at the quantizer inputs removes a quarter of the harmful flips and returns the margin to its full-precision value. Quantization lowers the median margin from 1.61 in full precision to 1.31 (Table 34 reports the mean, 2.05); recalibration restores all of it (1.61) and removes a quarter of the harmful flips (11.78% → 8.97% of images, −2.8 points), while the head-level methods, NEO and QuAR (Global-only), acting after every quantizer, recover about a third of the margin (1.40–1.42) and fewer flips; per corruption the reduction tracks the gain (Appendix Q). This is why small changes in clipping and error magnitude are enough. A flip depends on where the error sits against the margin, and recalibration leaves the error nearly unchanged but moves the operating point (Section 2.5).
The quantization-specific part of the gain is the excess over what the same correction gives the full-precision network, and Table G shows it as harmful flips. Applied to the full-precision network, the correction gains +3.8 points; applied to the quantized network, +6.1 to +7.6 (Fig. 1a). The first number is what per-channel statistics alignment buys on its own; the excess is what restoring the quantizer's operating point buys on top, and it exists only at the quantizer input, since the same affine after the quantizer or on the residual stream falls below No Adapt at W3A3 and W4A4 (Table 38). Table G splits the same way. Helpful flips add +3.9 points, the general-alignment part, against the +3.8 of the full-precision network; harmful flips drop −2.8 points, the quantization-specific excess.
[Weakness 1-2] Why full matching restores more codes yet is less accurate
The gain the correction applies is an estimate, and full matching applies all of its error. The correction rescales each channel by σs/σt with σt estimated from the stream, so the gain carries an error, 2.3–2.8% after a LayerNorm and 9.9% and 42.6% at the post-attention and post-GELU inputs (Table 26, Appendix K). The strength λ sets how much of the correction, and of that error, each activation receives. The code diagnostic averages over tokens and batches, so it sees the restoration, which only improves with λ, but not the per-activation error; accuracy sees both, and since the injected error grows faster than the restoration the best strength is partial (the shrinkage of Corollary V.2). Our paper fixes λ=0.5 once; Table H shows both sides.
Table H. Local-branch match strength λ (W6A6, seed 0; code restoration pooled over the 16 local inputs and the head input).
Code restoration rises with every step of λ, 8.8% to 84.3% (the 8.8% at λ=0 is the global branch's share at the head input), while top-1, harmful flips and the margin peak around the deployed λ=0.5 and worsen again at λ=1 (−0.8 top-1 and +1.2 points of harmful flips). The diagnostic shows where the method acts; the flips and the margin show what it does to the decisions, which is why they track accuracy and the diagnostic does not. The revision adds one sentence to Section 2.5 and both tables to Appendix Q.