- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3(+1)
- Presentation
- 2(+1)
- Soundness
- 2
Read review
Paper summary
This work proposes InvAR, a post-hoc output rectification method for low-bit quantized vision models. Instead of directly regressing full-precision (FP) logits on quantized (Q) logits, InvAR estimates a classwise FP-to-Q affine distortion and analytically inverts it. To stabilize inversion with limited calibration data, it shrinks unreliable classwise deviations toward a shared affine reference. Experiments on ImageNet-1K show consistent accuracy gains when InvAR is applied to models quantized with FIMA-Q, LS-ViT, and APHQ-ViT, with additional results on CNNs and COCO.
Critical strengths
-
Novel perspective on output rectification. In prior work, output-level rectification has been performed in the Q-to-FP direction. Reversing this idea and performing the rectification in the FP-to-Q direction is a novel perspective.
-
Compatibility with existing PTQ methods. The proposed method can be combined with other PTQ methods.
Critical weaknesses
-
Missing theoretical justification for FP-to-Q inversion. While the paper empirically shows that estimating the FP-to-Q distortion and inverting it outperforms direct Q-to-FP regression, it provides no theoretical justification for this advantage.
-
Missing comparisons with existing rectification methods. The authors apply vanilla InvAR and InvAR to models quantized with FIMA-Q, LS-ViT, and APHQ-ViT, but the only competing baseline is ForwardAffine, a direct Q-to-FP regression constructed by the authors. However, CAT already rectifies the outputs of quantized models in a post-hoc manner, and quantization error compensation methods such as QwT and QwT-v2 can also serve as baselines for output rectification when applied to the final head outputs . To demonstrate the effectiveness of InvAR, the authors should apply these methods to the same quantized models and compare them directly with InvAR.
Minor weakness
Presentation and clarity. Overall, the manuscript does not feel well organized, which makes it difficult to follow. First, the caption of Figure 1 is too long, and the figure itself is so small that the caption asks readers to zoom in. Second, Section 3.3 introduces several statistical components, such as HC3 variance estimates and orthogonal residual shrinkage (HC3-ORS), in a dense manner without sufficient intuition, and key details are deferred to Appendix A. Third, there is a typographical error in the Introduction ("Sections 4.2 and 4.2"). More broadly, the writing reads more like a technical report than a research paper. The method is described largely through procedural steps and implementation rules, such as the slope floor and the numerical thresholds in Appendix A, while the motivation behind the design choices in Section 3.3 is discussed only briefly. I encourage the authors to revise the manuscript to improve its overall organization and readability.
Questions for authors
Q1. The study appears to be overly focused on rectifying class logits (in image classification, and object detection and instance segmentation). However, I think that extending quantization methods to dense prediction tasks is a key challenge in recent quantization research. Can the authors generalize the proposed method to dense prediction tasks?
Q2. Can the authors provide a theoretical justification for why the FP-to-Q approach necessarily outperforms the Q-to-FP approach?
Q3. Can the authors apply CAT, as well as QwT and QwT-v2 applied to the final head outputs , to models quantized with FIMA-Q, LS-ViT, and APHQ-ViT, and evaluate the effectiveness of InvAR through a direct comparison with these methods?
Q4. The experimental settings for Table 11 do not seem to be specified in detail. I would ask the authors to clarify them (e.g., bit-width).
7 comments
Response 2/3: Comparisons with CAT, QwT, and QwT-v2 on the same models (W2/Q3)
W2 / Q3. The only baseline, ForwardAffine, is constructed by the authors. Compare InvAR directly with CAT, and with QwT and QwT-v2 applied to the final outputs, on the same quantized models.
As requested, new Table 3 applies CAT, QwT, and QwT-v2 to the final outputs of 27 configurations, each run with three random seeds: FIMA-Q, LS-ViT, and APHQ-ViT on DeiT-S, ViT-S, and Swin-S at W2/A3, W3/A3, and W4/A4. All rectifiers share the frozen checkpoints and the original 1,024 unlabeled calibration images. Entries are mean Top-1 gains (pp) over the unrectified checkpoints; Foldable indicates whether a rectifier folds into a compatible quantized head without changing integer weights (Section 3.4).
| Output rectifier | W2/A3 | W3/A3 | W4/A4 | Mean | Extra params | Foldable |
|---|---|---|---|---|---|---|
| CAT (K=64) | -1.44 | -1.24 | -0.63 | -1.10 | No | |
| QwT-logit | -1.30 | -0.79 | -0.18 | -0.76 | No | |
| QwT-feature | +1.27 | +0.11 | +0.03 | +0.47 | No | |
| QwT-v2 (final outputs) = ForwardAffine | -2.05 | -0.88 | -0.19 | -1.04 | Yes | |
| Vanilla InvAR | +3.25 | +0.75 | +0.12 | +1.37 | Yes | |
| InvAR | +3.25 | +0.75 | +0.11 | +1.37 | Yes |
Both InvAR variants have higher mean gains than all external baselines in Table 3 at every bit-width. The strongest competitor, QwT-feature, additionally uses head-input features, stores about 190-385x more rectification parameters ( or ), and is not foldable, yet gains 0.90 pp less on average.
Why ForwardAffine is a meaningful baseline. We implement ForwardAffine ourselves, but it corresponds to existing output rectifiers: within the input-independent classwise affine form used throughout the paper (closed-form fitting, analytic inversion, coefficients; Section 3.1, Appendix G), it is the classwise least-squares fit inside CAT and coincides with QwT-v2's channelwise compensation applied to final outputs, so the two share one row above. Comparing it with inversion therefore isolates the fitting direction while holding form, parameters, and data fixed.
Adaptations (Appendix C.2). With the official CAT implementation, every tested K/PCA setting reduces mean accuracy, including K=1 (Table 9; LS-ViT on three architectures). As you suggested, all methods act on the final outputs, and our conclusions are limited to this output-level setting.
Response 3/3: Presentation, dense prediction (Q1), and detection settings (Q4)
Presentation. Figure 1 is too small and its caption too long; Section 3.3 is dense and lacks intuition; the paper reads like a technical report; and there is a typo.
We revised the manuscript along the four points you raised:
- Figure 1: now full-width, with a shorter caption that no longer asks readers to zoom in.
- Section 3.3: before Eqs. (4)-(6), it now explains why near-zero slope estimates destabilize inversion, why the shared reference preserves the Q model's rankings, and how estimated uncertainty controls retention; the HC3-ORS acronym is removed.
- Organization: Section 3 presents motivation before procedure, with the slope-attenuation derivation moved to Appendix A; Section 4 is organized around the research questions, with experiments grouped under each.
- Typo: the duplicated "Sections 4.2 and 4.2" is corrected.
Q1. Can the method be generalized to dense prediction tasks beyond class logits?
In the two settings we tested, yes:
- Pixelwise semantic segmentation (Section 4.6; Appendix F, Table 20). For RepQ-ViT-quantized Swin-T/S UPerNet on ADE20K, InvAR is fitted on 256 unlabeled images with variances estimated per image, because pixels within an image are correlated (Appendix F.2). InvAR improves mIoU at W4/A4 by 2.08/1.05, whereas direct regression reduces it by 5.81/4.84. Both inverse variants outperform the quantized model and direct regression on all 12 checkpoints (two models, W4/A4 and W6/A6, three seeds).
- Continuous outputs beyond class logits (Appendix E, Table 19). For W4/A4 Mask R-CNN (LS-ViT and APHQ-ViT; Swin-T/S), rectifying raw box offsets alone improves box AP more than rectifying class logits alone in all four configurations (+0.8 to +2.0 vs. +0.3 to +1.1), and rectifying both is best (+1.2 to +3.0 box AP, +0.6 to +1.6 mask AP).
Q4. Please specify the experimental settings of Table 11 (e.g., bit-width).
Original Table 11 is now Table 18. All models are W4/A4 Mask R-CNN or Cascade Mask R-CNN with Swin-T/S backbones, quantized with LS-ViT, FIMA-Q, or APHQ-ViT; "+Joint" denotes InvAR on both classification and box heads. InvAR is fitted on 256 unlabeled COCO train2017 images and evaluated on all 5,000 val2017 images, with detector weights and quantization parameters fixed. FP and Q outputs are paired on the Q model's proposals; Appendix E gives the pairing for both detectors.
We hope the revision addresses your concerns, and we would appreciate your taking it into account in your assessment. If any question remains, we would be glad to address it and further improve the manuscript.
Response 1/3: Theoretical justification for FP-to-Q inversion (W1/Q2)
Thank you for the careful review and for recognizing the FP-to-Q perspective as novel. References are to the revised manuscript; items marked new were added in the revision.
Short answer. The revision now derives how the two directions differ for classification and states sufficient conditions under which inversion preserves decisions that direct regression changes. This is not a universal guarantee: if all classes share the same rc and FP calibration mean, both maps rank classes identically, and with very scarce data unshrunk inversion is noisier, which InvAR's shrinkage addresses (Section 3.3). In the evaluated settings, the experiments below consistently favor inversion.
Theory. Beyond the slope attenuation in the original Section 3.2, the new Proposition 1 establishes the exact finite-sample relation
T^cDir(z)=rc2T^cInv(z)+(1−rc2)Zˉcfp
for OLS with intercepts in both directions, nonzero sample variances, and no regularization, slope clipping, or residual-distribution assumption. Here rc>0 is the paired-logit sample correlation and Zˉcfp the FP calibration mean. Direct regression thus replaces a fraction 1−rc2 of each class's image-specific inverse output with that class's calibration mean. Classification depends on cross-class rankings, and because rc varies across classes, this contraction can reorder classes even while reducing logit MSE. Appendix A.1, Eqs. (9)-(10), gives sufficient conditions under which inversion preserves a correct FP decision that direct regression changes.
Empirical evidence (identical checkpoints and calibration images):