- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes GLF-Q, a block-wise post-training quantization framework for Vision Transformers. The authors argue that logit-guided block reconstruction tends to overfit the limited calibration data, and that Hessian-guided block reconstruction suffers from approximation errors at low bit-widths.
The framework consists of three components. First, the Global-Local Feature (GLF) loss propagates the output of the quantized block through the frozen full-precision downstream layers and aligns the representation immediately before the task head, together with a local MSE on the block output as regularization. Second, offline Hadamard rotations are applied after converting LayerNorm into RMSNorm, which spreads activation outliers across channels without runtime overhead. Third, GELU is replaced with ReLU followed by MLP reconstruction. The weights and quantization scales are then optimized with STE for 3k iterations per block.
The method is evaluated on ImageNet with ViT, DeiT, and Swin at W3A3 and W4A4, and on COCO object detection and instance segmentation at W4A4. The authors report state-of-the-art results with standard uniform quantizers, particularly at W3A3. Additional experiments include ablations on the loss and each component, out-of-domain calibration with SUN397, 8-bit inference latency on TensorRT, and reconstruction time.
Critical strengths
- Strong W3A3 accuracy with uniform quantizers. GLF-Q achieves the best W3A3 accuracy on all seven ImageNet models while using only standard uniform quantizers, and the results are stable across seeds (Table 10). This is the main practical value of the paper for hardware-friendly deployment.
- Reconstruction efficiency. Most existing reconstruction methods require 20k iterations, whereas GLF-Q converges within 3k iterations. As a result, it is 2.0–2.6× faster than FIMA-Q in total training (reconstruction) time (Table 7). This matters for PTQ, where low calibration cost is the main advantage over QAT.
Critical weaknesses
- Limited novelty. The GLF-Q framework is a combination of existing techniques: downstream supervision with local regularization from PD-Quant [1], offline rotation from QuaRot [2] and SliceGPT [3], and MLP reconstruction from APHQ-ViT [4]. GLF differs from PD-Quant mainly in moving the supervision point from the logits to the penultimate-layer representation. The paper does not present any new analysis, observation, or motivation of its own. The stated motivation for each component largely restates that of the original work. As a result, the state-of-the-art performance appears to come from stacking existing methods rather than from a new idea.
- Insufficient contribution of the GLF loss. As the name GLF-Q suggests, GLF is presented as the main contribution, so it should account for a substantial part of the performance gain. However, the ablation in Table 4 shows the opposite. Without Hadamard rotation and MLP reconstruction, GLF alone performs worse than FIMA-Q [5] on six of the seven models, even though FIMA-Q uses neither of these components. On ViT-S, for example, GLF alone reaches only 60.37%, while FIMA-Q achieves 64.09%. Even with MLP reconstruction, which APHQ-ViT also uses, GLF performs only comparably to APHQ-ViT and is clearly worse on ViT-B. These results suggest that the state-of-the-art performance comes mainly from the borrowed components rather than from GLF itself.
[1] Liu, Jiawei, et al. "Pd-quant: Post-training quantization based on prediction difference metric." 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023.
[2] Ashkboos, Saleh, et al. "Quarot: Outlier-free 4-bit inference in rotated llms." Advances in Neural Information Processing Systems 37 (2024): 100213-100240.
[3] Ashkboos, Saleh, et al. "Slicegpt: Compress large language models by deleting rows and columns." International Conference on Learning Representations. Vol. 2024. 2024.
[4] Wu, Zhuguanyu, et al. "Aphq-vit: Post-training quantization with average perturbation hessian based reconstruction for vision transformers." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.
[5] Wu, Zhuguanyu, et al. "FIMA-Q: Post-training quantization for vision transformers by fisher information matrix approximation." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.
Questions for authors
- What limitation of ViT PTQ does this work address? The paper identifies two limitations. The first is that logit-guided reconstruction tends to overfit the limited calibration data, and Hessian-guided reconstruction suffers from approximation errors. The second is that activation outliers and the negative tail of GELU waste the quantization range. However, the overfitting of global supervision was already identified and addressed by PD-Quant, activation outliers by QuaRot, and the negative tail of GELU by APHQ-ViT. Which of these limitations is not already addressed by existing methods? If the existing methods have not fully resolved them, the authors should point out where these methods fall short. Could the authors clarify the specific problem in ViT PTQ that this work aims to solve?
- What is the novelty of this work? The GLF-Q framework combines downstream supervision with local regularization from PD-Quant, offline rotation from QuaRot and SliceGPT, and MLP reconstruction from APHQ-ViT. Could the authors clarify how their work differs from these methods in terms of observation, motivation, analysis, and proposed method? In particular, PD-Quant already identified the overfitting of global supervision and addressed it with local regularization. What new insight does GLF provide beyond moving the supervision point from the logits to the penultimate-layer representation?
- Can the authors provide an analysis of the approximation errors in Hessian-guided block reconstruction? In reconstruction-based methods such as BRECQ [6], QDrop [7], APHQ-ViT, and FIMA-Q, it is widely accepted that Hessian approximations introduce structural and truncation errors. Although the paper presents this as one of its main problems, it does not provide any analysis to support it. For example, how large is the gap between the loss change predicted by the second-order approximation and the actual loss change at low bit-widths?
- How sensitive is GLF to the size of the calibration set? Since the main motivation is overfitting on limited calibration data, an evaluation with different calibration set sizes is needed. It would be helpful to show how the gap between GLF and logit-guided block reconstruction changes as the calibration set becomes smaller.
- Can GLF be extended to dense prediction tasks? Since GLF supervises the representation immediately before the task head, extending it to other tasks with ViT backbones would be a natural direction. For dense prediction tasks such as depth estimation and super-resolution, however, there is no single global representation like the class token. It would be interesting to see how the global supervision target should be defined for such tasks and whether GLF remains effective on them.
[6] Li, Yuhang, et al. "Brecq: Pushing the limit of post-training quantization by block reconstruction." arXiv preprint arXiv:2102.05426 (2021).
[7] Wei, Xiuying, et al. "Qdrop: Randomly dropping quantization for extremely low-bit post-training quantization." arXiv preprint arXiv:2203.05740 (2022).