- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper focuses on how calibrated data affects the damage robustness of low-bit training and quantization of visual Transformer. It links effective calibration transformation with the reduction of activation energy. It proposes Entropy-K, selecting the transformation by using the spatial entropy of image difference energy without model inference or increasing the number of calibration samples. Experiments covering different architectures and PTQ methods on ImageNet. The ablation studies and downstream task evaluations show that Entropy-K can improve damage robustness while typically maintaining similar clean data accuracy.
Critical strengths
Easy for practical application. Entropy-K does not require an increase in the number of calibration samples, acquisition of damaged data from the target domain, or modification of the underlying PTQ algorithm to improve damage robustness. Therefore, it is easy to integrate into existing quantization processes.
Extensive experimental verification. The experiments cover various architectures, PTQ methods, and low-bit settings, demonstrating the applicability of this method beyond a single model or quantization method.
Critical weaknesses
The incremental gains are limited. Entropy-K outperforms clean data calibration, but compared to stronger single transformations such as PRIME, its average damage robustness gains are smaller. These comparisons lack uncertainty estimates from repeated experiments, so it is difficult to determine whether entropy-guided selection provides stable and practically significant advantages.
The method innovation is limited. Existing works such as MetaAug have explored improving PTQ by calibrating data transformations. Entropy-K mainly adds a spatial entropy sorting heuristic for selecting and combining existing transformations. Although the selection indicators without model inference and the analysis for damage robustness have independent contributions, I believe its advancement compared to existing calibration data methods is still relatively incremental.
Questions for authors
i. What evidence separates the value of entropy-based ranking from simply choosing established strong augmentations? Does the advantage remain against a mixture selected using prior robustness knowledge, without entropy, under the same calibration budget?
ii. If the Gaussian-noise candidates in Appendix B were added to the pool, would Entropy-K select them? What rule excludes high-entropy but ineffective candidates without using corruption-test results?
iii. How stable are the modest gains over PRIME across calibration samples and random seeds? Can the authors report paired differences and uncertainty for both clean and corruption accuracy?
iv. Which data and metrics determined K, augmentation strengths, and the candidate pool? Were any of these choices informed by ImageNet-C results? Please distinguish the data used to determine these settings from those used for the final entropy-based ranking.
6 comments
Response to Reviewer FW1p (2)
W2. Novelty and distinction from existing calibration augmentation methods.
Entropy- differs from existing calibration augmentation methods such as MetaAug (Pham et al., 2024) in both what the calibration data is designed for and how the transformations are obtained:
| MetaAug | Entropy- (ours) | |
|---|---|---|
| Goal | Mitigate calibration-set overfitting | Corruption robustness |
| Evaluation | Clean accuracy, CNNs | Clean and corruption accuracy, ViTs |
| Transformations | Learned by bi-level optimization | Selected from existing ones by |
| Training / inference | Per model and bit-width | None (once from input images) |
Hence, the novelty of our work lies not in the ranking rule alone, but in the following, which existing calibration-data methods, including MetaAug, do not examine: (i) we show that low-bit PTQ of ViTs preserves substantially less FP performance under common corruptions than on clean inputs (Fig. 1), and we address this gap on the calibration-data side without modifying the PTQ algorithm or the deployed model; (ii) we identify activation-energy reduction as a shared characteristic of effective calibration transformations and corrupted inputs, and explain it through reduced attention concentration and value alignment (Section 4); (iii) we derive from this analysis as a forward-free proxy, computed once from input images and reusable across architectures and bit-widths.
Response to Reviewer FW1p (3)
Q2. High-entropy Gaussian noise candidates.
As the reviewer pointed out, the three Gaussian-noise variants in Appendix B.4 (–) would rank above PRIME () if added to the pool. However, their high score has a different origin. Augmentation methods transform the image itself (e.g., by mixing images or altering colors), so the resulting change depends on the image content, and measures how this change is distributed. Additive noise is sampled independently of the image, so every patch receives nearly the same noise energy and regardless of the content.
Such perturbations can be identified from the inputs alone by checking whether depends on :
where and denote the mean-centered input and perturbation. Candidates with , a simple threshold, are flagged before ranking (1,024 calibration images).
| Transformation | Decision | ||
|---|---|---|---|
| Gaussian (4/255) | 0.9991 | 0.022 | Flagged |
| Gaussian (16/255) | 0.9986 | 0.046 | Flagged |
| Gaussian (32/255) | 0.9982 | 0.079 | Flagged |
| PRIME | 0.966 | 0.583 | Passed |
| MixUp | 0.952 | 0.687 | Passed |
| AugMix | 0.905 | 0.431 | Passed |
| RandAugment | 0.871 | 0.523 | Passed |
| HybridAugment++ | 0.871 | 0.405 | Passed |
| CutMix | 0.826 | 0.339 | Passed |
| TrivialAugment | 0.787 | 0.504 | Passed |
| RandomErasing | 0.637 | 0.349 | Passed |
| RVT-PWA | 0.469 | 0.120 | Passed |
All three noise variants are flagged and all nine original candidates pass, so the selection is unchanged. RVT-PWA, the candidate closest to the threshold (), also yields negligible gains over clean calibration in Table 1 ( and points under W3A3 and W2A3).
In the revised manuscript, we discussed this issue in Appendix B.5 (Table 6).
Response to Reviewer FW1p (1)
We thank the reviewer for the constructive and detailed feedback. We address each point below.
W1 & Q1. Comparison with PRIME and robustness-oriented mixture.
Comparison with PRIME. Averaged over the seven architectures in Table 23 (Appendix K), Entropy-K outperforms PRIME in both clean and corruption accuracy under all three bit-widths, whereas PRIME lowers clean accuracy below the clean-calibration baseline.
While the corruption gains over PRIME are modest, Entropy-K improves corruption robustness without sacrificing clean accuracy. The stability of these gains across calibration seeds is reported in our response to Q3.
Comparison with a prior-knowledge mixture. To separate the benefit of entropy-based ranking from simply choosing established strong augmentations, we evaluated Robust-3, a mixture of three established robustness-oriented augmentations (PRIME, AugMix, and HybridAugment++) selected without Sent, under the same calibration budget. Notably, these are also the three best single transformations under W3A3 in Table 1, so Robust-3 effectively reflects prior knowledge of ImageNet-C performance.
Entropy-3 achieves higher average accuracy than Robust-3 under both W3A3 and W2A3 (+0.46 and +0.18 points). Robust-3 and Entropy-3 share PRIME and AugMix and differ only in the third transformation: Sent selects MixUp, which is not typically regarded as a robustness-oriented augmentation, instead of HybridAugment++. This suggests that entropy-based ranking identifies effective calibration transformations beyond what prior robustness knowledge suggests, without using any corruption-test results.
In the revised manuscript, we added the comparison with Robust-3 in Section 6 (Table 2(a)) and Appendix G.3 (Table 16).