- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates how a classifier head influences the adversarial robustness of a pretrained vision encoder (kept frozen). They evaluate seven vision encoders with different pretraining objectives, comparing classifiers trained with and without BatchNorm while keeping the encoders frozen. They evaluate adversarial robustness using FGSM-20, PGD, and AutoAttack, and assess corruption robustness on ImageNet-C. They also investigate the effect of logit rescaling on adversarial evaluation.
Critical strengths
The study of the influence of linear probes on the adversarial robustness in the context of transfer learning from a pretrained vision encoder is an important problem.
Critical weaknesses
Weaknesses
1. Problem formulation and motivation
L038–L039: “how much can it change if the encoder stays fixed and only the head changes?” Hua et al. (2024) [1] already showed that the linear head matters. A more precise formulation of the research question is required.
L047: “A comparison assembled across such recipes varies the head alongside the encoder” Hua et al. (2024) [1] already evaluate different linear classifier heads on the same frozen encoder.
L048: “Our experiments measure the size of this confound in our setup; whether it explains disagreements across studies remains open.” Unclear. What does “size” mean here? What do you mean by “disagreements across studies”?
L050: “First, retraining a different head changes the classifier f = head ◦ encoder, even though the representation is fixed.” This is a truism. Why do you need to test it?
L052: “Second, an attack’s estimate can change even when the classifier’s decisions do not” It is not clear what this means, and the following sentence (L053) seems to contradict it.
2. Novelty and contributions
L080: Contribution 1 Predictable from Hua et al. (2024) [1], unless there is a specific aspect you want to study.
L081: Contribution 2 Scaling logits obfuscates gradients [2, 5]. AutoAttack [3] contains APGD with the Difference of Logits Ratio (DLR) loss, which is invariant to logit rescaling. For more on the proper evaluation of adversarial robustness, please read [5].
L084: Contribution 3 This is not a clearly stated contribution with an understandable, falsifiable claim.
3. Related work
L042: “Frozen-probe work reports objective- and architecture-dependent robustness rankings” The cited work is more about adversarial robustness in visual self-supervised representation systems than robustness evaluation in general.
L091: Related work on “Evaluating frozen representations” The content and citations in this paragraph are not coherent.
L098: Related work on “Self-supervision and robustness” This does not seem relevant.
4. Experimental evaluation and setup
L057 The evaluation at 1/255 is absolutely weak compared with the perturbation budgets used for most datasets in the adversarial robustness literature.
L119: “Models and heads” For backbones, please check [6] and [7].
References
[1] Hua et al. (2024). Initialization Matters for Adversarial Transfer Learning. CVPR 2024. https://arxiv.org/abs/2312.05716
[2] Carlini and Wagner (2017). Towards Evaluating the Robustness of Neural Networks. IEEE S&P 2017. https://arxiv.org/abs/1608.04644
[3] Croce and Hein (2020). Reliable Evaluation of Adversarial Robustness with an Ensemble of Diverse Parameter-Free Attacks. ICML 2020. https://arxiv.org/abs/2003.01690
[4] Athalye, Carlini, and Wagner (2018). Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. ICML 2018. https://arxiv.org/abs/1802.00420
[5] Carlini et al. (2019). On Evaluating Adversarial Robustness. arXiv preprint. https://arxiv.org/abs/1902.06705
[6] Heuillet et al. (2025). A Guide to Robust Generalization: The Impact of Architecture, Pre-training, and Optimization Strategy. NeurIPS 2025. https://arxiv.org/abs/2508.14079
[7] Goldblum et al. (2023). Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks. NeurIPS 2023 (Datasets and Benchmarks Track). https://arxiv.org/abs/2310.19909
Questions for authors
cf. weaknesses