acceptodds
Withdrawn from ICLR 2027

Silent Loss of Image Conditioning Under Direct Fooling Objectives: A Diagnostic for Amortized Adversarial Attacks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Amortized adversarial attacks initially train a model to generate images that can fool a victim classifier, without requiring access to it at attack time. Previous works mention that models trained with an attack objective may collapse and generate similar perturbations across all images, while maintaining high attack success rate and perceptual quality. Our experiments reveal a novel form of this failure where the perturbation remains diverse unlike the earlier failure mode, yet when the generated perturbations are applied to an image other than the one it was generated from, the attack success rate still remains quite high. We introduce a shuffled-image test that compares a perturbation paired with the source image and a non-source image to measure this failure. We also evaluate a partial solution to this problem by guiding the generator with an attacker during an initial warmup phase. This strategy successfully preserves source-image specificity during the warmup phase, keeping perturbations image-specific. However, this strategy cannot consistently keep specificity high after warmup ends and the attack objective starts. Only eight of twelve full-recipe runs kept specificity high to the end. Finally, we test 57 third-party public amortized attack model checkpoints and find high specificity among them, which points to a recipe-specific finding within that one lineage rather than a general defect of amortization.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
adversarial examples, adversarial attacks, amortized adversarial attacks, generative attack models, attack generators, universal adversarial perturbations, robustness evaluation, evaluation diagnostics, image-conditioned perturbations, teacher-guided training, diffusion-guided attacks, ImageNet

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xetXOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the losing of specificity in generated perturbations of amortized attackers. The authors observe that amortized attackers tend to generate pertubations that are both diverse and have good transferability. They propose a training objective consisting of teacher guidance, attack loss, and other regularization terms, and use it to study the effect of each term on the specificity and energy distribution of perturbations geneated by the learned amortized attacker. The warmup ablation demonstrates the benefits of teacher guidance in preserving specificity, but the full training recipe can still fail to maintain specificity.

Critical strengths

The authors propose a novel training objective and use the objective to study the effect of teacher guidance, attack loss, and other regularization terms on the specificity and energy distribution of generated perturbations. The objective itself is novel and an useful diagnosing tool for studying the behaviors of amortized attackers.

Critical weaknesses
  • The motivation of this work is unclear. In the Introduction, the authors observe that perturbations generated by amortized attackers are diverse and have good transferabilty, but do not explain why such diverse but generic perturbations should be considered as a failure that needs to be addressed.

  • Organizaton is poor and text is hard to follow. The Related Work section includes a paragraph discussing experimental results which is misplaced, and discusses text-base attacks which is irrelavant. Section 6 is difficult to follow, the corresponding tables are not explicitly referenced in the text (e.g., Image specificity discussion is not linked to any experimental results). Table 1 is also never explicitly referenced or discussed in the main text. Section 6 compare the proposed amortized attacker with other attack methods, but it’s unclear how this comparsion is related to the original silent loss problem. Most of the result discussions are enumerating observations without providing a clear conclusion.

Questions for authors

None

Reviewer_4QXMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies amortized adversarial attacks, their failure modes, and how these failures may be mitigated. Amortized attacks train a model to generate adversarial perturbations capable of fooling a victim classifier. Existing work has found that the training of amortized attacks can collapse such that the generator no longer meaningfully uses the source image to determine the perturbation, instead producing similar perturbations across images while maintaining high attack success rates. This paper identifies a new variant of this failure in which the perturbations remain diverse, but can still be applied to images other than those from which they were generated while maintaining a high attack success rate.

To address this failure, the authors utilize a teacher-student training setup in which the student generator is guided by the teacher's attack trajectories and intermediate states, particularly during an initial warmup phase, to encourage source-image-specific perturbations. In addition, the authors propose a diagnostic based on comparing the effectiveness of a perturbation on its source image with its effectiveness on a different image. Finally, the authors evaluate the proposed diagnostic and mitigation strategy on the ImageNet-1k dataset.

Critical strengths
  1. The paper identifies a novel variant of collapse in which perturbations remain diverse but can still be successfully applied to images other than those from which they were generated. This is an interesting observation that deserves further attention from the research community.

  2. The proposed diagnostic score is intuitive and reasonable. It measures how well a trained attack generator preserves source-image specificity.

  3. The experimental results provide evidence that the proposed teacher-guided training strategy can improve source-image specificity in some settings, and the diagnostic appears useful for distinguishing healthy from degraded generators.

Critical weaknesses
  1. The choice of the student architecture seems somewhat arbitrary. The authors use a U-Net-based generator but do not provide sufficient motivation for this architectural choice or explain whether it is consistent with architectures commonly used in closely related amortized attack methods.

  2. The authors do not sufficiently justify the choice of models used in the teacher ensemble. In particular, it is unclear why the adversarially trained ResNet-50 from Engstrom et al. (2019) was selected as one of the teacher's surrogate classifiers, rather than other available robust models.

  3. The objective used to train the student generator contains seven separate loss terms, each with its own weighting hyperparameter. This makes the method relatively complex and may introduce a considerable hyperparameter-tuning burden for users of the proposed approach. It also makes it difficult to understand the individual contribution of each component of the objective.

  4. The motivation for several of the individual loss terms is not sufficiently developed. While the paper briefly describes what each term does, the rationale for why all of these terms are necessary, and why this particular combination is appropriate, a better explanation would be beneficial. Stronger component-wise ablations would also help establish their individual contributions.

  5. The reviewer is curious as to why the authors mainly use non-amortized adversarial attacks such as PGD-40, Square Attack, C&W, and AutoAttack as baselines. Why are closely related amortized attacks not included as baselines?

  6. The experimental evaluation is limited to ImageNet-1k. Since the paper studies a potentially general failure mode of amortized adversarial attacks, experiments on additional standard datasets such as CIFAR-10, CIFAR-100, or SVHN would help establish whether the observed phenomenon and the proposed mitigation generalize across datasets and image resolutions.

Questions for authors
  1. I suggest that the authors address Weaknesses 1, 2, 4, and 5.

  2. How were the loss-weighting hyperparameters tuned? Could the authors provide an ablation study of the individual loss terms? Which terms are the most consequential, and why?

  3. Could the authors provide additional experimental results on CIFAR-10 and SVHN?

Reviewer_1qLxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • The paper studies generator-based adversarial attacks on ImageNet under an untargeted budget. It shows that generators can lose their dependence on the input image in two ways: perturbations can collapse into one shared pattern, as NAT previously observed visually, or remain diverse while still working on many other images.

  • The second case is the main new finding. Standard measures, including ASR, LPIPS/PSNR, the step curve and universal energy, still look normal, so they miss this loss of input dependence.

  • To catch this, the authors propose a shuffled-image test that compares ASR on the original image with ASR when the perturbation is applied to another image. They also test swaps within the same class and set a minimum ASR ratio of 2.0 in advance.

  • As a partial fix, they first train a small U-Net student to imitate per-image AdvAD teacher trajectories, then turn on the attack loss. This preserves input dependence in 8 of the 12 seeds, though it does not solve the problem consistently.

  • They also reimplement GAP with three seeds per arm and check 57 released BIA, LTP and NAT checkpoints. All 57 pass the test, suggesting that the failure is specific to their training recipe rather than a general problem with generator-based attacks.

  • One limitation is that all evaluation uses a single ResNet-50 victim, which is also included in the training ensemble.

Critical strengths
  • The paper points out a failure that standard checks can miss. Earlier work, especially NAT, identified generator collapse by looking at the perturbations. Here, the authors distinguish two forms of collapse. In the less obvious case, the perturbations still differ from one another but work on other images too, while attack success, image quality and universal energy all look normal. The gap_baseline:44 run is a good example: it achieves 0.92 ASR within the quality limit and has low universal energy, yet still fools the victim on most swapped images. The shuffled-image test is cheap and straightforward to add to an evaluation, which I see as a useful contribution.

  • I found the warmup experiment particularly helpful because it isolates the effect of teacher pairing. With the attack loss off, the authors change only whether the teacher is paired with the correct image. Universal energy falls with correct pairing and rises with mismatched pairing. They see the same pattern at a second seed, with a similar effect size. This gives some evidence that per-image teacher guidance affects the direction of training, at least during warmup. It is the main reason I would stop short of a clear rejection.

Critical weaknesses
  • Limited significance. My main concern is that the finding seems confined to the authors’ training recipe. The failure appears in their own models and in their GAP reimplementation trained with their teacher-free loss, while all 57 released checkpoints pass the test. The authors also acknowledge that these checkpoints cover only about three independent setups. More importantly, the paper does not show whether low specificity causes a practical problem, for example in adversarial training or robustness evaluation. If the attack still succeeds, what is lost when the perturbation becomes less dependent on the input? The diagnostic, while useful, is methodologically simple, the mitigation fails in four of twelve runs, and the “basin selector” explanation is not tested beyond the warmup phase. Taken together, these points leave me leaning toward rejection. I would need stronger evidence that the finding matters beyond this particular training recipe.

  • Narrow evaluation and insufficient evidence for the central claim. All experiments use ImageNet and a single ResNet-50 victim that is also part of the training ensemble. There are no results on ViTs, other CNNs, robust models from RobustBench, or a second dataset. The paper connects the failure to direct fooling objectives, but does not test the released Cross-Domain Attack generators (CDA; Naseer et al., NeurIPS 2019), which use that type of objective. I also have concerns about two baseline setups: Square remains below 50% ASR even after 5,000 queries, and DeepFool’s ASR is unchanged when the budget is doubled. These results need checking. Whether the failure comes from the objective, and whether it extends beyond the current setup, are central questions for this paper. The present evaluation does not give me enough evidence to answer either.

Questions for authors

I have four main questions that could change my assessment:

  1. Could the authors run the shuffled-image test on the released Cross-Domain Attack generators (CDA; Naseer et al., NeurIPS 2019)? CDA uses a relativistic cross-entropy loss on the classifier output. Since the paper links the failure to direct fooling objectives, this seems like an important case to test.

  2. Could the authors double-check the Square and DeepFool setups? The reported numbers seem much lower than published results. I would like to see the pixel range and normalization checked, along with an explanation of how the r=2ϵr = 2\epsilon convention is applied.

  3. How much does low specificity matter in practice? For example, would adversarial training with a degraded generator lead to different robustness from training with a healthy one? The test shows that the generator has lost input dependence, but the practical consequence is still unclear to me.

  4. Why is evaluation limited to ResNet-50, which is also part of the training ensemble? It would help to test the existing generators on classifiers they were not trained against. This should be possible without retraining the generators. I would suggest at least one ViT, such as ViT-B/16 or DeiT, another CNN, and an adversarially trained RobustBench ImageNet model other than the Engstrom ResNet-50 already used as a surrogate. I am particularly interested in the robust victim, since a generic perturbation that works across images on a standard model may stop doing so on a robust model. Does the same failure still appear there, or is it limited to undefended classifiers? A separate experiment on CIFAR-10 would also help establish whether the failure and the proposed mitigation extend beyond ImageNet.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.