acceptodds
Under review as a conference paper at ICLR 2027

UPD: Universal Perturbation Distillation from Individual Adversarial Attacks

Abstract

Deep neural networks remain highly susceptible to perturbation-based attacks, which seek small input modifications that induce model failure. These attacks manifest as either individual or universal adversarial perturbations (IAPs, UAPs), where the former are designed for specific inputs, whereas the latter are input-agnostic. While the simpler setting of IAPs has seen rapid methodological progress, UAP advancements remain comparatively limited, as adapting methods to the universal setting is often nontrivial. In this work, we propose *Universal Perturbation Distillation (UPD)*, a domain-decoupled formulation for learning universal adversarial perturbations from off-the-shelf IAP methods. By treating individual adversarial examples as representation-level supervision, UPD leverages IAP techniques for the universal setting. We instantiate UPD on both LLM jailbreak and image classification settings, achieving and often surpassing state-of-the-art performance, with substantial improvements on robust models.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Adversarial Attacks, Universal Adversarial Perturbations, Large Language Models, Image Classification Models, Automatic Red Teaming, Jailbreak

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_2vdWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The core idea of the paper is to distill a universal perturbation that can be reused across different inputs from multiple successful individual attacks. The method can be summarized in three steps: (1) apply off-the-shelf individual attack methods to each sample and retain only successful results; (2) use the intermediate representations induced by these successful attacks as supervision targets; and (3) keep the model weights fixed while optimizing a single shared perturbation so that, for each input, it induces internal representations close to those produced by the corresponding successful individual attack.

Critical strengths

1.The proposed method is conceptually clear. It uses intermediate representations from successful individual attacks to supervise a universal perturbation, providing an effective way to reuse existing attack methods.

2.The framework has broad applicability and is experimentally evaluated in different application settings, namely image classification and LLM jailbreaking.

Critical weaknesses
  1. I would like to see a numerical comparison of the training-set and test-set ASR achieved by the same final perturbation, to clearly demonstrate the generalization capability of the universal perturbation.

  2. Why does increasing the cosine similarity between the intermediate representations produced by the teacher and student paths improve attack effectiveness? In other words, can cosine similarity capture differences in the key features that influence the network’s predictions? I would appreciate further theoretical explanation, or empirical evidence showing that cosine similarity is a comparatively suitable choice among commonly used metrics for measuring feature differences.

Questions for authors

Please refer to the questions raised in the weaknesses above.

1 comment
AuthorsOpenReview ↗

Rebuttal to Reviewer 2vdW

We thank the reviewer for their thoughtful evaluation and for recognizing the strengths of our work. We address both concerns below and have incorporated the corresponding analyses and clarifications into the revised manuscript. We would be happy to provide further details or experiments.

Q1: Train vs Test Performance

... [please present] numerical comparison of the training-set and test-set ASR achieved by the same final perturbation, to clearly demonstrate the generalization capability of the universal perturbation.

We thank the reviewer for raising this question. We clarify that all results reported in the paper, except RQ1 (which examines optimization dynamics), are evaluated on held-out test sets. We additionally present cross-dataset generalization in Table 2, where UPD also leads.

The table below presents UPD ASR across data splits for LLMs, aggregated across models and datasets. There is no substantive difference between train and test ASRs, demonstrating generalization. Validation ASR is higher since it's used for best checkpoint selection. We have incorporated these results, alongside vision results into the revised manuscript (Appendix E.2).

Model KindJudgeTrainValTest
StandardHB87.490.687.7
RobustHB74.681.876.7
StandardSR68.772.168.3
RobustSR56.963.256.6

Q2: Theoretical Justification for Similarity Function

Why does increasing the cosine similarity between the intermediate representations produced by the teacher and student paths improve attack effectiveness? I would appreciate further theoretical explanation, or empirical evidence showing that cosine similarity is a comparatively suitable among commonly used metrics.

We thank the reviewer for this question. The similarity function aims to steer representations into the failure region. At the penultimate layer, the key insight is that failure depends on representation direction rather than magnitude, and cosine similarity exactly respects this property. On the other hand, L2 penalizes differences in magnitude even when they do not affect predictions. This motivates cosine similarity at the penultimate layer, which UPD uses across both modalities on all models. Our existing ablation (Section 5.5) further shows that cosine similarity outperforms output-level cross-entropy, supporting the proposed objective.

For earlier layers, the failure-region geometry is unknown, but cosine similarity remains suited to the multi-layer objective, as its normalization prevents layers with larger activation norms from dominating the loss.

Formal justification. Let zz be a penultimate representation, followed by a linear head WW with no additive bias term, which yields the prediction arg⁡max⁡j(Wz)j\arg\max_j (Wz)_j. For any α>0\alpha > 0: arg⁡max⁡j(W(αz))j=arg⁡max⁡j(Wz)j\arg\max_j (W(\alpha z))_j = \arg\max_j (Wz)_j, and therefore z∈F(x)  ⟺  αz∈F(x)z \in F(x) \iff \alpha z \in F(x). Membership in the failure region thus depends only on the direction of zz, not on its norm.

For LLMs, the final unembedding matrix has no bias term, so the invariance holds exactly, and token rankings are retained. For vision models, the classification head often includes a bias term, so we evaluate the invariance empirically, and it is retained in 99.19% of cases across nine models and six UAP methods.

We added this discussion to the revised manuscript (Appendix B.3).

Reviewer_zdjuOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper looks to study the generation of Universal Adversarial Perturbations (UAP). Two broad categories of UAP are scenarios where a UAP is directly optimized to satisfy some joint objective, and another where Individual Adversarial Perturbations (IAP) are combined to form a UAP. This paper focuses on the latter. While baseline methods generally consider this combination in pixel space, this work goes a step further and defines the combination in a domain agnostic latent space through the usage of generic operations that may apply to different domains such as the image or language domains.

Critical strengths
  • Smoother optimization surface than standard UAP losses
  • Improved results on difficult to optimize landscapes, such as adversarially trained models (e.g. LLM Circuit Breakers, R2D2, and Latent Adversarial Training models)
  • Generalization to multiple domains (image and language)
Critical weaknesses
  • Need to recompute adversarial perturbations using current UAP seed every iteration for unsatisfied samples (computationally expensive)
  • Language application relies on continuous tokens, not realistically applicable
  • UPD underperforms against baselines in the standard regime of non adversarially trained models, so it is not universally applicable, though those baselines are already saturated in terms of ASR and well studied
  • Sensitivity to selection of what layers to compute cosine similarity loss on
Questions for authors

Could UPD distill from a static, precomputed cache of offline IAP examples generated once prior to training, rather than dynamically querying an IAP teacher during optimization (meaning there would exist a fixed number of examples to train from as opposed to recomputing new ones every iteration)?

2 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer zdju (1/2)

We thank the reviewer for their thoughtful review, for recognizing the strengths of our work, and for the valuable suggestion in Q1. We have incorporated new experiments, additional details, and clarifications into the revised manuscript. If any concerns remain insufficiently addressed, we would be happy to provide further clarification or additional experiments.

W1: Computational Requirements

Need to recompute adversarial perturbations every iteration for unsatisfied samples (computationally expensive)

The reviewer raises an important concern, which we address through empirical evidence and computational analysis. First, we find that even the cheapest single-step IAP attack (FGSM) is sufficient to attain SOTA on all robust vision models (Appendix C.3.4), emerging as a strong default and helping reduce runtime. Second, our new static-corpus experiment (Q1) demonstrates that UPD can operate without repeatedly optimizing the IAP teacher, providing an alternative that avoids this recurring cost.

Runtime: For vision, UPD is faster than UAPGD and DM-UAP, requiring approximately 30 minutes per run on an L40S GPU. For LLMs, UPD is faster than AG-UAP, completing within two hours. Moreover, since UPD produces a UAP, no additional optimization is required during deployment. Runtime comparisons were previously reported in Appendices C.2 and D.2 and have now been added to the main text.

W2: Embedding-Space Attack

Language application relies on continuous tokens, not realistically applicable

Applicability to closed LLM APIs remains a challenge. UPD targets the white-box setting, applicable to open LLMs, which becomes increasingly relevant as highly capable models are released with open weights. The relevance extends beyond this setting. Prior work demonstrates that robustness to continuous embedding attacks transfers to robustness against discrete jailbreaks [1,2]. Moreover, Soft-Prompt [3] jailbreaks an open model via an embedding attack, then uses it as the attacker LLM that generates natural-language jailbreaks. This setting is also shared by other established baselines. We discuss the relevance of this setting length the relevance in Appendix D.1.1.

W3: Performance in the Standard Regime

UPD underperforms against baselines in the standard regime, though those baselines are already saturated in terms of ASR and well studied

We agree with the reviewer that UPD underperforms in the standard regime, but this is limited to vision models, where all compared methods already achieve high ASR (>88%). Importantly, UPD outperforms the baselines on standard LLMs, achieving an average ASR of 76.5%, compared to 70.7% for AG-UAP (also proposed in our work) and 68.1% for Ortho, the strongest prior baseline (Table 2).

W4: Sensitivity to Layer Selection

Sensitivity to selection of what layers to compute cosine similarity loss on

We acknowledge that layer selection affects UPD performance. Based on our ablations, we selected a fixed four-layer configuration (uniform@inp) that performs well on both standard and robust models (Appendix D.5.3 provides further analysis). Importantly, this configuration achieved strong results across all ten evaluated LLMs without further tuning. We additionally provide practical layer-selection guidelines for both modalities in Appendix B.4.

Q1: Distillation from Static Adversarial Corpora

Could UPD distill from a static, precomputed cache of offline IAP examples generated once prior to training?

We thank the reviewer for this suggestion. We evaluate UPD using two static, precomputed IAP corpora [4,5] across four LLMs. The table reports ASR (%) on AdvBench, alongside the original corpora's ASR for comparison.

UPD successfully distills a single universal perturbation from static IAP corpora, in several cases surpassing the ASR of the original attacks. These results are consistent with RQ1 (Section 5.2), where UPD similarly outperforms its IAP teacher. Notably, the corpus attacks operate in discrete token space, whereas UPD optimizes in continuous embedding space, providing greater flexibility that may partly explain these improvements. We have incorporated these results into the revised manuscript (Appendix D.3.10).

ModelCorporaSR-ASR (UPD)SR-ASR (Corpora)HB-ASR (UPD)HB-ASR (Corpora)
Llama2-7B[4]79.460.073.173.8
Llama3-8B[4]90.067.590.077.5
Qwen2.5-1.5B[5]32.540.051.948.8
Phi3-mini[4]46.349.449.453.8
AuthorsOpenReview ↗

References (2/2)

References

  • [1] Xhonneux, Sophie, et al. "Efficient adversarial training in llms with continuous attacks." Advances in Neural Information Processing Systems 37 (2024): 1502-1530.
  • [2] Sheshadri, Abhay, et al. "Latent adversarial training improves robustness to persistent harmful behaviors in llms." arXiv preprint arXiv:2407.15549 (2024).
  • [3] Schwinn, Leo, et al. "Soft prompt threats: Attacking safety alignment and unlearning in open-source llms through the embedding space." Advances in Neural Information Processing Systems 37 (2024): 9086-9116.
  • [4]: Sun, Chung-En, et al. "Iterative self-tuning llms for enhanced jailbreaking capabilities." Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.
  • [5]: Ben-Tov, Matan, Mor Geva, and Mahmood Sharif. "Universal jailbreak suffixes are strong attention hijackers." Transactions of the Association for Computational Linguistics 14 (2026): 1028-1050.
Reviewer_o1AROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Universal Perturbation Distillation (UPD), which generates universal adversarial perturbations by distilling intermediate representations from individual adversarial attacks. Experiments cover image classification and LLM jailbreaking.

Critical strengths

The paper presents a unified framework for constructing universal adversarial perturbations across different tasks.

Critical weaknesses
  1. The idea of using intermediate representations from adversarial examples to guide optimization has already been explored by ILA (Huang et al., ICCV 2019). Directly extending ILA to universal perturbation optimization would achieve comparable results.
  2. The distillation objective relies on the surrogate model, without explaining transferability to unseen models. Table 5 also shows inconsistent improvements, particularly against robust models.
  3. The jailbreak experiments operate in continuous embedding space under white-box access, without mapping perturbations to discrete tokens. The effectiveness in real-world LLM APIs remains unverified.
  4. The paper lacks evaluation under common transformations such as JPEG compression, resizing, and blurring, leaving the practical robustness of the proposed UAPs unclear.
Questions for authors
  1. How does UPD fundamentally differ from ILA (Huang et al., ICCV 2019)?
  2. Why should distilling intermediate representations from a surrogate model improve cross-model transferability?
  3. Can the embedding-space perturbations be converted into text tokens while preserving their effectiveness?
  4. How does UPD perform under common image transformations, particularly JPEG compression?
6 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer o1AR (1/2)

We thank the reviewer for taking the time and effort to review our paper. The new experiments and clarifications have been added to the revised version. If any aspect remains insufficiently addressed, we would be happy to provide further details and experiments.

Q1: Comparison with Universal ILA

We thank the reviewer for suggesting this comparison. To address this question, we compare Universal ILA (UILA) and UPD both conceptually and empirically, implementing UILA for both vision and LLM jailbreaking. UPD outperforms UILA on all 19 evaluated models across both modalities. The table reports mean ASR by category (LLMs: AdvBench, SR-ASR). We added full results to revised Appendix E.1.

ModalityModelsUILAUPD
VisionStandard74.894.7
VisionRobust8.213.3
LLMStandard32.975.4
LLMRobust25.163.9

Conceptually, UPD differs from ILA in three key aspects. First, the transfer objectives differ, as ILA targets cross-model transfer of individual attacks, whereas UPD targets cross-input generalization of a single UAP. Second, the objective functions differ, as ILA maximizes projection along a fixed adversarial direction, while UPD aligns representations with dynamically generated adversarial targets. Finally, the optimization dynamics differ, as UPD warm-starts IAPs from the evolving UAP, enabling their mutual improvement. Further discussion is provided in Appendices A.4-A.6.

Q2: Distillation Objective and Surrogate Model

The distillation objective relies on the surrogate model. Why should distilling intermediate representations from a surrogate model improve cross-model transferability?

We appreciate the opportunity to clarify this distinction. UPD targets cross-input generalization, not cross-model transferability; no separate surrogate model is required. The similarity loss (Eq. 6) serves as a surrogate for the UAP objective (Eq. 1), steering representations toward failure regions (Sections 3.3-3.4).

Regarding cross-model transfer, prior work demonstrates that intermediate-level optimization can improve transfer by reducing dependence on source-specific decision boundaries and exploiting features shared across models [1,2]. UPD extends this approach to the universal setting and may benefit from similar mechanisms.

Table 5 shows inconsistent gains, particularly on robust models.

Regarding Table 5, our intention is to treat the source model as a variable alongside the method. Across all methods, standard-source transfer is substantially weaker (ASR <3.5%), consistent with prior literature. UPD trails in this setting, by ~1%. In the more effective robust-source regime, UPD achieves the highest aggregate ASR (10.31% vs 6.78% for the strongest baseline). We have clarified these findings in the revised manuscript.

Q3: Embedding-Space Attacks and Discretization

We thank the reviewer for raising this point. UPD targets the white-box setting, applicable to open LLMs, which becomes increasingly relevant as highly capable models are released with open weights. The relevance extends beyond this setting. Prior work demonstrates that robustness to continuous attacks transfers to robustness against discrete jailbreaks [3,4]. Moreover, [5] jailbreaks an open model via an embedding attack, then uses it as the attacker LLM that generates natural-language jailbreaks. This setting is also shared by other established baselines. We further discuss this setting in Appendix D.1.1.

Applicability to text-only APIs remains limited, which we added to the revised limitations. In preliminary experiments, discretization of final embeddings to the nearest vocabulary tokens did not preserve attack effectiveness. We consider it an important direction for future research.

Q4: Robustness to Image Transformations

We appreciate this suggestion and have conducted the requested evaluation. We evaluate UPD under post-perturbation transformations with original and transformation-augmented training. The table below reports aggregate ASR over model categories, full results in revised Appendix C.3.7.

Under transformations, robust models exhibit little degradation, while standard models degrade; however, augmented-training substantially restores effectiveness for both. Notably, augmented UPD can achieve higher ASR on robust models than in the original untransformed setting (Table 4), suggesting that augmented training may be particularly effective against robust models. Most of the transformations themselves do not materially affect clean accuracy.

UPD TrainingModelJPEG Q75JPEG Q50Downsample ×0.75Random Crop ×0.75
OriginalStandard49.7832.0944.7251.97
OriginalRobust11.4711.3712.7713.74
AugmentedStandard91.5889.1693.9883.08
AugmentedRobust27.7024.1918.5115.77
AuthorsOpenReview ↗

References (2/2)

References

  • [1] Inkawhich, Nathan, et al. "Feature space perturbations yield more transferable adversarial examples." 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2019.
  • [2] Salzmann, Mathieu. "Learning transferable adversarial perturbations." Advances in Neural Information Processing Systems 34 (2021): 13950-13962.
  • [3] Sheshadri, Abhay, et al. "Latent adversarial training improves robustness to persistent harmful behaviors in llms." arXiv preprint arXiv:2407.15549 (2024).
  • [4] Xhonneux, Sophie, et al. "Efficient adversarial training in llms with continuous attacks." Advances in Neural Information Processing Systems 37 (2024): 1502-1530.
  • [5] Schwinn, Leo, et al. "Soft prompt threats: Attacking safety alignment and unlearning in open-source llms through the embedding space." Advances in Neural Information Processing Systems 37 (2024): 9086-9116.

I thank the authors for their rebuttal and additional experiments. However, I still have several concerns.

Although UPD outperforms UILA, I am not fully convinced by the authors' explanation of their methodological differences. Both methods use intermediate representations of adversarial examples. In my view, UPD mainly extends this idea to universal perturbations, with successful-sample filtering strategy.

UPD only uses successful adversarial examples for representation distillation. I suspect this may encourage the perturbation to overfit the source model's specific failure patterns. The results in Table 8 support this concern. Perturbations optimized on standard models show poor transferability to other models.

The authors emphasize the white-box setting, but most real-world commercial AI services provide only black-box access. For LLM attack, the authors acknowledge that their embedding-space perturbations cannot be effectively converted into discrete tokens.

The additional experiments actually highlight the weakness of the original method. For example, ASR against Swin-V2 drops from 93.4% to 11.26% after JPEG Q75 compression (Table 10&12).

Overall, I appreciate the additional experiments, but I do not think they fully address my concerns about novelty, transferability, and practical applicability. I therefore maintain my original rating of 2 (Weak Rejection).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Comment

We sincerely thank all reviewers for their thoughtful evaluations and constructive feedback. We appreciate their positive assessments of UPD's contributions:

  • Reviewer o1AR "The paper presents a unified framework for constructing universal adversarial perturbations across different tasks."
  • Reviewer zdju "Smoother optimization surface than standard UAP losses"; "Improved results on difficult to optimize landscapes, such as adversarially trained models"; "Generalization to multiple domains (image and language)"; "I have no substantive concerns about the paper’s soundness."
  • Reviewer 2vdW "The proposed method is conceptually clear"; "The framework has broad applicability and is experimentally evaluated in different application settings, namely image classification and LLM jailbreaking."

In response to the reviewers' questions and suggestions, we have revised the manuscript with additional experiments, theoretical analysis, and clarifications. Below, we summarize the main additions and addressed concerns, with detailed responses provided in the individual reviewer discussions.

1. Comparison with Universal ILA (o1AR; Appendix E.1). We implemented Universal ILA (UILA) for both vision and LLM jailbreaking. UPD outperforms UILA on all 19 evaluated models, demonstrating that a direct extension of ILA does not achieve comparable results. We also clarify the conceptual and methodological distinctions.

2. Generalization to Unseen Inputs (2vdW; Appendix E.2). Complementing our held-out test evaluations and cross-dataset generalization results, we added train/validation/test ASR comparisons for the final perturbations across both modalities. The results demonstrate comparable training and test performance, indicating minimal overfitting.

3. Theoretical Justification for Cosine Similarity (2vdW; Appendix B.3). We provide a geometric justification based on latent failure regions, establishing exact scale invariance for bias-free prediction heads and empirical invariance in 99.19% of evaluated vision cases.

4. Distillation from Static Adversarial Corpora (zdju; Appendix D.3.10). We demonstrate that UPD can learn from precomputed IAP corpora without online teacher optimization. Across four LLMs, UPD successfully distills universal perturbations, in several cases outperforming the original attacks.

5. Robustness to Image Transformations (o1AR; Appendix C.3.7). We evaluate UPD under JPEG compression, downsampling, and random cropping. While standard models exhibit degradation, transformation-augmented training largely restores effectiveness. On robust models, augmented training can even improve ASR beyond the original untransformed setting.

6. Computational Efficiency (zdju; Section 5.1, Appendix C.3.4). We highlight that inexpensive single-step FGSM teachers suffice to achieve SOTA on all robust vision models. We also elevate runtime comparisons to the main text, demonstrating that UPD is faster than the evaluated UAPGD and DM-UAP vision baselines and AG-UAP for LLMs.

7. Standard-Model Performance (zdju; Tables 2, 4). We clarify that UPD's weaker performance in the standard regime is limited to vision models, where competing methods already achieve high ASR. UPD outperforms the evaluated baselines on standard LLMs.

8. LLM Embedding-Space Applicability (o1AR, zdju; Appendix D.1.1, Limitations). We clarify the relevance of embedding-space attacks to open-weight LLMs. Furthermore, prior work demonstrates that robustness to continuous attacks can transfer to discrete jailbreaks as well, highlighting the broader relevance. We also acknowledge the limited applicability to text-only APIs and the challenges of discretization.

We believe these additions substantially strengthen the empirical and theoretical foundations of UPD while addressing the reviewers' concerns, and kindly ask that they be taken into account when updating their assessments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.