- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes to study “systemic adversarial robustness” as models increasingly use knolwegege distilation and similar training techniques / datasets. The authors propose a novel metric, SAR, to measure the minimum number of distinct attacks that together compromise a pool of models. The authors develop a top-down greedy algorithm to estimate SAR and conduct experiments using different teacher models / attack strategies and demonstrate that knowledge distillation causes lower SAR of the student models.
Critical strengths
This paper studies a novel problem: the attack coverage efficiency of a pool of student models distilled from a common teacher model. The authors demonstrate that students are more prone to be compromised by a handful of attacks than their base models after distillation. The SAR metric quantifies this behavior.
Critical weaknesses
-
The motivation of the paper is measuring shared vulnerability as models increasingly use similar training techniques and datasets. However, attack and vulnerability don't have a one-to-one correspondence - a single attack may exploit multiple vulnerabilities, and multiple attacks may exploit the same vulnerability. In these cases, the number of distinct attacks does not represent the number of distinct vulnerabilities, and the proposed metric SAR becomes a poor metric for capturing shared vulnerabilities.
-
Existing work already shows distilled student inherits teacher-specific adversarial vulnerabilities [a]. So the finding that a pool of students distilled from a single teacher having low SAR is somewhat expected.
[a] Ojha, Utkarsh, et al. "What knowledge gets distilled in knowledge distillation?." Advances in Neural Information Processing Systems 36 (2023): 11037-11048.
Questions for authors
None
3 comments
Author Rebuttal (Part 2)
Related Work
We value the reviewer's nuanced points about existing related work that recent work has demonstrated adversarial vulnerabilities may be inherited via student-teacher distillation. We begin by first highlighting that Ojha et al. and another paper we found in our rebuttal literature review (Angell et al.) study whether, and why, vulnerabilities or jailbreaks transfer between models under distillation. In essence, while these papers are related to SAR, both papers evaluate transfer pairwise or for single inputs, whereas we evaluate a pool of models simultaneously. Furthermore, the results of these related papers show that students can inherit vulnerabilities from a teacher, but they do not ask whether students become vulnerable to the same inputs as each other, or how many distinct inputs are needed to compromise a target fraction of the pool. We argue that these differences in these questions are meaningfully distinct and lead to dramatically different analysis, quantifications, and algorithmic approaches. Our new controls also show that the outcome is not a foregone conclusion: on MNIST, ground-truth labels remove almost all of the SAR decline (Table 4). In particular, our definition of SAR is not found in either of these prior works. However, we understand the related nature of these papers, and we have now included them in our updated references with a dedicated discussion to highlight the similarities and differences.
Presentation
We also took the reviewer's rating of our presentation seriously. In the revision, all LLM experiments follow one unified protocol, the edge cases of SAR are stated explicitly, every comparison is reported with 95% confidence intervals, and the hyperparameters of every figure are listed in Appendix A.5.
Concluding remarks
We thank the reviewer again for their time, review, and pointing us to related work that was originally missed in our literature review. We believe our new clarifications on motivation and contextualization with related work fully address the concerns of the reviewer. We very politely ask the reviewer if the reviewer would consider a score upgrade, or ask further questions that may remain unclear.
References
Rico Angell, Jannik Brinkmann, and He He. 2026. Jailbreak Transferability Emerges from Shared Representations. In International Conference on Learning Representations (ICLR).
Xiaogeng Liu, Peiran Li, G. Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, Patrick McDaniel, Huan Sun, Bo Li, and Chaowei Xiao. 2025. AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs. In International Conference on Learning Representations (ICLR).
Utkarsh Ojha, Yuheng Li, Anirudh Sundara Rajan, Yingyu Liang, and Yong Jae Lee. 2023. What Knowledge Gets Distilled in Knowledge Distillation?. In Advances in Neural Information Processing Systems, volume 36, pages 11037–11048.
Jingyu Zhang, Ahmed Elgohary, Xiawei Wang, A. S. M. Iftekhar, Ahmed Magooda, Benjamin Van Durme, Daniel Khashabi, and Kyle Jackson. 2025. Jailbreak Distillation: Renewable Safety Benchmarking. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 25066–25089. Association for Computational Linguistics.
Dear Reviewer,
We are grateful for the time and effort that went into the original assessment of our submission. As the deadline approaches, we are following up on our response to ensure that all open questions and concerns have been addressed. We hope that our clarification of how we use the term "vulnerability" makes the manuscript easier to follow. We also recognize that our original positioning against related works such as Ojha et al. and Angell et al. was not exhaustive, and we have amended the manuscript to state the novelty of our work more clearly.
We thank the reviewer again for their consideration. We are more than happy to provide any further clarifications up until the deadline, and we hope our response has added clarity in the meantime.
Author Rebuttal (Part 1)
Response to Reviewer espH
We value the reviewer's time and dedication to the review process. We first thank the reviewer for acknowledging the novelty of the problem that we study, and for remarking that distilled student models are more “prone to be compromised by a handful of attacks”---a behavior our SAR metric captures. We now turn to the main questions raised by the reviewer, while highlighting that the global response contains a summary of all the main experimental and exposition updates to the manuscript.
Definition of Vulnerability
The reviewer’s central conceptual objection is that distinct attack inputs need not correspond to distinct underlying vulnerability mechanisms. We agree that vectors and mechanisms are distinct, but politely push back that it invalidates SAR as a measure of shared exposure. Specifically, in this paper we use vulnerability in an operational, input-level sense: a model’s susceptibility to a particular adversarial input. SAR quantifies how many distinct inputs suffice to compromise a specified fraction of the model pool. A single input that compromises multiple models represents shared exposure to that input, regardless of whether it exploits the same underlying mechanism in each model. We believe that this interpretation of SAR was insufficiently explicit in our original submission, and we thank the reviewer’s comment that enabled us to robustly clarify this in the revision.
Intuitive definition of vulnerability
In particular, when we use the term vulnerability we, by definition, mean susceptibility to a particular attack vector. This is now explicitly highlighted in Section 3.1 of the revised manuscript. This stands in contrast to the reviewer's initial assumption about our intention: that vulnerability is defined by the mechanism/concept/strategy used to exploit a model. We acknowledge that computing SAR with respect to mechanism/concept/strategy is an interesting future study, but remains orthogonal to what this work establishes. The notion of SAR that we use, where each vector represents a unique vulnerability, is an important modelling decision to justify, and one that we did not appropriately distinguish in our original submission, so we thank the reviewer for helping us make our work clearer. The reviewer's response implies (and we are happy to provide further corrections if this was not their intent) that one should define vulnerabilities to be in mechanism/concept/strategy space, e.g., vulnerable to hypothetical questions or to translation attacks. When viewed in this light, our assumption still operationally captures shared vulnerabilities the reviewer is concerned about: even when one perturbation exploits different mechanisms across models, its ability to compromise many models represents shared exposure to that input, which is what we want to capture.
While a definition of vulnerability rooted in mechanism/concept/strategy is sensible, it introduces substantial complications that limit its ability to be instantiated and used practically. Computationally, what belongs to one mechanism/concept or another is unclear: are modifications that ask a hypothetical question exploiting the same vulnerability as those that ask the model to only respond with hypotheticals to a non-hypothetical query? It is not obvious. Generally, while existing LLM-driven frameworks like AutoDAN-Turbo (Liu et. al., 2025) automatically create and store complex attack strategies, these methods still must somehow be augmented with a clear understanding of if and when two different vectors instantiating complex attack strategies exploit the same vulnerability. Addressing this difficulty is an important challenge to understanding LLM safety on a more general level, but it is one that is orthogonal to this work. We have added a discussion to this effect in a limitations paragraph in the main text of our revision. Further, we hope that clarifying the conceptual position of this work allows the reviewer to look beyond the assumptions of their initial review and re-evaluate the merits of our arguments, increasing their score if they feel it is warranted. We hope that by making our position clear, the reviewer will re-evaluate our contribution. We are happy to provide further clarifications if they have any additional concerns.