acceptodds
Under review as a conference paper at ICLR 2027

Exposing Vulnerability Monoculture Through Systemic Adversarial Robustness

Abstract

Beneath the apparent diversity of foundation models lies an increasingly homogeneous technical stack of common training pipelines, distillation methods, and data. These shared algorithmic choices can expose an otherwise distinct model ecosystem to concentrated risks leading to a vulnerability monoculture. Yet current practice in assessing adversarial robustness is restricted to individual robustness scores and attack transfer rates, leaving open the unexplored fundamental question: how many attacks suffice for catastrophic widespread compromise? In this paper, we investigate the empirical nature of shared vulnerabilities within a model pool and introduce Systemic Adversarial Robustness (SAR), a novel measure of shared vulnerability of a model pool through the attack diversity needed to exploit it. Specifically, SAR interpolates between single-attack transferability and independent per-model robustness: small values reveal a vulnerability monoculture, whereas large values indicate that failures are dispersed across models. We theoretically formulate SAR as a partial set-cover problem with provable approximation guarantees—enabling the development of scalable algorithms to empirically measure the systemic adversarial robustness. We empirically test the shared vulnerability of vision and large language models and characterize their shared collective vulnerability curve. Specifically, we demonstrate that model pools distilled from DeepSeek-R1 and Claude Opus 4.5 exhibit lower estimated systemic robustness than their corresponding parent pools, revealing a potential systemic cost of frontier-model distillation. Our framework seeks to establish vulnerability monoculture as a measurable target for a more wholistic adversarial evaluation of model populations.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLMs, safety, distillation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_espHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes to study “systemic adversarial robustness” as models increasingly use knolwegege distilation and similar training techniques / datasets. The authors propose a novel metric, SAR, to measure the minimum number of distinct attacks that together compromise a pool of models. The authors develop a top-down greedy algorithm to estimate SAR and conduct experiments using different teacher models / attack strategies and demonstrate that knowledge distillation causes lower SAR of the student models.

Critical strengths

This paper studies a novel problem: the attack coverage efficiency of a pool of student models distilled from a common teacher model. The authors demonstrate that students are more prone to be compromised by a handful of attacks than their base models after distillation. The SAR metric quantifies this behavior.

Critical weaknesses
  • The motivation of the paper is measuring shared vulnerability as models increasingly use similar training techniques and datasets. However, attack and vulnerability don't have a one-to-one correspondence - a single attack may exploit multiple vulnerabilities, and multiple attacks may exploit the same vulnerability. In these cases, the number of distinct attacks does not represent the number of distinct vulnerabilities, and the proposed metric SAR becomes a poor metric for capturing shared vulnerabilities.

  • Existing work already shows distilled student inherits teacher-specific adversarial vulnerabilities [a]. So the finding that a pool of students distilled from a single teacher having low SAR is somewhat expected.

[a] Ojha, Utkarsh, et al. "What knowledge gets distilled in knowledge distillation?." Advances in Neural Information Processing Systems 36 (2023): 11037-11048.

Questions for authors

None

3 comments
AuthorsOpenReview ↗

Author Rebuttal (Part 1)

Response to Reviewer espH

We value the reviewer's time and dedication to the review process. We first thank the reviewer for acknowledging the novelty of the problem that we study, and for remarking that distilled student models are more “prone to be compromised by a handful of attacks”---a behavior our SAR metric captures. We now turn to the main questions raised by the reviewer, while highlighting that the global response contains a summary of all the main experimental and exposition updates to the manuscript.

Definition of Vulnerability

The reviewer’s central conceptual objection is that distinct attack inputs need not correspond to distinct underlying vulnerability mechanisms. We agree that vectors and mechanisms are distinct, but politely push back that it invalidates SAR as a measure of shared exposure. Specifically, in this paper we use vulnerability in an operational, input-level sense: a model’s susceptibility to a particular adversarial input. SAR quantifies how many distinct inputs suffice to compromise a specified fraction of the model pool. A single input that compromises multiple models represents shared exposure to that input, regardless of whether it exploits the same underlying mechanism in each model. We believe that this interpretation of SAR was insufficiently explicit in our original submission, and we thank the reviewer’s comment that enabled us to robustly clarify this in the revision.

Intuitive definition of vulnerability

In particular, when we use the term vulnerability we, by definition, mean susceptibility to a particular attack vector. This is now explicitly highlighted in Section 3.1 of the revised manuscript. This stands in contrast to the reviewer's initial assumption about our intention: that vulnerability is defined by the mechanism/concept/strategy used to exploit a model. We acknowledge that computing SAR with respect to mechanism/concept/strategy is an interesting future study, but remains orthogonal to what this work establishes. The notion of SAR that we use, where each vector represents a unique vulnerability, is an important modelling decision to justify, and one that we did not appropriately distinguish in our original submission, so we thank the reviewer for helping us make our work clearer. The reviewer's response implies (and we are happy to provide further corrections if this was not their intent) that one should define vulnerabilities to be in mechanism/concept/strategy space, e.g., vulnerable to hypothetical questions or to translation attacks. When viewed in this light, our assumption still operationally captures shared vulnerabilities the reviewer is concerned about: even when one perturbation exploits different mechanisms across models, its ability to compromise many models represents shared exposure to that input, which is what we want to capture.

While a definition of vulnerability rooted in mechanism/concept/strategy is sensible, it introduces substantial complications that limit its ability to be instantiated and used practically. Computationally, what belongs to one mechanism/concept or another is unclear: are modifications that ask a hypothetical question exploiting the same vulnerability as those that ask the model to only respond with hypotheticals to a non-hypothetical query? It is not obvious. Generally, while existing LLM-driven frameworks like AutoDAN-Turbo (Liu et. al., 2025) automatically create and store complex attack strategies, these methods still must somehow be augmented with a clear understanding of if and when two different vectors instantiating complex attack strategies exploit the same vulnerability. Addressing this difficulty is an important challenge to understanding LLM safety on a more general level, but it is one that is orthogonal to this work. We have added a discussion to this effect in a limitations paragraph in the main text of our revision. Further, we hope that clarifying the conceptual position of this work allows the reviewer to look beyond the assumptions of their initial review and re-evaluate the merits of our arguments, increasing their score if they feel it is warranted. We hope that by making our position clear, the reviewer will re-evaluate our contribution. We are happy to provide further clarifications if they have any additional concerns.

AuthorsOpenReview ↗

Author Rebuttal (Part 2)

Related Work

We value the reviewer's nuanced points about existing related work that recent work has demonstrated adversarial vulnerabilities may be inherited via student-teacher distillation. We begin by first highlighting that Ojha et al. and another paper we found in our rebuttal literature review (Angell et al.) study whether, and why, vulnerabilities or jailbreaks transfer between models under distillation. In essence, while these papers are related to SAR, both papers evaluate transfer pairwise or for single inputs, whereas we evaluate a pool of models simultaneously. Furthermore, the results of these related papers show that students can inherit vulnerabilities from a teacher, but they do not ask whether students become vulnerable to the same inputs as each other, or how many distinct inputs are needed to compromise a target fraction of the pool. We argue that these differences in these questions are meaningfully distinct and lead to dramatically different analysis, quantifications, and algorithmic approaches. Our new controls also show that the outcome is not a foregone conclusion: on MNIST, ground-truth labels remove almost all of the SAR decline (Table 4). In particular, our definition of SAR is not found in either of these prior works. However, we understand the related nature of these papers, and we have now included them in our updated references with a dedicated discussion to highlight the similarities and differences.

Presentation

We also took the reviewer's rating of our presentation seriously. In the revision, all LLM experiments follow one unified protocol, the edge cases of SAR are stated explicitly, every comparison is reported with 95% confidence intervals, and the hyperparameters of every figure are listed in Appendix A.5.

Concluding remarks

We thank the reviewer again for their time, review, and pointing us to related work that was originally missed in our literature review. We believe our new clarifications on motivation and contextualization with related work fully address the concerns of the reviewer. We very politely ask the reviewer if the reviewer would consider a score upgrade, or ask further questions that may remain unclear.

References

Rico Angell, Jannik Brinkmann, and He He. 2026. Jailbreak Transferability Emerges from Shared Representations. In International Conference on Learning Representations (ICLR).

Xiaogeng Liu, Peiran Li, G. Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, Patrick McDaniel, Huan Sun, Bo Li, and Chaowei Xiao. 2025. AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs. In International Conference on Learning Representations (ICLR).

Utkarsh Ojha, Yuheng Li, Anirudh Sundara Rajan, Yingyu Liang, and Yong Jae Lee. 2023. What Knowledge Gets Distilled in Knowledge Distillation?. In Advances in Neural Information Processing Systems, volume 36, pages 11037–11048.

Jingyu Zhang, Ahmed Elgohary, Xiawei Wang, A. S. M. Iftekhar, Ahmed Magooda, Benjamin Van Durme, Daniel Khashabi, and Kyle Jackson. 2025. Jailbreak Distillation: Renewable Safety Benchmarking. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 25066–25089. Association for Computational Linguistics.

Dear Reviewer,

We are grateful for the time and effort that went into the original assessment of our submission. As the deadline approaches, we are following up on our response to ensure that all open questions and concerns have been addressed. We hope that our clarification of how we use the term "vulnerability" makes the manuscript easier to follow. We also recognize that our original positioning against related works such as Ojha et al. and Angell et al. was not exhaustive, and we have amended the manuscript to state the novelty of our work more clearly.

We thank the reviewer again for their consideration. We are more than happy to provide any further clarifications up until the deadline, and we hope our response has added clarity in the meantime.

Reviewer_khoxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Systemic Adversarial Robustness (SAR), the minimum number of distinct adversarial inputs needed to compromise a specified fraction of a model pool, normalized by pool size. It formulates estimation as partial set cover and proposes a greedy procedure using joint attacks. Experiments on MNIST classifiers and LLM pools report lower estimated SAR following shared-teacher training or post-training, which the authors interpret as evidence of vulnerability monoculture.

Critical strengths
  1. The paper asks a useful question about model ecosystems. Measuring how many distinct inputs are needed to compromise a model pool captures attack reuse and complementarity that per-model success rates or single-input coverage do not directly summarize.
  2. The paper develops joint versions of several attacks for heterogeneous model pools and explores both vision and language settings. The reported divergence between individual attack success and pool-level attack counts provides an interesting motivation for further study.
Critical weaknesses
  1. The reported mean SAR combines failure to reach the coverage target with the number of inputs required when the target is reached. In Table 1, the largest reduction occurs at the threshold requiring only one compromised model, which cannot establish cross-model vulnerability sharing. The paper should separately report the fraction of behaviors reaching each target, the conditional attack count, and cross-model success overlap.
  2. GCG and AutoDAN assign one hypothetical attack to every model the search fails to compromise. This does not establish a verified attack set and differs from Definition 1’s treatment of an unattainable target. Models failing on unmodified requests are also counted at zero cost. The contribution of both conventions should be reported, and their relationship to the formal metric and approximation guarantee clarified.
  3. The MNIST distillation experiment changes the training data and teacher supervision together. All students receive the same 240K EMNIST images, so their convergence could result from shared fine-tuning data rather than a shared teacher. Training on the same images with ground-truth labels or different teachers would separate these effects. Comparing the relative changes in SAR and individual robustness does not resolve this confound.
  4. The DeepSeek-R1 comparison pairs base models with instruction- or reasoning-tuned descendants. It therefore combines teacher distillation with changes in post-training and refusal behavior. Comparisons with matched instruction-tuned parents or students trained under the same procedure with different teachers are needed. The individual ASR comparison should also use matched requests, since it is conditioned on each model’s clean refusals.
  5. The Opus AutoDAN comparison uses different behavior sets selected separately for attack success, with 25 behaviors for parents and 22 for descendants. The resulting difference cannot be attributed solely to the model pools. The community checkpoints’ teacher attribution is also unverified, limiting conclusions about a shared Opus teacher even if the observed difference survives matched evaluation.
  6. The LLM comparisons use small behavior sets without paired uncertainty estimates. This makes it difficult to assess whether the reported attack-count differences are stable across behaviors and attack runs. Per-behavior results and confidence intervals are particularly important because a few changes in whether the coverage target is reached can substantially alter mean SAR.
  7. The positioning against the closest prior work is incomplete. Angell et al. [1] and Ojha et al. [2] already investigate vulnerability inheritance through distillation. JBDistill’s BestPerGoal algorithm [3] also greedily selects prompts that compromise previously uncovered models for the same harmful goal. The paper should clarify what its pool-level metric and estimator add beyond these findings and coverage procedures.

Minor:

  1. The cross-attack comparison changes the attack format as well as the search method. GCG and BEAST optimize suffixes, whereas AutoDAN rewrites requests. Specify the admissible inputs and budgets, and describe the ten-model pool used for this comparison.
  2. Reconcile the BEAST behavior source, described as HarmBench in the main text and AdvBench in Table 3. Clarify whether Table 1 averages over 24 or 25 behaviors and whether additional averaging explains the reported values.
  3. Explain the relationship between the eight search calls in Table 1 and the maximum of four in Table 3. The AutoDAN sample counts of 15 in Table 5 and 25/22 in Figure 5 also need reconciliation.
  4. For joint AutoDAN, the same HarmBench classifier guides search and determines success. An independent judge or human audit would help assess evaluator-specific optimization. Specify whether reasoning traces are included in the judged response.
  5. In Figure 2, higher diversity also introduces different architectures. Report per-model ASR to distinguish increased individual robustness from reduced vulnerability overlap. Broader vision claims should also be qualified given the restriction to MNIST.
  6. Explicitly derive or cite the maximum-coverage guarantee stated in the contributions. Clarify how the returned BEAST beams and AutoDAN archives are counted as distinct inputs, and define GCG-RR and AutoDAN-RR.

References:

[1] Angell, Rico, Jannik Brinkmann, and He He. Jailbreak transferability emerges from shared representations. In ICLR, 2026.

[2] Ojha, Utkarsh, Yuheng Li, Anirudh Sundara Rajan, Yingyu Liang, and Yong Jae Lee. What knowledge gets distilled in knowledge distillation?. In NeurIPS, 2023.

[3] Zhang, Jingyu, Ahmed Elgohary, Xiawei Wang, A. S. M. Iftekhar, Ahmed Magooda, Benjamin Van Durme, Daniel Khashabi, and Kyle Jackson. Jailbreak distillation: Renewable safety benchmarking. In Findings of EMNLP, 2025.

Questions for authors
  1. For each coverage threshold, what fraction of behaviors reaches the target, and how many inputs are required conditional on reaching it?
  2. How many hypothetical singleton attacks contribute to each pool’s results? What changes when only verified successful inputs are counted?
  3. Does the MNIST effect remain when the same images are labeled with ground truth or by different teachers?
  4. How do the DeepSeek-R1 results compare with matched instruction-tuned parents? Is individual ASR evaluated on common clean-refused requests?
  5. Does the Opus AutoDAN difference remain on a matched behavior set under the same search budget?
  6. Can the authors provide per-behavior results and uncertainty estimates, using paired analysis where behaviors are matched?
3 comments
AuthorsOpenReview ↗

Author Rebuttal (Part 1)

We thank the reviewer for their careful review and constructive feedback. In particular, we highlight that all LLM results below use the unified protocol of the global response (100 matched HarmBench behaviors, AmpleGCG + BEAST, our own fine-tuning). We report the medium and large pools here and defer the full results to Tables 1, 7 and 8 of the revised manuscript.

W1 - Disaggregating sources of failure in SAR

Firstly, in Table 1 of the original submission, the δ=0.875\delta = 0.875 is given as a diagnostic value, not as evidence of shared vulnerability. Secondly, we now report unconditional SAR alongside conditional verified SAR and the number of behaviors whose coverage targets are reached using verified inputs. We additionally report the share of the unconditional estimated cover contributed by hypothetical singleton inputs. These quantities are reported in Tables 1 and 8 and reproduced here for convenience:

PoolStageClean (%)Attack (%)None (%)Unconditional SAR (approx.)Conditional verified SAR (n)Hyp. share of unconditional cover (%)
Medium (N=6)Base29.850.719.50.5430.39 ± 0.04 (32)36
Medium (N=6)R18.767.224.20.5120.28 ± 0.03 (29)47
Medium (N=6)R1 → Opus 4.647.748.53.80.2880.25 ± 0.02 (85)13
Large (N=8)Base9.442.648.00.5550.25 ± 0.05 (16)65
Large (N=8)R117.954.227.90.3960.20 ± 0.03 (44)46
Large (N=8)R1 → Opus 4.624.451.024.60.3690.20 ± 0.02 (51)41

The reported ± values are half-widths of 95% confidence intervals. Clean, Attack, and None are percentages of model–behavior pairs.

Within this table, we highlight:

  • Clean (%): the share of model–behavior pairs producing harmful responses to the unmodified request increases at nearly every fine-tuning stage representing a general trend of loss of alignment contributing to SAR.

  • None (%): the share of model–behavior pairs not compromised by any tested input show a downward trend save for an increase in the medium pool after R1 fine-tune.

  • Conditional verified SAR: the conditional mean decreases on both the medium and large pools fortifying our claim, albeit with fewer samples it is not statistically significant.

  • Hypothetical share of the unconditional cover (%): the share attributed to hypothetical singleton inputs shows a decreasing trend save for the R1 finetune of the medium model.

These results show that the unconditional SAR decline is accompanied by increased verified target attainment and a lower hypothetical share of the estimated cover between the initial and final pools; this further supports our paper's findings. The revised manuscript reports coverage targets from 50% to 100% (Table 6) and adds per-behavior results (Figures 9 and 10).

W2 - Assumptions about models without successful attacks

This question concerns the two edge cases of SAR evaluation that we clarify in the global response: clean vulnerabilities and robust models, which our definition handles with a consistent semantics. Clean vulnerabilities are not counted at zero cost: the unmodified request is an admissible input, and this null vector counts towards the cover's cardinality. For robust models, where the search cannot find a compromising vector, we count one individual vector for that model. The reviewer is correct that a cover completed in this way is not a verified attack set and that we did not report how much robust models contribute to it. The Table under W1 now does (“Hyp. share” and “None”), and the conditional SAR column shows what changes when only verified inputs are counted.

Regarding the approximation guarantee, it is stated under an explicit attack-oracle assumption; where the search fails, that assumption is not met, so for those behaviors k̂ is an estimate under the stated convention, as emphasized in the revised manuscript.

W3 - Separating shared teacher and shared dataset influences on MNIST

In MNIST, we now fine-tune the same pool on EMNIST images but vary the diversity of the teacher and the diversity of the training dataset:

ConditionSAR₀.₈₅ change (%)1 − ASR change (%)
(A) One teacher, all data−22.3 ± 2.5−7.1 ± 2.7
(B) One teacher, disjoint 12k subsets−23.0 ± 2.4−8.0 ± 2.7
(C) 20 different teachers−17.6 ± 2.3−8.3 ± 2.3
(D) Ground-truth labels−3.1 ± 2.7+6.5 ± 2.3

Change after 6.4k steps; ± is the half-width of a paired bootstrap 95% CI over images. [Table 4, Figure 3]

Disjoint data has a negligible effect (B vs A), 20 different teachers give a less steep decline (C), and ground-truth labels remove almost all of it (D).

AuthorsOpenReview ↗

Author Rebuttal (Part 2)

W4 - Unifying the post-training procedure used

Regarding the post-training method used to derive each student, we highlight that under the revised experimental protocol, each stage is now a single consistent fine-tuning protocol that we control, so stages differ in the teacher traces used. The exception to this, as the reviewer highlights, is the official DeepSeek R1 fine-tuned model. The control the reviewer requests, training students under the same procedure with different teachers, is reported for MNIST in Appendix A.4.1 (Table 4) and discussed under W3.

W5 - Unifying behaviour sets

Both issues are resolved: every pool and stage uses the same 100 behaviors, attack, and budget, and we fine-tune on Claude Opus 4.6 traces ourselves, so every descendant shares a known training source. Under this matched evaluation (table under W6) SAR declines by 43.6% in the medium pool after Opus 4.6 distillation; in the large pool, the decline arrives at the R1 stage (−24.8%), with a further non-significant −6.1% after Opus 4.6.

W6 - Introduction of Confidence Intervals

As every stage is evaluated on the same 100 behaviors, we now report paired bootstrap 95% CIs over behaviors and the number of behaviors whose SAR decreases, ties, or increases (Table 7; δ = 0 shown):

PoolTransitionΔ SAR [95% CI]Rel. changeDown / tie / up
Medium (N=6)Base → R1−0.032 [−0.073, +0.010]−5.8%44 / 31 / 25
MediumR1 → Opus 4.6−0.223 [−0.265, −0.182]−43.6%70 / 25 / 5
MediumBase → Opus 4.6−0.255 [−0.287, −0.223]−46.9%84 / 15 / 1
Large (N=8)Base → R1−0.168 [−0.209, −0.128]−24.8%66 / 24 / 10
LargeR1 → Opus 4.6−0.031 [−0.069, +0.006]−6.1%44 / 25 / 31
LargeBase → Opus 4.6−0.199 [−0.241, −0.158]−29.4%72 / 16 / 12

The cumulative declines are broad.

W7 - Additional prior works

We now discuss these references in the related work. Angell et al. and Ojha et al. study whether, and why, vulnerabilities transfer under distillation, but evaluate transfer pairwise or for single inputs. They show that students can inherit vulnerabilities from a teacher; they do not ask whether students become vulnerable to the same inputs as each other, or how many distinct inputs are needed to compromise a target fraction of a pool. JBDistill's BestPerGoal closely resembles the greedy step of our estimator, but uses coverage as a criterion for selecting benchmark prompts. Our contribution is to treat the minimum cover size itself as a measure of pool-level robustness thus distinguishing it. Further discussion in Section 5.

Minor

M1–M3. These discrepancies arose from mixing attacks, behavior sets, and budgets. Under the unified protocol, they no longer apply to the main results, and the hyperparameters of every experiment are stated in Appendix A.5.

M4. All models run with thinking off, so the judge only ever receives the final output. We recompute SAR_{0.85} under six additional judges which do not change our high-level conclusions (Table 9).

M5. We now report per-model ASR (Table 5). We observe that the high-diversity pool is also individually more robust: at ε = 0.05 on 1,000 random images, ASR is 0.56–0.60 for the low-diversity MLPs and 0.02–0.08 for the high-diversity CNNs. However, at ε = 0.15 every model is broken on every SAR image, yet the high-diversity pool still needs more inputs.

M6. The guarantee follows from the standard greedy maximum-coverage analysis with an approximate oracle (Goundan and Schulz, 2007), now stated in Appendix A.1. Each BEAST search keeps its top 3 beams as distinct inputs, and only inputs selected in the cover are counted. RR denotes round-robin over the pool.

Questions

Q2 - "How many hypothetical singleton attacks contribute? What changes when only verified inputs are counted?"

Between 0.23 and 2.89 per behavior, i.e., 13% to 65% of k̂ (Table 8).

Q4 - "How do the DeepSeek-R1 results compare with matched instruction-tuned parents?"

Parents and descendants are now matched by construction, as each descendant is our own fine-tune of its parent (save for the medium pool's R1 stage).

Concluding remarks

We thank the reviewer for their time and detailed feedback. Our new rebuttal results and additional clarifications seek to definitively answer all the main questions. If the reviewer feels satisfied, we would be thrilled if they might consider a potential score upgrade. We are also happy to answer any further questions.

We would like to once again thank the reviewer for their time and care in reviewing our original submission. As the deadline is fast approaching, we just wanted to ensure that our responses and added experiments adequately address the main concerns regarding teacher isolation, conditional attack breakdowns, and per-behavior analysis. We point out the following facts that may help with any further evaluation: 1) We have unified our experimental pipeline to ensure that the effect of distillation can be isolated more clearly and find that with more control over the fine-tuning process our original claims are either maintained or further substantiated. 2) We have disaggregated our original data to get a better understanding of any emerging vulnerability patterns. 3) We have added confidence intervals to strengthen the credibility of our claims. 4) We have extended the related works section to position ourselves against some related works pointed out by the reviewer to further demonstrate the necessity of our work.

We would be happy to engage in any further discussion if there are some remaining unclear points. We are appreciative of the extensiveness and the time that went into the original review and hope that our responses and updated experiments are detailed enough to potentially merit a score adjustment.

Reviewer_u9SUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Systemic Adversarial Robustness (SAR): the normalized minimum number of distinct inputs needed to compromise a target fraction of a model pool. It formulates SAR as partial set cover and reports reduced estimated SAR after shared-teacher training or post-training.

Critical strengths
  • Measuring attack reuse across a model population is useful and complements individual robustness and single-input transferability.

  • The set-cover formulation is clear, with an approximation guarantee under an explicit oracle assumption.

  • Experiments span vision and language models and reveal potentially interesting collective failure patterns.

Critical weaknesses
  1. The empirical estimates do not consistently implement the defined metric. GCG and AutoDAN assign hypothetical singleton attacks to models never successfully compromised. These are not verified covers and therefore do not establish an upper bound on true SAR. BEAST instead assigns SAR=1 when search fails, mixing target attainability with attack count. Report target-reaching frequency and conditional counts separately, including the contribution of hypothetical attacks and clean failures.

  2. The experiments do not isolate a shared-teacher effect. MNIST students receive the same new images and teacher labels simultaneously. DeepSeek comparisons combine distillation with instruction/reasoning post-training. Opus AutoDAN compares separately selected behavior sets—25 for parents and 22 for descendants. Matched behaviors and shared-data/different-teacher controls are necessary for causal conclusions.

  3. Evidence of increased cross-model sharing remains limited. Table 1’s largest reduction occurs at a one-model target, which requires no shared vulnerability. Higher-coverage differences are smaller and lack paired uncertainty estimates. Success-overlap matrices and per-behavior results would help distinguish greater individual attackability from greater vulnerability overlap.

Additional issue: normalization makes SAR sensitive to model duplication. If every model is replicated (r) times, the minimum attack count at a fixed coverage fraction remains unchanged, while normalized SAR falls by (r). Thus, adding identical copies lowers the score without changing the attack repertoire. This does not invalidate the fixed-size comparisons, but limits interpretation across ecosystems. Report raw attack counts and clarify whether models represent distinct behaviors, checkpoints, or deployment instances.

Other concerns

Clarify novelty relative to prior distillation-transfer studies and greedy prompt-coverage methods. Reconcile behavior counts and search budgets, and independently audit judge-dependent successes.

Questions for authors
  • Do the conclusions hold using only verified attacks on matched behaviors?

  • How much of each SAR change reflects target attainability rather than fewer attacks conditional on success?

  • Does the MNIST effect survive shared-data and different-teacher controls?

3 comments
AuthorsOpenReview ↗

Author Rebuttal (Part 1)

We would like to thank the reviewer for their thoughtful review and for providing clear and actionable feedback. We are grateful that the reviewer agrees with our main thesis that measuring SAR across a model population is “useful and complements individual robustness”. We also thank the reviewer for stating that they find our set-cover formulation to be “clear”, and for providing an approximation guarantee. Finally, we appreciate that the reviewer found our diverse experiments across vision and language models to “reveal potentially interesting failure patterns”. We next systematically address the chief outstanding questions raised in the review.

W1 - Empirical estimates

We acknowledge the reviewer's point that our empirical estimates may not initially appear to implement SAR consistently. We now clarify two important aspects of the definition of SAR, in particular the edge cases for when (A) a model produces a harmful output without an attack or (B) an attack is unable to compromise a member of the pool. Within our definition of SAR, these are handled with a consistent underlying semantic. When a model produces a harmful output without any additional vector, it is still a harmful response induced by the null vector. For (B), where an attack is unable to compromise a member of the pool, we consider this as requiring an individual vector modelling the assumption that even if we cannot find a vector, we cannot rule out that another much stronger adversary could find such a vector.

To this end, we now report unconditional SAR under our stated convention alongside conditional verified SAR. Unconditional SAR includes hypothetical singleton inputs where required to complete the cover. Conditional verified SAR is computed only over the n behaviors whose coverage target is reached using verified inputs, including the unmodified request when selected.

We report these quantities in Table 1 of the revised manuscript and reproduce them here for convenience.

PoolStageClean (%)Attack (%)None (%)Unconditional SAR (approx.)Conditional verified SAR (n)
Medium (N=6)Base29.850.719.50.5430.39 ± 0.04 (32)
Medium (N=6)R18.767.224.20.5120.28 ± 0.03 (29)
Medium (N=6)R1 → Opus 4.647.748.53.80.2880.25 ± 0.02 (85)
Large (N=8)Base9.442.648.00.5550.25 ± 0.05 (16)
Large (N=8)R117.954.227.90.3960.20 ± 0.03 (44)
Large (N=8)R1 → Opus 4.624.451.024.60.3690.20 ± 0.02 (51)

The reported ± values are half-widths of 95% confidence intervals. Clean, Attack, and None are percentages of model–behavior pairs.

Within this table, we quantify and observe:

  • Unconditional quantities: These are under our original assumptions and display the same clear trend observed in our submission.

  • Verified target attainment, reported through n: between the initial and final pools the number of hypothetical attacks needed reduces, fortifying our claim.

  • Conditional verified SAR: the conditional mean decreases on both the medium and large pools fortifying our claim, albeit with fewer samples it is not statistically significant.

  • Clean (%): the share of model–behavior pairs producing harmful responses to the unmodified request increases at nearly every fine-tuning stage representing a general trend of loss of alignment contributing to SAR.

  • None (%): the share of model–behavior pairs not compromised by any tested input show a downward trend save for an increase in the medium pool after R1 fine-tune.

W2 - Isolating a shared-teacher effect

In our new unified, local fine-tuning experiments we vary labelling regimes for MNIST fine-tuning to explore the relationship between data, teacher, and dependents better (conditions A-D in the table below). In Figure 3 of the main text of our revised submissions, we see that ground-truth labels have the smallest effect on SAR, followed by 20 different teachers, followed by a draw between using all data and disjoint data. This supports the claim in our original submission. We provide concrete values with uncertainty estimates in Table 4, seeing that A, B, and C are statistically significantly different from method D:

ConditionSAR₀.₈₅ change (%)1 − ASR change (%)
(A) One teacher, all data−22.3 ± 2.5−7.1 ± 2.7
(B) One teacher, disjoint 12k subsets−23.0 ± 2.4−8.0 ± 2.7
(C) 20 different teachers−17.6 ± 2.3−8.3 ± 2.3
(D) Ground-truth labels−3.1 ± 2.7+6.5 ± 2.3

Additional issue - Normalization

We appreciate this point; however, we would like to clarify that our work states a model pool as being comprised of individual or unique models (now emphasized in our revision). Using duplicate models would conceptually have the effect of implying a higher severity of compromising one model over others.

AuthorsOpenReview ↗

Author Rebuttal (Part 2)

W3 - Evidence of cross-model sharing

We agree with the reviewer that, for very high delta values, as shown in Table 1 of the original submission, one cannot make meaningful conclusions about shared vulnerabilities. In the original submission, this number was given as a reference, not as evidence of shared vulnerability. In our revised paper, the trend across delta values is much clearer (Figure 5 in the manuscript); we have also amended the manuscript with Table 7 showing 95% confidence intervals over SAR_{0.85}. Based on the findings of Figure 5 and Table 7, the claim of general SAR decline should have more significance. We also add per-behavior results (Figures 9-10).

PoolTransitionδ = 0: Δ SAR [95% CI]Rel. changeDown / tie / upδ = 0.15: Δ SAR [95% CI]Rel. changeDown / tie / up
Medium (N=6)Base → R1−0.032 [−0.073, +0.010]−5.8%44 / 31 / 25−0.032 [−0.073, +0.010]−5.8%44 / 31 / 25
Medium (N=6)Base → R1 + Opus 4.6−0.255 [−0.287, −0.223]−46.9%84 / 15 / 1−0.255 [−0.288, −0.223]−46.9%84 / 15 / 1
Medium (N=6)R1 → R1 + Opus 4.6−0.223 [−0.265, −0.182]−43.6%70 / 25 / 5−0.223 [−0.267, −0.182]−43.6%70 / 25 / 5
Large (N=8)Base → R1−0.168 [−0.209, −0.128]−24.8%66 / 24 / 10−0.159 [−0.199, −0.121]−28.6%65 / 26 / 9
Large (N=8)Base → R1 + Opus 4.6−0.199 [−0.241, −0.158]−29.4%72 / 16 / 12−0.186 [−0.228, −0.146]−33.6%70 / 20 / 10
Large (N=8)R1 → R1 + Opus 4.6−0.031 [−0.069, +0.006]−6.1%44 / 25 / 31−0.028 [−0.062, +0.007]−6.9%40 / 31 / 29

SAR change for delta = 0 and delta = 0.15 on the medium and large pools. [Table 7 in revised manuscript]

Questions

Q1 - "Do the conclusions hold using only verified attacks on matched behaviors?"

Yes, in point estimate. Conditional verified SAR decreases between the initial and final medium pools (0.39 → 0.25) and large pools (0.25 → 0.20). However, these conditional results do not establish a statistically significant paired decrease due to the small number of verified-cover behaviors.

Q2 - "How much of each SAR change reflects target attainability rather than fewer attacks conditional on success?"

Our disaggregation shows changes in both target attainability and conditional cover size. In both the medium and large pool, the proportion of behaviors whose coverage target is reached by verified inputs increases while conditional verified SAR decreases. These results show why the unconditional SAR decline should be interpreted alongside both quantities. Increased verified target attainment is clearly observed, while the evidence for fewer inputs conditional on success is more limited.

Q3 - "Does the MNIST effect survive shared-data and different-teacher controls?"

Yes. This is clearly demonstrated in Figure 3 of the revised manuscript.

Novelty related to prior work

We thank the reviewers for pointing us to further related work on the effects of distillation on attack transferability, in particular Angell et al. (2026), Ojha et al. (2023) and JBDistill (Zhang et al., 2025), which we now discuss in the related works section. Angell et al. and Ojha et al. study whether, and why, vulnerabilities or jailbreaks transfer between models under distillation. While this relates closely to our work, both papers evaluate transfer pairwise or for single inputs, whereas we evaluate a pool of models simultaneously. Their results show that students can inherit vulnerabilities from a teacher, but they do not ask whether students become vulnerable to the same inputs as each other, or how many distinct inputs are needed to compromise a target fraction of the pool. JBDistill’s BestPerGoal procedure greedily selects prompts that compromise previously uncovered models, which closely resembles the greedy step of our estimator. Its goal, however, is to construct effective, diverse, and separable safety benchmarks, using coverage as a criterion for selecting prompts. Our contribution is to treat the minimum cover size itself as a measure of pool-level robustness, to formalise it as a partial set cover problem with approximation guarantees under an approximate attack oracle, and to show empirically that it can diverge from individual robustness.

Concluding remarks

We greatly appreciate the reviewers' time and constructive feedback, which enabled us to greatly improve the clarity and empirical results of our paper in this rebuttal. We believe we have answered all the outstanding questions with our new experimental results and clarified theoretical exposition. We hope our rebuttal here enables the reviewer to consider a fresh evaluation of our paper, with a potential score upgrade if merited. We are also more than happy to engage with any lingering questions. Please do let us know!

Dear Reviewer,

We once again want to express our gratitude for all of the time and effort in dissecting our initial manuscript. We appreciate that our responses added a substantial amount of new information to be reviewed. As the deadline is approaching, however, we wanted to ensure that we have touched and aligned on the main open concerns: 1) We added clarifications to the definition of SAR to ensure that edge cases are covered and soundness is maintained. 2) We report conditional and unconditional SAR, showing that initially claimed trends continue to be followed even when further dissecting attack patterns. 3) Our updated MNIST experiments strengthen our original claims of shared teacher impact on vulnerability propagation.

We hope that our responses and updated experiments have helped strengthen our work in the reviewer’s view and are more than happy to further discuss any outstanding points. We thank the reviewer again for their time and effort and if our updated experimental evaluation and responses merit, a fresh evaluation of our work would be kindly appreciated.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response to Reviewers

We thank the reviewers for their careful and constructive reviews. We use this global response to highlight the main new experimental additions and textual clarifications. As several concerns are shared amongst the reviewers, we provide the high-level conclusions and headline numbers here, deferring detailed points to individual responses. In particular, we stress that we have re-run our results under a unified protocol that no longer relies on community fine-tuning checkpoints. The re-run experiments reproduced the same phenomenon stated in the original submission, which reinforces the shared vulnerability point we advance. We next address the shared concerns grouped by reviewer and theme.

Unified Experimental Harness for Streamlined Interpretation (u9SU and khox)

We value the constructive feedback from reviewers u9SU and khox that noted that our original LLM comparisons mixed attacks, behavior sets, and post-training procedures. We agree that this makes drawing conclusions from our experiments more difficult, and we have comprehensively addressed this throughout our revised manuscript. In particular, we would like to stress:

  • Unified evaluation data and attack: 100 HarmBench behaviors and attack algorithm (Ample-GCG + BEAST)
  • A unified fine-tuning approach: we now do all of the SFT manually rather than relying on community checkpoints, save for the R1 checkpoints supplied by DeepSeek, which are retained as a gold standard. The original submissions' findings were confirmed in this more consistent setting; moreover, the original submissions' experiments are now preserved as ablations which serve to underscore the phenomenon we observe in the original paper.

Clean Failures and Target Attainability (u9SU and khox)

Reviewers u9SU and khox both point out the lack of clarity around two edge cases in SAR evaluation that are now clearly stated in our revised manuscript: (A) Clean vulnerabilities - models which produce harmful outputs without an attack vector are considered naturally in our framework as the null/0 vector is a distinct vector, thus all such "clean vulnerabilities" count as one vector towards the cover.

(B) Robust models - when a model is found to be robust to all tested attack vectors, we took on the (clearly stated) convention that these would count as one vector towards the SAR. This encodes the assumption that even though we were unable to find such a vector, we assume that a stronger adversary would be able to find one.

We now clearly state the proportion of inputs in a pool that hit these two edge cases both in the main text of our submission and in our responses to individual reviewers. Each of the disaggregations supports the primary hypotheses we advance. In particular, the number of robust models (unattackable) decreases between the initial and final pools, while the number of clean vulnerabilities depends on whether the set covered by the null vector is overtaken by another vector. Further, there was a shared concern about the implicit assumption of (B) in our original submission that an attack's inability to find an effective vector does not imply that one does not exist. This is the correct assumption for non-exhaustive attacks as falsification does not imply certification; however, we take the reviewers' point that providing the number of behaviors covered explicitly by attacks is an informative metric with regards to SAR and now report it in the appendix and the paper.

Isolating the Teacher Effect (u9SU and khox)

Reviewers u9SU and khox both point out that our experiments train the pool using a shared teacher as well as a shared dataset. This confounds our conclusion, as the shared vulnerabilities could come from either the shared teacher or the shared dataset that is used. We isolate the contribution of the teacher to the shared vulnerabilities by re-running our procedure using disjoint datasets and disjoint teachers. The results clearly demonstrate that, as hypothesized, shared vulnerabilities come primarily from teacher supervision rather than shared data. On MNIST, using disjoint data has negligible effect on the observed SAR decline, using 20 different teachers does not lead to as steep a decline in SAR, and ground-truth labels remove almost all SAR decline.

Uncertainty Estimates (u9SU and khox)

Reviewers u9SU and khox both point out that our experimental results could be improved with the introduction of statistical hypothesis testing to clearly communicate where our results are statistically significant. To this end, we have introduced 95% confidence intervals for each of our results. We thank the reviewers for this suggestion, as it reinforces the core results of our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.