acceptodds
Under review as a conference paper at ICLR 2027

Configuration-Aware LLM Red Teaming: A Theory-Guided Transfer Framework

Abstract

Recent advances have made automated red teaming increasingly efficient on open-weight LLMs. However, how surrogate configurations and attack strategies jointly shape transfer to unseen deployments remains poorly understood. We formulate a configuration-aware transfer framework for automated LLM red teaming by combining a localized discrepancy analysis with a strategy-conditioned decomposition of base-model and system-policy mismatch. The resulting bound connects ensemble-averaged success with the coverage supplied by the chosen configurations and strategies. Empirically, broader success across surrogates is associated with higher held-out transfer, and changing the system policy at fixed model weights reshapes response profiles. Based on these findings, we build CAST, Configuration-Aware Surrogate Transfer, which combines strategy-conditioned candidate generation, ensemble-averaged guard scores, and explicit control of the system policy its surrogates run under. Our evaluation spans 300 HarmBench behaviors and 34 deployment configurations. Under the primary evaluator, CAST outperforms leading automated red-teaming baselines in mean transfer attack success rate (ASR) on held-out open and hardened configurations, excluding optimization sources. Ablation studies empirically support the design implications of our theoretical bound for surrogate coverage and system-policy composition.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM red teaming, jailbreak transferability, AI safety evaluation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_zQyYOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The authors try to take a systematic look at attack transfer from surrogates and ensembles of surrogates to target models. The authors find some expected results (e.g., attacks that break more surrogates are more likely to break targets too), and some less obvious ones (e.g., diminishing gains after only ~4 ensembled models and only slight benefits from using hardened surrogates). The proposed transfer attack framework "CAST" relies on attacking an ensemble of four models and iteratively re-writing the attack prompt using TextGrad; the method shows improvements on open models and slight improvements on frontier models thought data there is weaker.

Critical strengths
  • The authors find clear connections between how many surrogates are defeated by an attack and how likely the attack is to transfer. This finding is supported by clear evidence and has implications for attacks in practice (e.g., to estimate the likelihood of transfer for new prompts).
  • The comparison study uses standard data and is quite thorough, with 34 models and nine baselines (though the most relevant attacks - transfer-focused ones like ArrAttack, or Diffusion-LLM-based attacks are missing). LLM Red Teaming historically suffered from fragmented evaluations and relying on stable, known baselines helps create trust in new results and makes them easier to reproduce.
Critical weaknesses

The key weakness of this paper is its unclear writing and presentation which makes it very difficult to understand what is being done/shown and to understand the motivations behind why it is being done. As an example, the theory section appears overcomplicated and out of place with the rest of the paper. It is not clear how the theory helps motivate or understand the findings/method, which are rather simple (nothing wrong with that, it just doesn't fit with the other section). The paper also uses a lot of different non-standard terms for similar concepts - for example candidate, configuration, policy, surrogate, source, ensemble, direction, group, ... which adds significant cognitive load to the reader.

Questions for authors
  • Can you clarify how the theory connects to the empirical results or the proposed method?
  • How do attacks designed for transfer, like ArrAttack or Diffusion-inpainting attacks compare to CAST?
  • Have you run experiments with different ensembles than the Llama-2, Llama-3.1, gpt-oss-20b, Qwen2.7 set mentioned in 4.2?
Reviewer_s2dXOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how to construct a surrogate ensemble to improve the transferability of adversarial prompts in LLM red-teaming. The paper presents a theoretical analysis of attack transferability and proposes an algorithm called CAST based on the design principles derived from this analysis. CAST combines multiple surrogate models and attack strategies to generate highly transferable attacks.

Critical strengths

Transferability in LLM red-teaming is an interesting and important problem that deserves attention from the research community. This paper provides systematic benchmarks and experiments on transferability, offering valuable insights and resources for the community.

The experimental finding that changing the system policy can alter a model's vulnerability, even when the underlying model remains the same, is interesting. Since system policies can modify a victim model's behavior relatively easily, they represent an important factor in red-teaming. The systematic analysis of model behavior under different system policies is a strength of this paper.

The finding that attacking victim models equipped with strict system policies can produce more transferable attacks may offer useful practical guidance.

Critical weaknesses

The methodological novelty is limited. CAST essentially extends TextGrad-based attack optimization by combining average rewards across multiple surrogate models with predefined attack strategies. However, ensemble-based transfer attacks are already well-established. Previous studies have also proposed exploiting vulnerabilities shared across multiple models to improve transferability [1, 2]. The main contribution of CAST appears to lie in constructing a surrogate ensemble using strict system prompts. While this is a sufficiently interesting finding, I believe it deserves greater emphasis in the paper.

The connection between the theoretical analysis and the actual algorithm is relatively weak. The theory addresses transferability in a general setting, but the proposed algorithm does not explicitly estimate or minimize the discrepancy. Although the theoretical framework provides a useful interpretation, I wonder whether it occupies a disproportionate amount of space in the paper. In particular, the theory does not explain why applying the same strict policy to all surrogate models would be optimal or advantageous. While the intuition is understandable, the connection between the theory and the algorithm could be strengthened, for example, by conducting a quantitative post-hoc analysis of the discrepancy.

References

[1] Chen et al. (2024), Rethinking Model Ensemble in Transfer-based Adversarial Attacks, ICLR 2024.

[2] Yang & Fu (2024), Transferable Ensemble Black-box Jailbreak Attacks on Large Language Models, arXiv.

Questions for authors

The idea of generating more transferable attacks by targeting more strongly defended models appears similar to [3]. Although [3] trains and employs a robust judge, I believe the underlying intuition is similar. How do the authors view this connection? On the other hand, attacks generated against strongly defended models might exploit vulnerabilities specific to those models more heavily, potentially resembling reward hacking. What are the authors' thoughts on this possibility?

Would it be possible to investigate the correlation between the strength of a strict system prompt and attack transferability? It seems that such an analysis might be feasible if the defensive strength of system prompts could be quantified.

[3] Li et al. (2025), One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs, ICLR 2025 (ArrAttack).

Reviewer_NeLaOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies transfer-based black-box jailbreaking. An attacker optimizes prompts on accessible surrogate models and sends each selected prompt once to an unseen deployment. The authors argue that the relevant unit of transfer is a deployment configuration: model weights, system policy, and fixed execution settings. They specialize the localized f-divergence analysis of Wang & Mao (2024) to this setting, treating attacks as hypotheses and configurations as examples. The result is a transfer bound (Theorem 2), a strategy-conditioned extension (Corollary 4), and a decomposition of the deployment shift into a base-model term and a system-policy term through a hybrid population (Eq. 4–5, Corollary 5). The bound motivates two design rules: (T1) optimize the mean success over surrogate configurations, and (T2) compose the surrogate ensemble, including its system policies, to cover the target population.

Critical strengths
  1. Treating the system policy as a coordinate of the deployment, and as a variable the attacker can control on the source side, is a practical idea. The finding that a strict policy at fixed Llama-3.1-8B weights reduces the rank correlation of vulnerability profiles to 0.271 (0.375 after reliability adjustment) is informative for red-teaming practice. It suggests that evaluating a base model without its deployed policy can misrepresent the deployed system.
  2. Table 2 separates source count at a fixed response budget (K = 1→4: 39.3→44.2 held-out) from policy composition at fixed weights (d = 0→4: 44.2→49.5 held-out, 53.3→63.7 hardened). This separation is cleaner than most ensemble-attack ablations. Appendix D.2 is a useful negative result: giving AutoDAN four sources raises its in-pool mean from 49.4 to 64.3 but drops its frontier mean from 6.2 to 0.2. This shows that naive multi-source fitting overfits the source side.
  3. The paper evaluates eleven arms on 34 configurations under five judges, reports all per-target numbers (Table 7), and separates the optimization reward (Qwen3Guard) from the primary judge. The authors are candid about negative findings. They state that the frontier lead rests on a single system (Sec. 5.2), and they report that ReNeLLM leads under LlamaGuard-3.
Critical weaknesses
  1. By Appendix B.1, each of the four held-out hardened configurations shares one coordinate with CAST's own ensemble. Two have the same weights (Llama-3.1-8B mild, Qwen2.5-7B mild), and two have the same strict policy text (Llama-3.2-3B strict, Ministral-8B strict). Table 7 shows that the hardened margin comes mostly from the shared-weights targets. There, CAST scores 87.6 and 88.3 against AutoDAN's 55.3 and 51.7. On the two policy-only-shared targets, the results are mixed: 1.9 vs. 7.7 on Llama-3.2-3B, and 77.1 vs. 59.3 on Ministral-8B. The baselines, which use Llama-2-7B-chat as their only source, see neither the weights nor the policy. The 20.2-point hardened margin, which is a headline claim, therefore partly measures source–target overlap rather than transfer to unseen deployments. A group of hardened targets that share neither weights nor policy with any source is needed.
  2. The abstract and contributions are careful to claim only open and hardened leads. Still, the most deployment-relevant group is where transfer is weakest. Excluding Gemini-3.1-flash-lite, CAST averages 7.6% on frontier APIs against ReNeLLM's 9.5%. On Claude Haiku 4.5, CAST gets 0.4% against ReNeLLM's 14.3%. On the more recent open models, CAST also drops sharply: Nemotron-3-Super-120B 34.2, Gemma-4-31B 27.2, and 13.2 on the latest Qwen 27B model. On Mistral-v0.3, the unmodified behavior (direct, 55.3) beats CAST (37.0). The framework is motivated as covering the target population, so it should explain why coverage fails exactly on the targets that are furthest from the source pool. As written, the evidence mainly supports transfer to older or same-family open models.
  3. The bound's discrepancy terms and feasibility constants cannot be estimated, as the authors acknowledge. Its two design rules, averaging over an ensemble and diversifying the ensemble, are well-established empirical lessons from ensemble-transfer attacks (Liu et al., 2017; Dong et al., 2018; Chen et al., 2024). The connection to experiments is correlational: Fig. 1b shows eight subset sizes, and the Spearman–Brown fit in Lemma 6 is descriptive and uses all eight subset-mean correlation points, although the accompanying table displays four representative values. The bound does not predict any design choice that could not have been made without it. For example, it does not say which policies to add, how many sources suffice, or when to stop adding them. The contribution relative to Zheng et al. (2026) is also not stated precisely. That work already analyzes localized model-distribution discrepancy and finite surrogate sampling for ensembles. In my reading, the delta is the strategy restriction and the product-population split of model and policy. The paper should say so explicitly and argue why this delta is non-trivial.
Questions for authors
  1. Can you report CAST and the baselines on hardened targets that share neither weights nor policy text with any CAST source? For example, a held-out model family under the unseen policy of Appendix D.3.
  2. What explains the weak transfer to Claude Haiku 4.5, the GPT-5 series, and the newest open models, and the case where direct prompting beats CAST on Mistral-v0.3? Does the bound offer any diagnostic for these failures?
  3. Precisely which results are new relative to Zheng et al. (2026)? Does the theory make any quantitative or falsifiable prediction that you tested?
Reviewer_HvVqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies transfer-based automated red teaming for large language models (LLMs) through a novel, configuration-aware perspective. The authors formulate a theoretical framework that combines a localized ff-divergence discrepancy analysis with a strategy-conditioned decomposition of base-model and system-policy mismatches. Based on this theory, they propose CAST (Configuration-Aware Surrogate Transfer), which integrates strategy-conditioned candidate generation, ensemble-averaged guard scores, and explicit control over system policies in surrogate environments. Through extensive experiments spanning 300 HarmBench behaviors and 34 deployment configurations (including open-weight models, hardened policies, and commercial frontier APIs), the paper demonstrates that CAST achieves superior transfer success rates compared to existing automated red-teaming baselines. Controlled ablations and policy-separation checks further validate the importance of surrogate coverage, composition, and system-prompt design.

Critical strengths
  1. Rigorous Theoretical Foundation: Unlike many heuristic-driven red-teaming papers, this work grounds transfer red teaming in a principled localized ff-divergence and domain-adaptation framework. The decomposition into base-model and system-policy shifts provides a clear, actionable explanation of why traditional transfer attacks fail across heterogeneous deployment setups.

  2. Comprehensive and Demanding Evaluation: The empirical evaluation is exceptionally thorough, testing across 300 behaviors and 34 diverse configurations (including open, hardened, and proprietary frontier systems). The inclusion of multiple guard evaluators and controlled ablations (such as source count versus policy composition) strongly substantiates the core claims of the paper.

Critical weaknesses
  1. Dependence on Guard Quality and Potential Reward Hacking: The optimization reward of CAST relies heavily on a specific guard model (Qwen3Guard-Gen-8B). Although evaluated across multiple independent judges, the framework's reliance on automated guard scores rather than human judgment may introduce vulnerability to guard-specific artifacts or reward hacking during prompt optimization.

  2. Limited Practical Applicability of Complete System-Policy Knowledge: The framework treats system policies as an explicit design coordinate and assumes access to or control over surrogate system prompts to harden them. In real-world black-box settings, obtaining exact target system prompts for surrogate composition can be challenging, and the paper's shared-text / unseen-policy checks leave room for deeper exploration of partial prompt leakage.

Questions for authors
  1. Clarification on Policy Leakage: In practical scenarios where the target's system policy is entirely proprietary and unknown to the red-teamer, how sensitive is CAST's performance to the choice or accuracy of the surrogate system policy?

  2. Future Work Direction: Could the proposed configuration-aware framework be extended to multi-turn conversational red teaming or multimodal inputs, where dialogue state and modality introduce additional complex configuration axes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.