Configuration-Aware LLM Red Teaming: A Theory-Guided Transfer Framework

Recent advances have made automated red teaming increasingly efficient on open-weight LLMs. However, how surrogate configurations and attack strategies jointly shape transfer to unseen deployments remains poorly understood. We formulate a configuration-aware transfer framework for automated LLM red teaming by combining a localized discrepancy analysis with a strategy-conditioned decomposition of base-model and system-policy mismatch. The resulting bound connects ensemble-averaged success with the coverage supplied by the chosen configurations and strategies. Empirically, broader success across surrogates is associated with higher held-out transfer, and changing the system policy at fixed model weights reshapes response profiles. Based on these findings, we build CAST, Configuration-Aware Surrogate Transfer, which combines strategy-conditioned candidate generation, ensemble-averaged guard scores, and explicit control of the system policy its surrogates run under. Our evaluation spans 300 HarmBench behaviors and 34 deployment configurations. Under the primary evaluator, CAST outperforms leading automated red-teaming baselines in mean transfer attack success rate (ASR) on held-out open and hardened configurations, excluding optimization sources. Ablation studies empirically support the design implications of our theoretical bound for surrogate coverage and system-policy composition.