acceptodds
Under review as a conference paper at ICLR 2027

RvB: Automating AI System Hardening via Iterative Red-Blue Games

Abstract

The dual offensive and defensive utility of Large Language Models (LLMs) highlights a critical gap in AI security: the lack of unified frameworks for dynamic, iterative adversarial adaptation hardening. To bridge this gap, we propose the Red Team vs. Blue Team (RvB) framework, formulated as a training-free, sequential, imperfect-information game. In this process, the Red Team exposes vulnerabilities, driving the Blue Team to learning effective solutions without parameter updates. We validate our framework across two challenging domains: dynamic code hardening against CVEs and guardrail optimization against jailbreaks. Our empirical results show that this interaction compels the Blue Team to learn fundamental defensive principles, leading to robust remediations that are not merely overfitted to specific exploits. RvB achieves Defense Success Rates of 90% and 45% across the respective tasks while maintaining near 0% False Positive Rates, significantly surpassing baselines. This work establishes the iterative adversarial interaction framework as a practical paradigm that automates the continuous hardening of AI systems.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM safety, AI system hardening, automated red teaming, adversarial adaptation, jailbreak defense, guardrail optimization, vulnerability remediation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_wtCQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a loop in which a fixed red-team agent attacks a persistent artifact, a fixed blue-team agent patches that artifact using only attacks that a checker or judge accepted, and the patched artifact then becomes the target for the next red round. The state is written as S_k = (X_k, C_k), with X_k the editable thing (a deployed service plus its source tree, or a set of guardrail rules) and C_k an audit archive. Three commitments are stated: only accepted attacks condition blue, updates persist, and security and utility are observed separately (Sections 3–4, Eqs. 1–5).

There are two instantiations. For code, CAI attacks two PHP applications (Pharmacy Management System v1.0 and OOP CMS BLOG 1.0) running as isolated Docker services, and Mini-SWE-Agent patches the source, restarts the service, and runs vulnerability plus regression checks. The main metric is TDSR — attack blocked and regression passes — with FDSR (blocked only) and SDR = FDSR − TDSR; ASC counts distinct reproduced exploits. Reported: Pharmacy 60%→80%→90% with ASC 5→7; OOP CMS 60%→40%→80%→100%→100% with ASC 5→8; a Qwen3-Max backbone record 30%→50%→60%→60%→60% with ASC 1→7. A "cooperative" reference (same agents, five-epoch cumulative union) is put at 70% / 60% / 50%.

For guardrails, a Composition-of-Principles jailbreaker attacks HarmBench targets through Qwen2-7B-Instruct behind NeMo guardrails, and blue rewrites the external rules. Per target there are at most three refinement rounds of at most ten attempts. Reported: DSR 0%, 0%, 10%, 10%, 25%, 45%; attempts among successful attacks 1.50→11.09; FPR on round-specific benign sets 0–8.77%; retained-attack replay improves inside each fixed set (round 1: 84.2→94.7%; round 2: 11.1/44.4/61.1%; round 3: 63.2/66.7%); four external benchmarks start at 96–100% and move by 0–3 points.

Critical strengths

The measurement interface is right, and it is actually implemented. Splitting "attack blocked" from "attack blocked and the service still works" is a real fix for a real failure mode: a patch that disables an interface or breaks a regression test trivially defeats an exploit. Figure 6 shows why it matters — the cooperative run has a 60-point SDR gap while RvB's gap is zero on the recorded checks. I'd like to see other agentic-security papers adopt this.

The paper reports evidence against its own thesis. The OOP CMS regression and the Qwen plateau are not flattering, and they are the most interesting results in the paper. Most iterative red-team papers show monotone improvement; this one shows that the trajectory has to be inspected, and the two case studies support that claim.

The two domains are genuinely different. Table 9 isn't decoration: one case updates source code with executable checks and regression tests, the other updates external rules with judged attacks and benign-input screening. That is what makes "shared protocol" more than a slogan.

Critical weaknesses
  1. The headline comparison is not apples-to-apples. The cooperative baseline uses a five-epoch cumulative-union estimator, RvB reports a final-state estimator. The paper says so in several places (Section 1, Section 4, Section 5.1, Table 1 caption) and then still puts "significantly surpassing baselines" in the abstract. These two numbers answer different questions: a union over fixes found across five artifacts says nothing about the security of any single deployed artifact. Appendix D even spells out the fix — replay a fixed exploit set on each final artifact — and then it isn't done. Until both methods' final artifacts are scored on the same held-out suite, 90/100 versus 70/60 is not evidence for the protocol.

  2. The denominator is unknown and looks tiny. The paper never states N, the number of checks. The reported values pin it down: 60/70/80/90/100% on Pharmacy, 60/40/80/100% on OOP CMS, 30/50/60% on Qwen — all exact multiples of 10. That is consistent with N = 10 checks per application. If so, "90% vs 70%" is a gap of two checks out of ten, and the OOP CMS "regression" is two checks. The paper also doesn't say whether the check population is fixed across rounds. If newly discovered exploits become new checks, DSR can fall while every old check still passes — so the round-2 drop might be a denominator artifact, not a bad patch. The paper admits it can't tell these apart. That is the single biggest problem, because the trajectory is the paper's main object.

The guardrail side has the same issue at a different scale: the percentages imply roughly 19–20 harmful targets (10% = 2/20, 25% = 5/20, 45% = 9/20), and benign sets of about 19, 57 and 85 prompts (5.26% = 1/19, 3.51% = 2/57, 7.02% = 4/57, 8.77% = 5/57, 2.35% = 2/85). With n≈20, one target moves DSR by 5 points, and 0→45% is nine targets. There is no confidence interval anywhere in the paper.

  1. The guardrail evaluation is circular in two places. Table 6: the attack instantiator and the attack judge are both Qwen3-32B, and the guard classifier and the rule patcher are both GPT-4o mini. First, a jailbreak counts as successful when the same model family that generated it says so. Second, the defence is a classifier whose decisions and whose rule updates come from the same model, so part of the measured gain may be co-adaptation to that classifier rather than a general defensive principle. The paper flags the first ("Attack generation and judging use the same model family") and then doesn't measure it. A second judge from a different family, plus a small human-labelled subset, would tell us how much of 0→45% survives.

  2. One run per configuration, no ablations, and cost is hard to read. The code results are single runs — "one representative code record". No seeds, no variance, no CIs. Nothing isolates the structured report, the persistent state, the retry budget or blue's tool access. The paper says this itself ("without component ablations it does not isolate the causal contribution"), which is honest but leaves the mechanism unexplained. On cost: code uses 5.11M tokens for RvB versus 6.25M for the cooperative run (−18.35%), and guardrail tokens grow from 4.04M at baseline to 12.22M at round 4 — the iteration is not cheap — yet there is no wall-clock or budget-matched comparison, so the −18% figure doesn't tell me much.

  3. The abstract and the body disagree. "Near 0% False Positive Rates" vs FPR up to 8.77% in Figure 10. "Significantly surpassing baselines" vs a comparison the paper itself calls descriptive. "Robust remediations that are not merely overfitted to specific exploits" vs the paper's own statement that CRDE gains "do not distinguish a general defensive principle from several targeted rules". "90% and 45% DSR" without denominators, with a non-independent judge, and with 55% of harmful targets still jailbroken at the end. A reader who stops at the abstract gets a materially wrong picture.

  4. Reproducibility is partial. The anonymous links are there (three anonymous.4open.science repos on page 10), which is good. But there are no model snapshots or decoding settings ("gemini-3.0-flash" with no provider or version; "Qwen3-Max" with no revision), no per-attempt logs, no per-set denominators, and no explanation of the task-grid anomaly.

Questions for authors

What is N, the number of checks per application? Are the ten checks on Pharmacy the same ten in every round? Please give counts (k/n) rather than percentages for TDSR, FDSR and CRDE. Please run the comparison Appendix D describes: score the cooperative final artifact and the RvB final artifact on the same fixed, held-out exploit suite, with at least five seeds and intervals. If the gap goes away, which sentence in the abstract survives? For OOP CMS, is round 2's drop caused by a bad patch, by new checks from newly discovered exploits, or by an unavailable service? A per-check table for rounds 1–5 would settle it. Rerun the guardrail DSR/AAT with a judge from a different model family and report judge–judge agreement. Is the guard classifier (GPT-4o mini) independent of the rule patcher (also GPT-4o mini)? What happens with (a) free-text attack reports instead of the structured report, (b) a state reset (original artifact redeployed each round), (c) a cooperative baseline at matched token/call budget? These would tell us which ingredient matters. How many harmful targets and benign prompts per round were actually used? Are the guardrail token totals per round or cumulative, and what is the wall-clock cost per round? What are task-grid entries 34 and 39, and why do they exceed the 3×10 cap? Do the authors stand by "significantly surpassing baselines" and "near 0% False Positive Rates"? If not, please revise the abstract. Several sentences refer to "the manuscript" in the third person. Were these produced by an AI consistency check and left in? If yes, they should be removed, because they make the claims impossible to attribute.

Reviewer_sKzaOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes RvB, an automated security hardening framework based on iterative attack–defense feedback. The framework uses the Red Team to continuously discover system vulnerabilities and the Blue Team to modify the target system based on the attack results, allowing the updated system to undergo further attack testing in subsequent rounds. The entire process does not require updating LLM parameters, as continuous optimization is achieved by modifying external code or security rules. The paper evaluates RvB on two tasks: Code Hardening for vulnerable PHP applications and Guardrail Optimization against LLM jailbreak attacks. The experimental results show that the overall defense success rate improves in Code Hardening, while the defense success rate in Guardrail Optimization increases over successive iterations. The guardrail experiments also demonstrate improved resistance to previously observed attacks while maintaining relatively low false-positive rates on benign inputs.

Critical strengths
  1. The paper adopts a dynamic Red–Blue interaction mechanism. Compared to static attack evaluation and one-time repair, this design better reflects the continuous attack–defense process in real-world scenarios.

  2. RvB is a training-free framework that reduces the need for model parameter access and additional training resources.

Critical weaknesses
  1. The research positioning and core contribution of the paper remain unclear. RvB continuously modifies external artifacts through iterative Red–Blue interaction, but it is not fully explained whether its main contribution is a more effective defense optimization method or a general framework for dynamic attack–defense interaction and evaluation. If the focus is on defense optimization, a more comprehensive evaluation of the final system's security and its advantages over existing methods is needed. If the focus is on a general interaction framework, the technical novelty compared to existing Red–Blue methods needs further clarification. The current method and experimental results do not sufficiently support either positioning.

  2. Iterative Red–Blue adversarial optimization is not a new concept, so developing reusable attack and defense mechanisms across different LLM and agent security tasks should be an important technical contribution of this work. However, the Code Hardening and Guardrail Optimization tasks still rely on separately designed attack procedures, defense mechanisms, tools, and validation methods. It remains unclear whether these components can be automatically configured or transferred to new tasks. Therefore, the generality of RvB is mainly reflected in its high-level interaction workflow, while more specific reusable technical mechanisms across tasks are lacking.

  3. The paper evaluates RvB on only two types of tasks. Specifically, Code Hardening involves only two PHP applications, while Guardrail Optimization is evaluated on a single target LLM with one primary jailbreak method. The limited coverage of tasks and models is insufficient to fully validate the effectiveness and generality of RvB across different security scenarios.

  4. The existing experiments do not fully demonstrate the advantages of iterative Red–Blue feedback. For example, the experiments of Guardrail Optimization lacks a systematic evaluation of the final defense system against independent attack methods. In addition, the paper lacks comparisons with alternative approaches, such as a Static Red Team and Blue-only Iterative Optimization, as well as ablation studies of key components, making it difficult to isolate the contribution of iterative adversarial feedback.

Questions for authors

See Weakness.

Reviewer_doJZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes game between the red team and the blue team. The red team's objective is to find security problems in the artifact, whereas the blue team aims to protect it by adding security patches. They show that over time, both of the sides get better. E.g., the artifact will become safer and the the attacks will become more sophisticated.

Critical strengths

1- The paper is written very well and the figures are clear.

2- The motivation of the paper is strong. The current attack and defense evaluations work in a static setting. The paper proposes a game between the sides the can improve both.

3- The experimental setting (i.e. PHP source tree and NEMO based guardrail) are convincing. Then, the results are significant and show the improvements for the defense side, where the average attack time increases and the searching effort increases.

Critical weaknesses

1- My main concern is that there is no vivid improvement on the attacking side. E.g., the attacking side does not really improve and it's capabilities are capped. Whereas the introduction motivates it as a game that improves both sides.

2- That said, as the paper explains in lines 229 and 230: "ASC measures coverage, not attack sophistication; FDSR alone may include an unavailable service." So can the authors please provide any measure of progress on the attacking side? How can we achieve more sophisticated attacks if the model weights and the tools it uses remain the same?

3- If the attacker's power does not change, the iterations will quickly saturate -- this can actually be seen in Figures 4 and 5 that the ASC plateaus. Then, why do we even need the iterative game. We could instead run the red teaming in many parallel streams and patch all safety issues.

Questions for authors

Please read above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.