- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes a loop in which a fixed red-team agent attacks a persistent artifact, a fixed blue-team agent patches that artifact using only attacks that a checker or judge accepted, and the patched artifact then becomes the target for the next red round. The state is written as S_k = (X_k, C_k), with X_k the editable thing (a deployed service plus its source tree, or a set of guardrail rules) and C_k an audit archive. Three commitments are stated: only accepted attacks condition blue, updates persist, and security and utility are observed separately (Sections 3–4, Eqs. 1–5).
There are two instantiations. For code, CAI attacks two PHP applications (Pharmacy Management System v1.0 and OOP CMS BLOG 1.0) running as isolated Docker services, and Mini-SWE-Agent patches the source, restarts the service, and runs vulnerability plus regression checks. The main metric is TDSR — attack blocked and regression passes — with FDSR (blocked only) and SDR = FDSR − TDSR; ASC counts distinct reproduced exploits. Reported: Pharmacy 60%→80%→90% with ASC 5→7; OOP CMS 60%→40%→80%→100%→100% with ASC 5→8; a Qwen3-Max backbone record 30%→50%→60%→60%→60% with ASC 1→7. A "cooperative" reference (same agents, five-epoch cumulative union) is put at 70% / 60% / 50%.
For guardrails, a Composition-of-Principles jailbreaker attacks HarmBench targets through Qwen2-7B-Instruct behind NeMo guardrails, and blue rewrites the external rules. Per target there are at most three refinement rounds of at most ten attempts. Reported: DSR 0%, 0%, 10%, 10%, 25%, 45%; attempts among successful attacks 1.50→11.09; FPR on round-specific benign sets 0–8.77%; retained-attack replay improves inside each fixed set (round 1: 84.2→94.7%; round 2: 11.1/44.4/61.1%; round 3: 63.2/66.7%); four external benchmarks start at 96–100% and move by 0–3 points.
Critical strengths
The measurement interface is right, and it is actually implemented. Splitting "attack blocked" from "attack blocked and the service still works" is a real fix for a real failure mode: a patch that disables an interface or breaks a regression test trivially defeats an exploit. Figure 6 shows why it matters — the cooperative run has a 60-point SDR gap while RvB's gap is zero on the recorded checks. I'd like to see other agentic-security papers adopt this.
The paper reports evidence against its own thesis. The OOP CMS regression and the Qwen plateau are not flattering, and they are the most interesting results in the paper. Most iterative red-team papers show monotone improvement; this one shows that the trajectory has to be inspected, and the two case studies support that claim.
The two domains are genuinely different. Table 9 isn't decoration: one case updates source code with executable checks and regression tests, the other updates external rules with judged attacks and benign-input screening. That is what makes "shared protocol" more than a slogan.
Critical weaknesses
-
The headline comparison is not apples-to-apples. The cooperative baseline uses a five-epoch cumulative-union estimator, RvB reports a final-state estimator. The paper says so in several places (Section 1, Section 4, Section 5.1, Table 1 caption) and then still puts "significantly surpassing baselines" in the abstract. These two numbers answer different questions: a union over fixes found across five artifacts says nothing about the security of any single deployed artifact. Appendix D even spells out the fix — replay a fixed exploit set on each final artifact — and then it isn't done. Until both methods' final artifacts are scored on the same held-out suite, 90/100 versus 70/60 is not evidence for the protocol.
-
The denominator is unknown and looks tiny. The paper never states N, the number of checks. The reported values pin it down: 60/70/80/90/100% on Pharmacy, 60/40/80/100% on OOP CMS, 30/50/60% on Qwen — all exact multiples of 10. That is consistent with N = 10 checks per application. If so, "90% vs 70%" is a gap of two checks out of ten, and the OOP CMS "regression" is two checks. The paper also doesn't say whether the check population is fixed across rounds. If newly discovered exploits become new checks, DSR can fall while every old check still passes — so the round-2 drop might be a denominator artifact, not a bad patch. The paper admits it can't tell these apart. That is the single biggest problem, because the trajectory is the paper's main object.
The guardrail side has the same issue at a different scale: the percentages imply roughly 19–20 harmful targets (10% = 2/20, 25% = 5/20, 45% = 9/20), and benign sets of about 19, 57 and 85 prompts (5.26% = 1/19, 3.51% = 2/57, 7.02% = 4/57, 8.77% = 5/57, 2.35% = 2/85). With n≈20, one target moves DSR by 5 points, and 0→45% is nine targets. There is no confidence interval anywhere in the paper.
-
The guardrail evaluation is circular in two places. Table 6: the attack instantiator and the attack judge are both Qwen3-32B, and the guard classifier and the rule patcher are both GPT-4o mini. First, a jailbreak counts as successful when the same model family that generated it says so. Second, the defence is a classifier whose decisions and whose rule updates come from the same model, so part of the measured gain may be co-adaptation to that classifier rather than a general defensive principle. The paper flags the first ("Attack generation and judging use the same model family") and then doesn't measure it. A second judge from a different family, plus a small human-labelled subset, would tell us how much of 0→45% survives.
-
One run per configuration, no ablations, and cost is hard to read. The code results are single runs — "one representative code record". No seeds, no variance, no CIs. Nothing isolates the structured report, the persistent state, the retry budget or blue's tool access. The paper says this itself ("without component ablations it does not isolate the causal contribution"), which is honest but leaves the mechanism unexplained. On cost: code uses 5.11M tokens for RvB versus 6.25M for the cooperative run (−18.35%), and guardrail tokens grow from 4.04M at baseline to 12.22M at round 4 — the iteration is not cheap — yet there is no wall-clock or budget-matched comparison, so the −18% figure doesn't tell me much.
-
The abstract and the body disagree. "Near 0% False Positive Rates" vs FPR up to 8.77% in Figure 10. "Significantly surpassing baselines" vs a comparison the paper itself calls descriptive. "Robust remediations that are not merely overfitted to specific exploits" vs the paper's own statement that CRDE gains "do not distinguish a general defensive principle from several targeted rules". "90% and 45% DSR" without denominators, with a non-independent judge, and with 55% of harmful targets still jailbroken at the end. A reader who stops at the abstract gets a materially wrong picture.
-
Reproducibility is partial. The anonymous links are there (three anonymous.4open.science repos on page 10), which is good. But there are no model snapshots or decoding settings ("gemini-3.0-flash" with no provider or version; "Qwen3-Max" with no revision), no per-attempt logs, no per-set denominators, and no explanation of the task-grid anomaly.
Questions for authors
What is N, the number of checks per application? Are the ten checks on Pharmacy the same ten in every round? Please give counts (k/n) rather than percentages for TDSR, FDSR and CRDE. Please run the comparison Appendix D describes: score the cooperative final artifact and the RvB final artifact on the same fixed, held-out exploit suite, with at least five seeds and intervals. If the gap goes away, which sentence in the abstract survives? For OOP CMS, is round 2's drop caused by a bad patch, by new checks from newly discovered exploits, or by an unavailable service? A per-check table for rounds 1–5 would settle it. Rerun the guardrail DSR/AAT with a judge from a different model family and report judge–judge agreement. Is the guard classifier (GPT-4o mini) independent of the rule patcher (also GPT-4o mini)? What happens with (a) free-text attack reports instead of the structured report, (b) a state reset (original artifact redeployed each round), (c) a cooperative baseline at matched token/call budget? These would tell us which ingredient matters. How many harmful targets and benign prompts per round were actually used? Are the guardrail token totals per round or cumulative, and what is the wall-clock cost per round? What are task-grid entries 34 and 39, and why do they exceed the 3×10 cap? Do the authors stand by "significantly surpassing baselines" and "near 0% False Positive Rates"? If not, please revise the abstract. Several sentences refer to "the manuscript" in the third person. Were these produced by an AI consistency check and left in? If yes, they should be removed, because they make the claims impossible to attribute.