- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
SafeEvolve extends agent safety beyond policy alignment through a harness–policy co-evolution framework. It iteratively refines external safety prompts and a hierarchical SkillBank from on-policy success and failure trajectories, then internalizes these behaviors via harness-use supervised fine-tuning followed by harness-augmented reinforcement learning. On AgentDojo, SafeEvolve reduced the attack success rate of Qwen3.5-4B from 2.37% to 0.79% while improving utility from 59.79% to 61.86%; on AgentHarm, harmfulness decreased from 56.45 to 12.27.
Critical strengths
- The authors propose SafeEvolve, a framework for continual agent-safety improvement that couples harness evolution with policy optimization through shared on-policy experience.
- SafeEvolve substantially reduces attack success and harmful compliance while preserving, and in some cases improving, benign-task performance.
Critical weaknesses
- Clarity and notation. Several concepts are introduced before being formally defined; for example, component appears before its definition. The notation and component definitions should be reorganized for clarity and consistency.
- Limited model scale. The evaluation is restricted to relatively small models. Experiments on substantially larger models, for example a 27B-scale model, are needed to establish scalability and generalizability.
- Limited model diversity. The evaluated models cover only a small number of model families, and several comparison models are relatively outdated. The evaluation should include stronger and more recent models from additional families.
- Limited benchmark coverage. The experiments rely on only three datasets. Evaluating on one or two additional agent-safety benchmarks, such as ASB, would strengthen the empirical evidence.
- Unclear co-evolution claim. Harness evolution is performed for 10 rounds with the policy frozen, whereas RL subsequently runs for 200 rollout steps. This largely sequential training protocol does not clearly demonstrate genuine harness–policy co-evolution; stronger evidence of iterative mutual adaptation is needed.
- Limited agentic baselines. The comparison with existing agent-safety methods is incomplete. Additional agentic defense baselines, such as StruQ and PromptArmor, should be considered where applicable.
- Figure 1. Figure 1 occupies substantial space while conveying relatively limited information. It should be made more compact or redesigned to better communicate the overall methodology.
Questions for authors
- Potential information leakage. The paper should examine whether safety-related metadata or privileged information encoded in the harness can leak into the policy during training, and whether such leakage contributes to the reported safety improvements.
- Table formatting. Tables should follow standard LaTeX conventions, using
\topruleand\bottomrulefor cleaner presentation. - Equation formatting. Unnumbered equations that are not referenced in the text should use
\begin{align*}...\end{align*}rather than numbered environments. - Unclear distinction from SSA-based RL defenses. The paper should clarify how the proposed RL method differs from SSA-style reinforcement-learning defenses that jointly constrain reasoning-trace safety and final-response safety, such as CRAFT. The RL objective and policy-optimization procedure appear conceptually similar, and the current presentation does not sufficiently establish the methodological novelty.
4 comments
W5. Unclear co-evolution claim In SafeEvolve, harness–policy co-evolution takes the form of alternating updates: the harness evolves with the policy frozen, then the policy is optimized with the harness fixed — a block-coordinate loop formalized in Eq. 13 and Algorithm 1. The paper's experiments run one pass of this loop. Updating the two sides alternately rather than simultaneously is a design choice grounded in Table 7: online harness updates during RL destabilize safety. To directly provide the requested evidence of iterative mutual adaptation, we supplemented a two-cycle alternating experiment with 5 evolution rounds + 100 RL steps, and the full lineage π₀→H₁→π₁→H₂→π₂. Cycle 2 is non-vacuous: reacting to π₁'s residual failure buckets, it accepted 7 new skills that never passed the gate in cycle 1. Results are as follows:
| Lineage | AH harmful | AH refusal | AH benign | Dojo ASR | Dyn ASR | Dyn U |
|---|---|---|---|---|---|---|
| π1 (bare) | 49.0 | 41.5 | 85.2 | 4.91 | 6.37 | 18.3 |
| π1 + H1 | 28.5 | 71.4 | 84.8 | 0.44 | 1.84 | 15.0 |
| π1 + H2 | 26.8 | 75.6 | 85.5 | 0.39 | 1.97 | 15.0 |
| π2 + H2 (end) | 32.4 | 70.9 | 86.6 | 0.17 | 1.55 | 18.3 |
Three findings. (1) The harness contributes large, immediately transferable safety: evaluated on π₁ at unchanged benign utility, it removes 20.5 pp harmful and adds 30 pp refusal — an attribution that is possible precisely because one side is frozen per phase. (2) The second RL cycle consolidates in-distribution safety, driving ASR on both injection suites to lineage lows while restoring AgentDyn utility and reaching the highest benign score. (3) It partially trades out-of-distribution harmful compliance, consistent with Table 7: OOD safety should come from further harness evolution rather than longer RL (Appendix D). Every update in the lineage remains versioned and auditable.
W6. Limited agentic baselines We have added StruQ [1], PromptArmor [2], Tool Result Parsing [3], and Thought-Aligner [4] to the comparison on our main backbone, all under the unified protocol of Table 1. The results show complementary failure modes: structured-separation training solves injection at a large utility cost; training-free purification preserves utility but does not address malicious queries; thought-side correction helps malicious queries but leaves injection unaddressed (AgentDojo ASR 5.93 vs. 2.37 undefended). SafeEvolve is the only method that remains strong on all three axes simultaneously.
| Method | AgentDojo U / ASR | AgentDyn U / ASR | AgentHarm harmful / benign |
|---|---|---|---|
| No defense | 59.79 / 2.37 | 15.00 / 6.88 | 56.45 / 83.09 |
| StruQ | 16.19 / 0.00 | 0.00 / ≤0.56 | 3.90 / 18.10 |
| PromptArmor | 87.43 / 0.00 | 51.67 / 0.46 | 56.70 / 83.60 |
| Tool Result Parsing | 46.09 / 0.76 | 8.33 / 1.18 | 42.10 / 61.70 |
| Thought-Aligner | 38.47 / 5.93 | 10.00 / 5.23 | 46.60 / 78.10 |
| SafeEvolve | 61.86 / 0.79 | 15.00 / 4.51 | 12.27 / 71.31 |
[1] Chen, Sizhe, et al. "{StruQ}: Defending against prompt injection with structured queries." 34th USENIX Security Symposium (USENIX Security 25). 2025.
[2] Shi, Tianneng, et al. "Promptarmor: Simple yet effective prompt injection defenses." arXiv preprint arXiv:2507.15219 (2025).
[3] Yu, Qiang, Xinran Cheng, and Chuanyi Liu. "Defense against indirect prompt injection via tool result parsing." arXiv preprint arXiv:2601.04795 (2026).
[4] Jiang, Changyue, et al. "Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correction." Forty-Third International Conference on Machine Learning, 2026, https://openreview.net/forum?id=x5VjErljHS.
Q1. Potential information leakage We address leakage from the design and the verification angles. Design: the proposer templates forbid adding benchmark-specific identifiers, hidden labels, or ground-truth information; the acceptance panel is held out from the evolution pool; and rewards and acceptance use only environment-visible evidence, never benchmark feedback. Verification: (1) on ASB, unseen during evolution and training, SafeEvolve attains the lowest ASR on all attack families (B.4, Table 8), showing generalization; (2) with the runtime harness removed, the trained policy still improves over the base policy (B.3, Table 6), so part of the safety lives in the parameters; (3) a string audit of all evolved skill banks, safety prompts, and 256 rendered training prompts finds zero occurrences of benchmark vocabulary (suite names, attack names, benchmark-specific terms) anywhere policy-visible. In sum, no benchmark-specific information enters any harness artifact.
Q4. Distinction from SSA-based RL defenses. We agree that both methods use RL-style optimization, but they are distinct in task scenarios and overall pipeline. CRAFT aligns a single reasoning policy by contrasting hidden representations of safe and unsafe traces and adding a latent–textual consistency objective between reasoning and responses. SafeEvolve optimizes a tool-use policy conditioned on an evolved harness: the safety prompt and retrieved skills enter the rollout context, and rewards come from executable, rule-based trajectory verifiers that separately measure task completion, risk success, and tool validity — with no hidden-state access and no reasoning–response consistency term. Beyond the training signal, the outputs differ in kind: SafeEvolve produces an explicit, versioned, auditable harness evolved under held-out regression gates, with policy training as one block of the alternating harness–policy loop, whereas in CRAFT all safety resides in the policy parameters. Finally, the objectives differ: multi-step agent trajectory safety under tool-mediated injection and malicious requests in SafeEvolve, versus single-turn reasoning-trace consistency in CRAFT.
Thanks for the author rebuttal. I still have some concern about the work. 1, W2 I think you should consider some model such as 14b phi4 or 20b gpt-oss 2. W3 same as the w2, and I think you should consider three model families to make sure you have diversity. also, how about your performance with baselines. Q4 I just curious how you fit the SSA issue in agent coevolve problem. But it is not main issue and it not influence my judgement of your paper. W5 I still agree my own judgement and I still consider about that, but thanks for your experiment and explanation. I will increase my score to 2 to encourage your hard work on experimentation in rebuttal period, but I still lean weak reject of paper and this potentially my final judgement.
We thank the reviewer for a careful review. We will respond to each of your concerns and add necessary supplementary content in the revised version.
W1 + W7 + Q2/Q3. Presentation. All four points are fixed together in the revision. (1) Notation: every symbol in Section 3 now has a one-line definition at first use, M(Hk) and cik are defined before first use, and notation is cross-checked with the appendix. (2) Figure 1 will be compacted to a single conceptual panel. (3) Tables are reformatted with booktabs rules throughout. (4) Equations keep their numbers, and every numbered equation will have an explicit in-text reference.
W2. Limited model scale. We agree that the current results do not establish 27B scalability; our reasons are as follows. (1) What is established: the evolved harness transfers to frozen 1.7B/4B/8B policies (Appendix B.1), and the full pipeline runs end-to-end at 8B on a second model family (Llama-3.1-8B-Instruct; see W3) — so the evidence spans 1.7B–8B. (2) Why we do not add 27B here: all experiments in the paper run on 4×H200; full-parameter RL at 27B exceeds this budget, and a shortened configuration would produce a result we would not stand behind. (3) The paradigm itself is scale-agnostic: rewards come from rule-based verifiers independent of model size, the harness enters the rollout as plain context, and each backbone keeps its native tool-call format (Appendix A.5) — moving to larger models changes the engineering budget, not the method. We will list larger-scale evaluation as the immediate next step in the future-work discussion (Appendix D).
W3. Limited model diversity. We have supplemented experiments on Llama-3.1-8B-Instruct with the full SafeEvolve pipeline — the evolved harness and two-stage SFT-RL policy training — evaluated on the same three benchmarks as the main results. 2×2 results are as follows:
Three findings. (1) Swapping the harness on the frozen base does little, the gain mainly come through training. (2) Training under the evolved harness is the main effect: it substantially reduces AgentHarm harmful score while benign utility is preserved. (3) On the trained policy, the evolved harness provides marginal gains.
W4. Limited benchmark coverage. A held-out study on ASB is already presented in Appendix B.4, Table 8. SafeEvolve attains the lowest attack success on all three ASB attack families, whereas policy-only GRPO degrades under the strongest attack (PoT backdoor: 2.50 for SafeEvolve vs. 91.00 for GRPO) — the safety gain is carried by the harness–policy combination, not by the policy alone. On overall coverage: the three main benchmarks span both attack channels studied (indirect prompt injection on AgentDojo/AgentDyn; malicious queries on AgentHarm), and ASB adds a held-out fourth suite with three further attack families — a combined scale consistent with recent agentic-safety evaluations.