acceptodds
Desk rejected at ICLR 2027

SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignment

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

16 views

Abstract

The performance of LLM-based agents is jointly shaped by the base model and the harness used when interacting with the environment. This exposes them to safety risks in both harmful final responses and multi-step execution trajectories. Existing safety alignment mechanisms often rely on either external harness updates or policy optimization, yet applying either paradigm in isolation fails to bridge runtime control with intrinsic safety. We propose SafeEvolve, an experience-driven self-evolving framework for agent safety alignment. SafeEvolve leverages safety experience from completed on-policy trajectories to drive a continual loop of harness-policy co-evolution. On the harness side, SafeEvolve converts trajectory-level safety evidence into bounded, component-level updates across safety rules and hierarchical skills, yielding auditable and reversible harness artifacts. On the policy side, SafeEvolve follows a two-stage SFT-RL paradigm, where harness-use SFT bootstraps the policy to actively leverage evolved harness artifacts, and harness-augmented RL further shapes autonomous safety behaviors during multi-step exploration via verifier-decomposed rewards. Through harness-policy co-evolution, SafeEvolve converts safety experience into an evolved runtime harness and improved policy behavior. Experiments on agentic safety benchmarks show that SafeEvolve achieves a stronger safety-utility tradeoff than existing baselines. For Qwen3.5-4B, SafeEvolve achieves a ASR reduction on AgentDojo while improving benign utility from 59.79% to 61.86%.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Agent Safety, Harness-Policy Co-Evolution, Experience-Driven Self-Improvement

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hfVtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

SafeEvolve extends agent safety beyond policy alignment through a harness–policy co-evolution framework. It iteratively refines external safety prompts and a hierarchical SkillBank from on-policy success and failure trajectories, then internalizes these behaviors via harness-use supervised fine-tuning followed by harness-augmented reinforcement learning. On AgentDojo, SafeEvolve reduced the attack success rate of Qwen3.5-4B from 2.37% to 0.79% while improving utility from 59.79% to 61.86%; on AgentHarm, harmfulness decreased from 56.45 to 12.27.

Critical strengths
  • The authors propose SafeEvolve, a framework for continual agent-safety improvement that couples harness evolution with policy optimization through shared on-policy experience.
  • SafeEvolve substantially reduces attack success and harmful compliance while preserving, and in some cases improving, benign-task performance.
Critical weaknesses
  • Clarity and notation. Several concepts are introduced before being formally defined; for example, component cc appears before its definition. The notation and component definitions should be reorganized for clarity and consistency.
  • Limited model scale. The evaluation is restricted to relatively small models. Experiments on substantially larger models, for example a 27B-scale model, are needed to establish scalability and generalizability.
  • Limited model diversity. The evaluated models cover only a small number of model families, and several comparison models are relatively outdated. The evaluation should include stronger and more recent models from additional families.
  • Limited benchmark coverage. The experiments rely on only three datasets. Evaluating on one or two additional agent-safety benchmarks, such as ASB, would strengthen the empirical evidence.
  • Unclear co-evolution claim. Harness evolution is performed for 10 rounds with the policy frozen, whereas RL subsequently runs for 200 rollout steps. This largely sequential training protocol does not clearly demonstrate genuine harness–policy co-evolution; stronger evidence of iterative mutual adaptation is needed.
  • Limited agentic baselines. The comparison with existing agent-safety methods is incomplete. Additional agentic defense baselines, such as StruQ and PromptArmor, should be considered where applicable.
  • Figure 1. Figure 1 occupies substantial space while conveying relatively limited information. It should be made more compact or redesigned to better communicate the overall methodology.
Questions for authors
  • Potential information leakage. The paper should examine whether safety-related metadata or privileged information encoded in the harness can leak into the policy during training, and whether such leakage contributes to the reported safety improvements.
  • Table formatting. Tables should follow standard LaTeX conventions, using \toprule and \bottomrule for cleaner presentation.
  • Equation formatting. Unnumbered equations that are not referenced in the text should use \begin{align*}...\end{align*} rather than numbered environments.
  • Unclear distinction from SSA-based RL defenses. The paper should clarify how the proposed RL method differs from SSA-style reinforcement-learning defenses that jointly constrain reasoning-trace safety and final-response safety, such as CRAFT. The RL objective and policy-optimization procedure appear conceptually similar, and the current presentation does not sufficiently establish the methodological novelty.
4 comments

We thank the reviewer for a careful review. We will respond to each of your concerns and add necessary supplementary content in the revised version.

W1 + W7 + Q2/Q3. Presentation. All four points are fixed together in the revision. (1) Notation: every symbol in Section 3 now has a one-line definition at first use, M(Hk)\mathcal{M}(\mathcal{H}^k) and cikc_i^k are defined before first use, and notation is cross-checked with the appendix. (2) Figure 1 will be compacted to a single conceptual panel. (3) Tables are reformatted with booktabs rules throughout. (4) Equations keep their numbers, and every numbered equation will have an explicit in-text reference.

W2. Limited model scale. We agree that the current results do not establish 27B scalability; our reasons are as follows. (1) What is established: the evolved harness transfers to frozen 1.7B/4B/8B policies (Appendix B.1), and the full pipeline runs end-to-end at 8B on a second model family (Llama-3.1-8B-Instruct; see W3) — so the evidence spans 1.7B–8B. (2) Why we do not add 27B here: all experiments in the paper run on 4×H200; full-parameter RL at 27B exceeds this budget, and a shortened configuration would produce a result we would not stand behind. (3) The paradigm itself is scale-agnostic: rewards come from rule-based verifiers independent of model size, the harness enters the rollout as plain context, and each backbone keeps its native tool-call format (Appendix A.5) — moving to larger models changes the engineering budget, not the method. We will list larger-scale evaluation as the immediate next step in the future-work discussion (Appendix D).

W3. Limited model diversity. We have supplemented experiments on Llama-3.1-8B-Instruct with the full SafeEvolve pipeline — the evolved harness and two-stage SFT-RL policy training — evaluated on the same three benchmarks as the main results. 2×2 results are as follows:

CellAH harmfulAH refusalAH benignDojo ASRDyn ASR
Base + seed26.8844.3245.671.631.74
Base + evolved27.4044.8944.971.531.79
Trained + seed17.0058.0042.001.202.50
Trained + evolved15.0060.0042.000.802.00

Three findings. (1) Swapping the harness on the frozen base does little, the gain mainly come through training. (2) Training under the evolved harness is the main effect: it substantially reduces AgentHarm harmful score while benign utility is preserved. (3) On the trained policy, the evolved harness provides marginal gains.

W4. Limited benchmark coverage. A held-out study on ASB is already presented in Appendix B.4, Table 8. SafeEvolve attains the lowest attack success on all three ASB attack families, whereas policy-only GRPO degrades under the strongest attack (PoT backdoor: 2.50 for SafeEvolve vs. 91.00 for GRPO) — the safety gain is carried by the harness–policy combination, not by the policy alone. On overall coverage: the three main benchmarks span both attack channels studied (indirect prompt injection on AgentDojo/AgentDyn; malicious queries on AgentHarm), and ASB adds a held-out fourth suite with three further attack families — a combined scale consistent with recent agentic-safety evaluations.

W5. Unclear co-evolution claim In SafeEvolve, harness–policy co-evolution takes the form of alternating updates: the harness evolves with the policy frozen, then the policy is optimized with the harness fixed — a block-coordinate loop formalized in Eq. 13 and Algorithm 1. The paper's experiments run one pass of this loop. Updating the two sides alternately rather than simultaneously is a design choice grounded in Table 7: online harness updates during RL destabilize safety. To directly provide the requested evidence of iterative mutual adaptation, we supplemented a two-cycle alternating experiment with 5 evolution rounds + 100 RL steps, and the full lineage π₀→H₁→π₁→H₂→π₂. Cycle 2 is non-vacuous: reacting to π₁'s residual failure buckets, it accepted 7 new skills that never passed the gate in cycle 1. Results are as follows:

LineageAH harmfulAH refusalAH benignDojo ASRDyn ASRDyn U
π1 (bare)49.041.585.24.916.3718.3
π1 + H128.571.484.80.441.8415.0
π1 + H226.875.685.50.391.9715.0
π2 + H2 (end)32.470.986.60.171.5518.3

Three findings. (1) The harness contributes large, immediately transferable safety: evaluated on π₁ at unchanged benign utility, it removes 20.5 pp harmful and adds 30 pp refusal — an attribution that is possible precisely because one side is frozen per phase. (2) The second RL cycle consolidates in-distribution safety, driving ASR on both injection suites to lineage lows while restoring AgentDyn utility and reaching the highest benign score. (3) It partially trades out-of-distribution harmful compliance, consistent with Table 7: OOD safety should come from further harness evolution rather than longer RL (Appendix D). Every update in the lineage remains versioned and auditable.

W6. Limited agentic baselines We have added StruQ [1], PromptArmor [2], Tool Result Parsing [3], and Thought-Aligner [4] to the comparison on our main backbone, all under the unified protocol of Table 1. The results show complementary failure modes: structured-separation training solves injection at a large utility cost; training-free purification preserves utility but does not address malicious queries; thought-side correction helps malicious queries but leaves injection unaddressed (AgentDojo ASR 5.93 vs. 2.37 undefended). SafeEvolve is the only method that remains strong on all three axes simultaneously.

MethodAgentDojo U / ASRAgentDyn U / ASRAgentHarm harmful / benign
No defense59.79 / 2.3715.00 / 6.8856.45 / 83.09
StruQ16.19 / 0.000.00 / ≤0.563.90 / 18.10
PromptArmor87.43 / 0.0051.67 / 0.4656.70 / 83.60
Tool Result Parsing46.09 / 0.768.33 / 1.1842.10 / 61.70
Thought-Aligner38.47 / 5.9310.00 / 5.2346.60 / 78.10
SafeEvolve61.86 / 0.7915.00 / 4.5112.27 / 71.31

[1] Chen, Sizhe, et al. "{StruQ}: Defending against prompt injection with structured queries." 34th USENIX Security Symposium (USENIX Security 25). 2025.

[2] Shi, Tianneng, et al. "Promptarmor: Simple yet effective prompt injection defenses." arXiv preprint arXiv:2507.15219 (2025).

[3] Yu, Qiang, Xinran Cheng, and Chuanyi Liu. "Defense against indirect prompt injection via tool result parsing." arXiv preprint arXiv:2601.04795 (2026).

[4] Jiang, Changyue, et al. "Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correction." Forty-Third International Conference on Machine Learning, 2026, https://openreview.net/forum?id=x5VjErljHS.

Q1. Potential information leakage We address leakage from the design and the verification angles. Design: the proposer templates forbid adding benchmark-specific identifiers, hidden labels, or ground-truth information; the acceptance panel is held out from the evolution pool; and rewards and acceptance use only environment-visible evidence, never benchmark feedback. Verification: (1) on ASB, unseen during evolution and training, SafeEvolve attains the lowest ASR on all attack families (B.4, Table 8), showing generalization; (2) with the runtime harness removed, the trained policy still improves over the base policy (B.3, Table 6), so part of the safety lives in the parameters; (3) a string audit of all evolved skill banks, safety prompts, and 256 rendered training prompts finds zero occurrences of benchmark vocabulary (suite names, attack names, benchmark-specific terms) anywhere policy-visible. In sum, no benchmark-specific information enters any harness artifact.

Q4. Distinction from SSA-based RL defenses. We agree that both methods use RL-style optimization, but they are distinct in task scenarios and overall pipeline. CRAFT aligns a single reasoning policy by contrasting hidden representations of safe and unsafe traces and adding a latent–textual consistency objective between reasoning and responses. SafeEvolve optimizes a tool-use policy conditioned on an evolved harness: the safety prompt and retrieved skills enter the rollout context, and rewards come from executable, rule-based trajectory verifiers that separately measure task completion, risk success, and tool validity — with no hidden-state access and no reasoning–response consistency term. Beyond the training signal, the outputs differ in kind: SafeEvolve produces an explicit, versioned, auditable harness evolved under held-out regression gates, with policy training as one block of the alternating harness–policy loop, whereas in CRAFT all safety resides in the policy parameters. Finally, the objectives differ: multi-step agent trajectory safety under tool-mediated injection and malicious requests in SafeEvolve, versus single-turn reasoning-trace consistency in CRAFT.

Thanks for the author rebuttal. I still have some concern about the work. 1, W2 I think you should consider some model such as 14b phi4 or 20b gpt-oss 2. W3 same as the w2, and I think you should consider three model families to make sure you have diversity. also, how about your performance with baselines. Q4 I just curious how you fit the SSA issue in agent coevolve problem. But it is not main issue and it not influence my judgement of your paper. W5 I still agree my own judgement and I still consider about that, but thanks for your experiment and explanation. I will increase my score to 2 to encourage your hard work on experimentation in rebuttal period, but I still lean weak reject of paper and this potentially my final judgement.

Reviewer_jSAMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes SafeEvolve, a framework for improving the safety of LLM agents by evolving both the harness and the policy. On the harness side, failures observed in agent rollouts are used to propose and validate small edits to the harness. On the policy side, the model is trained with SFT followed by RL under the evolved harness, using a reward that combines safety and utility. Experiments on agentic safety benchmarks show improved safety over several training and safety alignment baselines.

Critical strengths

The paper studies an interesting direction for agent safety by considering the harness and the policy together rather than in isolation. The motivation is reasonable, as harness-only methods may provide guidance that a weak policy cannot reliably follow, while policy-only training under a fixed harness may not adapt to new risks, and the paper provides some evidence for both observations.

Critical weaknesses
  1. The presentation of the paper is not clear, as many notations are either not explained or only loosely explained through other terminology without concrete definitions, such as τH\tau_H, NN, bib_i, mim_i, Apply(⋅)\mathrm{Apply}(\cdot), G(⋅,⋅)G(\cdot,\cdot), Pref(⋅,⋅)\mathrm{Pref}(\cdot,\cdot), Retrieve(⋅)\mathrm{Retrieve}(\cdot), SgS_g, SkS_k, SmS_m, A(τ)A(\tau), and ww. In addition, several steps of the proposed method require further clarification, as detailed in the questions below.
  2. The fairness of the experimental comparisons is unclear. The authors state that they “compare SafeEvolve with two types of alignment methods on the same tasks.” However, harness evolution and SFT-RL are conducted in an internally constructed simulator, while the benchmarks form separate evaluation environments covering “similar scenarios and safety risks”, which raises concerns about the fairness of the comparison, particularly with respect to the safety-alignment baselines MetaSecAlign and AgentAlign.
Questions for authors
  1. How is the initial harness H0\mathcal{H}^0 constructed? The paper states that it is initialized by distilling established defense principles into a seed safety prompt and skillbank, but the distillation procedure is unclear. What are the sources of these principles, and how are they converted into the initial prompt and skills?
  2. What is the harness evolution proposer used to select the target component cjkc_j^k and generate the mutation Δjk\Delta_j^k? How is the proposer implemented, and what specifically makes a mutation “bounded”?
  3. SafeEvolve accepts a candidate harness only when it improves over the parent and satisfies the acceptance criteria (section 3.2). How many candidate mutations are evaluated at each evolution step, and what happens if a candidate is rejected? Does the proposer continue generating new candidates until one is accepted, and if so, how is the process prevented from stalling after repeated rejections?
  4. How are the rule-based verifiers (section 3.3) defined? In particular, what criteria are used to determine task completion, risk success, and tool validity from a trajectory, and are these criteria manually specified for each task or generated automatically?
  5. How exactly is a skill defined? Section 4.4 evaluates different types of skills, but the paper does not clearly explain the distinction between general, task-specific, and common-mistake skills beyond the examples in Appendix C.
5 comments

We thank the reviewer for a careful review. We will respond to each of your concerns and add necessary supplementary content in the revised version.

W1. Unclear/loosely defined notation. We fully acknowledge your concern and have fixed it in place: every symbol in Section 3 now carries a one-line definition at its first occurrence, directly at the corresponding equations; the component map M(Hk)\mathcal{M}(\mathcal{H}^k) and components cikc_i^k are defined before first use in Section 3.2; and notation has been cross-checked between Section 3 and the appendix.

W2. Fairness of the experimental comparison. This is a fair concern; we clarify the comparison structure as follows. The comparison has two layers, both under the same protocol (same benchmarks and evaluation metrics; all baselines share the seed harness):

  • Layer 1: SFT, DPO, and GRPO are trained in the same simulator with the same rule-based verifiers as SafeEvolve (Appendix A.2); GRPO uses the same verifier-decomposed reward without the evolved harness, isolating the effect of harness evolution.

  • Layer 2: MetaSecAlign and AgentAlign are trained on our backbone following their official implementations, and all methods are evaluated under each benchmark's native protocol.

On wording, "compare … on the same tasks" (Section 4.1) referred to the unified evaluation, not the training environment; the phrase has been replaced with "on these benchmarks." For proximity between the simulator and the benchmarks, we audited all simulator tasks against all benchmark tasks and found no task-level or template-level overlap, and benchmark tasks never enter training or evolution. The held-out ASB study provides additional out-of-distribution evidence.

Q1. How is the initial harness H0 constructed?. H0 = seed safety prompt + seed skillbank. We clarify its construction pipeline here. We will add the analyzer prompt and the full procedure to Appendix C.

(1) Sources of the principles. Only general defenses for the main threat channel: instruction hierarchy and trusted/untrusted separation (observed content is evidence, never instructions) against indirect prompt injection, and refusal without tool assistance against malicious queries. We deliberately exclude attack-specific or benchmark-specific mechanisms. The seed harness provides a safety floor, and harness evolution performs refinement.

(2) Distillation in three steps. (i) Aggregation : verifier-labeled base-policy rollouts are grouped by failure signature and scope. (ii) Analysis: an analyzer — the same external model as the proposer, invoked with a different prompt — attributes the collected principles and the grouped evidence into the seed artifacts: each principle is instantiated in the fixed skill schema, generalized away from specific attacks and domains, and condensed to the retrieval budget; recurring failure groups become common-mistake skills; the three cross-cutting principles form the seed safety prompt. (iii) Curation: an entry is kept only if its failures recur and its scope stays general, yielding the 26-entry seed bank (3 general / 14 task-specific / 9 common-mistake). Appendix C shows the seed prompt and a representative skill.

Q2. What is the proposer, and what makes a mutation "bounded"? Implementation. Harness evolution invokes an external LLM proposer through two structured prompt templates, one per component; each call receives rollout evidence and returns a single structured edit. The main experiments use GPT-5.5 as the proposer (now stated in Section 4.1), and the choice is not critical: Appendix B.1 shows that DeepSeek-Chat and GLM-5.1 evolve harnesses with comparable safety–utility gains. "Bounded" is enforced by four mechanisms: (1) single-component scope — the proposer may edit only the target component, never tools, task data, or verifier and reward logic; (2) local edit semantics — a mutation adds, updates, or merges one artifact, never a thorough rewrite; (3) regression budgets — a hard gate bounds regressions on a held-out panel; (4) text checks — benchmark-specific terms and unconditional-continuation clauses are forbidden. A candidate failing any condition is rejected and the parent retained (Eq. 7).

Q3. Candidates per step and behavior after rejection On rejection, the last accepted harness is retained: every candidate is compared against its parent, so evolution is monotone and safety never regresses within it (Eq. 7). One candidate is proposed per step and judged as a whole, and the proposer never retries until acceptance — rejection advances to the next step, where a new proposal is made from freshly aggregated evidence; previously rejected edits can reappear and be accepted once the evidence changes. The loop is bounded by a maximum number of steps (10 rounds in the main experiments), so it cannot stall. Empirically, across 5 runs × 10 rounds (50 proposals, 19 accepted), every rejection streak recovered within the step budget, the single unsafe candidate observed was blocked by the hard gate, and rejected skills re-proposed later were eventually accepted. In the reported run, 5 of 10 updates were accepted, growing the skillbank from 26 to 47 entries (Appendix C, Figure 4); per-round statistics will be added to the appendix.

Q4. How are rule-based verifiers defined? The verifiers are native to the task suite: each task specification carries its own verifier metadata and is generated together with its simulator, and policy training and harness acceptance share the same verifiers. The criteria are therefore generated automatically, not hand-specified per task. Each verifier inspects the trajectory, tool calls, tool outputs, and final answer, combining outcome-level and process-level checks that instantiate the reward primitives of Section 3.3: task completion — whether requested facts are grounded and the request completed, scored as U(τ)∈[0,1]U(\tau)\in[0,1] with partial credit; risk success — whether injected instructions were followed or a harmful request was made more actionable, with the safety score S(τ)S(\tau) as its complement; and tool validity — whether tool calls are valid across turns, with invalid calls and parse errors penalized (Table 4).

Q5. How exactly is a skill defined? A skill is a structured unit (name / type / principle / when-to-apply / workflow / avoid) of one of three types, which differ in when they fire and what they encode: general skills (SgS_g) are task-agnostic principles, always retrievable; task-specific skills (SkS_k) are procedures scoped by episode metadata (domain, tool family, scenario, task type), retrieved on scope match; common-mistake skills (SmS_m) are repairs keyed to failure signatures (e.g., ambiguous identifiers, missing arguments, no-progress loops, detected injection), retrieved on trigger match. At rollout time, Eq. 8 retrieves at most one skill per type within a fixed character budget, prioritizing dynamically evolved skills. Each design choice is validated by the retrieval ablation (Table 3): general-only lowers ASR at a utility cost, and removing dynamic-skill priority raises harmful compliance. Complete examples are in Appendix C, and formal definitions will be added to Section 3.3.

Dear Reviewer jSAM,

We sincerely appreciate your time and effort in reviewing our paper and providing valuable feedback. We have carefully addressed your concerns in our rebuttal and uploaded the revised version.

As the discussion phase is approaching its end, we gently request that you review our responses and let us know if our clarifications have adequately addressed your concerns. We remain happy to provide any further clarification if needed.

Thank you again for your valuable contribution to our work.

Sincerely, The Authors

Reviewer_5GjGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a study on harness-policy co-evolution from agent experience for safety alignment. Specifically, the authors have pointed out that existing safety alignment mechanisms often rely on either external harness updates or policy optimization but applying either paradigm in isolation fails to bridge run-time control with intrinsic safety. To address this point, the authors have proposed SafeEvolve as an experience-driven self-evolving framework for agent safety alignment. The experimental results indicate that SafeEvolve achieves a stronger safety-utility tradeoff than existing baselines.

Critical strengths
  1. The authors have proposed SafeEvolve as an experience-driven self-evolving framework for agent safety alignment.
  2. SafeEvolve enables continual safety improvements through two interleaved processes.
  3. The experimental results show that SafeEvolve achieves a stronger safety-utility tradeoff against risks such as environment injection and malicious query.
Critical weaknesses
  1. SafeEvolve has been compared with some baseline methods published in 2024 or 2025 but this paper would be strengthened further by adding one or two methods published in 2026 for baseline comparisons.
  2. The authors have evaluated how effectively SafeEvolve can achieve safety-utility tradeoff on Qwen models but it is also worth to extend the evaluation on other large language models to demonstrate the capability of SafeEvolve more comprehensively.
  3. Section 5 lacks a suggestion of future work on harness-policy co-evolution for safety alignment.
Questions for authors

Please address the points mentioned in the list of weaknesses.

3 comments

We thank the reviewer for the positive assessment and the constructive suggestions. We will respond to each of your concerns and add necessary supplementary content in the revised version.

W1. Baselines from 2026. We add four recent defenses on our main backbone (Qwen3.5-4B), all evaluated under the same protocol as Table 1 — Tool Result Parsing [1] and Thought-Aligner [2], the two 2026 methods as requested, and StruQ [3] and PromptArmor [4]. Results are as follows:

MethodAgentDojo U / ASRAgentDyn U / ASRAgentHarm harmful / benign
No defense59.79 / 2.3715.00 / 6.8856.45 / 83.09
StruQ16.19 / 0.000.00 / 0.563.90 / 18.10
PromptArmor87.43 / 0.0051.67 / 0.4656.70 / 83.60
Tool Result Parsing46.09 / 0.768.33 / 1.1842.10 / 61.70
Thought-Aligner38.47 / 5.9310.00 / 5.2346.60 / 78.10
SafeEvolve61.86 / 0.7915.00 / 4.5112.27 / 71.31

Each defense improves one axis at the cost of another. StruQ reduces injection ASR to zero but collapses task utility (AgentDyn utility 0.00; AgentHarm benign 18.10). PromptArmor preserves utility but leaves malicious-query compliance essentially unchanged. Tool Result Parsing reaches its 0.76% ASR only at a 7–21 pp utility cost, but with no malicious-query defense. Thought-Aligner is the mirror image: the best malicious-query trade among the defenses (harmful 56.45→46.60, refusal +9.6 pp, benign −4.99 pp) but no injection defense — AgentDojo ASR even rises to 5.93. SafeEvolve is the only method that is simultaneously strong on malicious-query safety (harmful 12.27), benign utility (71.31), and injection robustness (ASR 0.79).

[1] Yu, Qiang, Xinran Cheng, and Chuanyi Liu. "Defense against indirect prompt injection via tool result parsing." arXiv preprint arXiv:2601.04795 (2026).

[2] Jiang, Changyue, et al. "Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correction." Forty-Third International Conference on Machine Learning, 2026, https://openreview.net/forum?id=x5VjErljHS.

[3] Chen, Sizhe, et al. "{StruQ}: Defending against prompt injection with structured queries." 34th USENIX Security Symposium (USENIX Security 25). 2025.

[4] Shi, Tianneng, et al. "Promptarmor: Simple yet effective prompt injection defenses." arXiv preprint arXiv:2507.15219 (2025).

W2. Evaluation beyond Qwen models. We agree and cover an experiment of the full SFT-RL pipeline on Llama-3.1-8B-Instruct. Results are as follows.

CellAH harmfulAH refusalAH benignDojo ASRDyn ASR
Base + seed26.8844.3245.671.631.74
Base + evolved27.4044.8944.971.531.79
Trained + seed17.0058.0042.001.202.50
Trained + evolved15.0060.0042.000.802.00

Three findings. (1) Swapping the harness on the frozen base does little, the gain mainly come through training. (2) Training under the evolved harness is the main effect: it substantially reduces AgentHarm harmful score while benign utility is preserved. (3) On the trained policy, the evolved harness still provides marginal gains.

W3. Future work on harness-policy co-evolution in Section 5. Future work on harness–policy co-evolution is already discussed in Appendix D, covering multi-timescale meta-evolution and safety-controlled evolution for open-ended tasks. A meta-controller could coordinate a fast harness loop that externalizes emerging risks into auditable rules with a slower policy loop that internalizes recurrent safety behaviors, replacing the current first-order, fixed-schedule recursion. Open-ended evolution — co-evolving red-team adversaries, safety curricula, and defenses — must stay bounded by non-evolvable safety anchors, independent verifiers, and regression budgets against evaluator drift and reward hacking (cf. Table 7), with sandboxed updates and rollback. We have added a summary sentence on future work at the end of Section 5 (Conclusion).

Dear Reviewer 5GjG,

We sincerely appreciate your time and effort in reviewing our paper and providing valuable feedback. We have carefully addressed your concerns in our rebuttal and uploaded the revised version.

As the discussion phase is approaching its end, we gently request that you review our responses and let us know if our clarifications have adequately addressed your concerns. We remain happy to provide any further clarification if needed.

Thank you again for your valuable contribution to our work.

Sincerely, The Authors

Thank you for the response. The clarification and the added results have addressed my comments so I suggest the authors to incorporate these results into the paper revision. My original score already reflects the good merit of this paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.