acceptodds
Under review as a conference paper at ICLR 2027

Beyond Handcrafted Security: Towards Self-Evolving Defense for LLM Agents

Abstract

Tool-using large language model (LLM) agents encounter attacks through user requests, external observations, persistent memory, and installed skills. Runtime defenses can intervene in these interactions, but revising them after failures typically requires manual diagnosis and editing. We introduce HARD, a framework that uses execution feedback to refine two runtime artifacts: a natural-language security policy and an executable gate over tool calls. A trace router assigns failures to specialized artifact editors while all model parameters remain fixed. Across direct injection, indirect injection, memory contamination, and skill poisoning, HARD achieves attack success rates of 15.4%, 1.0%, 6.7%, and 10.2%, respectively, with benign utility between 91.9% and 95.0%. Attack success rates are 10.2% and 17.8% under two adaptive protocols. Artifact and routing comparisons reveal tradeoffs among security, utility, and artifact size. Together, these findings advance autonomous security engineering for LLM agents. HARD provides a foundation for systematically refining agent safeguards as capabilities, operating environments, and adversarial threats evolve.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Agent, Security, Self-evolving

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ZbEFOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents HARD, a framework for evolving runtime defenses for tool-using LLM agents while keeping model parameters fixed. HARD uses execution failures and evaluator feedback to update a natural-language security policy and an executable tool-call gate, with a trace router assigning failures to specialized editors. The framework is evaluated on four static attack surfaces and two adaptive attack protocols. The paper reports reduced attack success against the selected non-evolving baselines and examines utility, artifact selection, routing, evolution backbones, and deployment costs.

Critical strengths
  1. The paper addresses the important problem of maintaining agent security as attacks evolve, with a modular approach that does not require updating the protected model.

  2. The evaluation covers multiple attack surfaces and separates attack success, benign utility, and utility under attack, providing a useful view of security–utility tradeoffs.

  3. The detailed editing prompts and diagnostic failure cases help explain how policy and gate refinements behave, including their limitations.

Critical weaknesses
  1. The distinction from SHE (Qu et al., 2026a), discussed in Appendix K.3, remains insufficiently established. Both methods decompose a safety harness and route trajectory failures to specialized artifact editors. Please explain what the two-interface formulation enables beyond SHE’s four-component design and provide evidence that this difference yields a substantive methodological benefit, while acknowledging the concurrent nature of the work.

  2. Current baselines are non-evolving, whereas HARD receives five rounds of task-specific failure feedback. This leaves unclear whether its gains arise from the proposed mechanism or additional optimization. Please compare with an evolving defense, such as Membrane (Choi et al., 2026), under matched feedback and budgets, and include Task Shield (Jia et al., 2025) or MELON (Zhu et al., 2025) where applicable.

  3. Section 4.1 and Appendix A.2 state that a separate defense is evolved for each attack category, without artifact transfer. The reported results therefore do not establish that one deployed configuration protects against all four attack surfaces. Please evaluate a shared defense trained on mixed categories, or systematically assess artifact composition, reporting interference, false refusals, and utility changes.

  4. The task-identifier split does not explicitly separate attack families, templates, or environments, leaving generalization beyond related instances unresolved. Please add an attack-family or template-level holdout evaluation and assess transfer to another protected execution model. The current backbone ablation varies the router/evolver rather than the protected model, so it cannot establish cross-model robustness of the deployed defense.

  5. Adaptive split sizes and query budgets are incompletely documented in Appendix F. Please specify attacker models, queries, restarts, maximum turns, feedback access, and knowledge of defense artifacts, and report ASR against attack budget. Both adaptive protocols use direct injection; evaluating adaptive indirect injection or artifact poisoning would clarify whether the reported robustness extends to the other protected attack surfaces.

  6. GLM-5 supplies both evolution feedback and final ASR/UA judgments, creating potential dependence on the evaluator’s preferences and blind spots. Disjoint tasks and repeated grading do not provide an independent outcome standard. Please validate a representative subset using executable checks, human inspection, or judges from different model families, and report disagreement rates and whether the main performance ranking remains unchanged.

  7. Under IPI, HARD reaches 1.0% ASR but only 19.0% utility under attack, compared with 20.9% for the official harness. This leaves unclear whether blocked attacks preserve useful task execution. Please distinguish false refusals, planning failures, execution errors, and successful recovery, and report joint attack prevention and task completion. The 25 separate benign tasks provide limited evidence about false positives on closely related legitimate operations.

  8. The routing comparisons do not match editor calls, tokens, trace exposure, or update order. Moreover, routed and broadcast ASRs under DAE are 12.7% and 13.9%, with a difference small relative to repeat variability. Please include budget-matched random or heuristic routing controls, test alternative update orders, and examine sensitivity to the 0.55 confidence threshold to establish the router’s independent contribution.

  9. Table 4 does not separate costs by task outcome, and evolution cost is unreported. Fewer calls could partly reflect earlier refusal or task termination. Please separate costs for successful legitimate tasks, blocked attacks, and failed tasks, and report evolution overhead. Since latency was measured on different days, qualify timing comparisons and clarify whether Llama Guard’s per-call latency includes its auxiliary classifications.

Questions for authors
  1. With matched failure feedback, editor calls, and token budgets, what benefit does HARD's two-artifact routing provide over an evolving baseline and simple heuristic routing?

  2. Can a single defense evolved on mixed attack categories retain security and utility on held-out attack families and a different protected execution model?

  3. Using executable checks or an independent judge family, what fraction of IPI runs both block the attack and complete the legitimate task, and does the main ranking persist?

Reviewer_tQWsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes HARD, a framework that refines two runtime defense artifacts for a tool-using agent from execution feedback, without updating model parameters: a natural-language security policy inserted into the model context, and an executable gate of tool-name/argument blocking rules. In each of five rounds, the defended agent (DeepSeek-V4-Flash) runs training scenarios; failures (GLM-5 safety score ≤ 0.5) are assigned by an LLM trace router to either a policy evolver or a gate evolver (GLM-5.2), which edit the artifacts in batches of eight. The final artifacts are frozen and evaluated on held-out AgentCanary instances. A separate defense is evolved per attack category. Against three handcrafted OpenClaw plugins and Llama Guard 3-8B, HARD reports the lowest ASR in all six settings: 15.4% (DPI), 1.0% (IPI), 6.7% (memory contamination), and 10.2% (skill poisoning), and 10.2% / 17.8% under two adaptive protocols (DAE, LPA), with benign utility of 91.9–95.0% on 25 PinchBench tasks. The paper also reports routing/artifact comparisons, an evolution-backbone ablation, and deployment cost. The problem is practical and timely, and the paper is unusually candid: Appendices C and F–J explicitly list many of the limitations below. My concern is that, taken together, those limitations leave the paper's two specific claims — that routed, failure-driven evolution is what produces the gains, and that it addresses emerging attacks — without direct support, and that the released material diverges from the paper on several settings that determine the headline numbers.

Critical strengths
  1. Practical problem and a sensible design space. Turning observed failures into defense updates without retraining fits how deployed agents are actually maintained, and separating a semantic policy from an executable gate is a reasonable decomposition.
  2. Consistent improvements across six settings. HARD has the lowest mean ASR in all four static and both adaptive settings, with gaps well beyond the reported repeat SDs (e.g., MC 32.2 → 6.7 vs. Llama Guard; SP 43.2 → 10.2 vs. Shield), and it raises utility under attack on MC (55.9 → 86.3) and SP (51.5 → 92.0).
  3. Lower recurring cost. Per-task execution-agent calls and tokens fall from 10.2 / 273K (official harness) to 4.6 / 116K, while the Llama Guard baseline is the most expensive (27.8 calls, 899K tokens).
  4. Candid limitations. The appendix states that the router's benefit is not identified (G), that the four repeats do not capture evolution variance (C, A.2), that the split separates instances, not attack families (F), that the judge is shared between feedback and evaluation (J), and that the refinement channel itself is an unevaluated attack surface (I). This candor is valuable and should be credited.
Critical weaknesses

W1. The trace router, part of the second contribution, has no demonstrated benefit. On MC, policy-only reaches lower ASR than routed HARD (4.8 ± 1.1 vs. 7.2 ± 3.6, at lower UA). On DAE, routed vs. broadcast (12.7 vs. 13.9) is "small relative to repeat variability." Appendix G states that random routing, heuristic routing, oracle routing, and a monolithic editor with a matched budget are not reported, that router accuracy/calibration is not measured, and that equal rounds do not equalize editor calls, tokens, or trace exposure. The 0.55 confidence cutoff is described as an operational rule. Yet the abstract and §1 present routing as the mechanism. Please either add a random-routing arm and a budget-matched single-editor arm (with router decision logs), or demote the router to an implementation detail and revise the abstract. W2. The baselines do not test the paper's positioning. The comparison covers three handcrafted OpenClaw plugins and Llama Guard as a tool-call classifier. Missing are: (a) published runtime defenses for the same threats — Task Shield, MELON, DRIFT (named in App. G), and Progent/CaMeL — under the same harness; (b) any other evolution method, e.g., GEPA applied to the policy text, or SHE/Membrane, although the paper's claim is about evolution; and (c) simple ablations of iteration itself, e.g., a single-shot policy/gate written by the same GLM-5.2 from all training failures at once, or an equal-effort handcrafted policy and gate. Without (b) and (c), it is unclear whether the gains come from iterative, failure-driven evolution or simply from adding an LLM-written policy and rule set tuned to the training split. W3. The evaluation is in-distribution and partly non-independent, so "emerging attacks" is not tested. The research question asks how defenses adapt to emerging attacks. However, each category gets its own defense trained and tested on instances of the same category; App. F states the split "does not explicitly hold out strategies, attack families, templates, or environments." Two further points:

  • In the released splits, the LPA train/test pools are exactly the DPI train/test pools with task IDs shifted by 42000 (82/82 and 83/83 match). LPA therefore reuses the DPI tasks as a multi-turn protocol; it is not an independent task distribution, and DPI and LPA should not be read as two separate generalization results.
  • The adaptive attacks are black-box with respect to the frozen artifacts, and their budgets and split sizes are "incompletely documented" (App. F). An attacker who can read the policy and gate rules — realistic for a deployed plugin — is not evaluated, and App. B already shows a gate bypass via variable-based command reconstruction. A cross-category transfer experiment (evolve on some categories, test on a held-out one) and a held-out attack-template split would directly test the "emerging" claim. A sequential version would test the "self-evolving" framing more directly: evolve one defense through stages (e.g., DPI → IPI → MC → SP), and after each stage report ASR on every category. This matrix would show adaptation to the new category, transfer to unseen ones, and forgetting of earlier ones. Five offline rounds on a fixed category followed by freezing is better described as offline failure-driven defense optimization than as continual self-evolution, which App. F and G partly acknowledge. W4. Measurement validity: shared judge, single evolution run, and coarse utility.
  • GLM-5 provides both the evolution feedback and the final ASR/UA judgments (App. J), so improvements may partly reflect fitting the judge's criteria. No independent judge, executable effect check, or human audit of a trace sample is reported.
  • Each configuration is evolved once, and the ±SD in Tables 1–2 measures re-evaluation of one frozen artifact, not evolution variance. MC is reported as 6.7 (Table 1), 3.9 (Fig. 3), 7.2 (Fig. 4), and 4.3 (Table 3), and App. F says the run records needed to reconcile these are unavailable. The backbone ablation (MC ASR ranging 4.3–15.9 across editors) suggests evolution-to-evolution variance may be large. At least three independent evolution runs on two or three settings are needed.
  • Benign utility uses 25 PinchBench tasks, a small and possibly insensitive probe for a defense that blocks tool calls. On IPI, ASR falls to 1.0% while UA is 19.0% (vs. 20.9% for the official harness), and calls per task drop from 10.2 to 4.6. The paper cannot currently distinguish "blocked the attack and completed the task" from "stopped early." Please report the three-way breakdown (blocked + completed, blocked + not completed, not blocked) and a false-positive rate on benign tasks that involve write operations.
  • No acceptance test, so regressions are possible and unmeasured. §3.2 states that updates are applied "without an independent regression-based acceptance test." The MC curve does regress (11.5% after round 1 → 21.1% after round 2), and App. H notes that later policy edits may erase earlier behavior. App. I proposes staged validation with rollback but did not use it for the reported results. For a security mechanism, accepting a candidate only when validation ASR does not rise beyond a tolerance (and utility does not drop beyond one), with rollback otherwise, seems essential. An ablation with and without such a validator — reporting acceptance rate, final ASR/BU/UA, and regression on earlier failures — would show whether the current results depend on lucky trajectories.
  • One protected model. All results protect DeepSeek-V4-Flash; the backbone ablation varies the editor, not the protected agent. A second execution model (e.g., one closed and one open model) would show whether the evolved artifacts, or the procedure, carry over.
  • Total cost is not comparable. Deployment cost per task is lower, but evolution cost was not logged (App. D). Without it, readers cannot tell after how many deployed tasks the one-time evolution cost is recovered. W5. The supplementary material diverges from the paper on settings that determine the headline numbers. I inspected the package read-only and found:
  • Evolver temperature. The paper states that routing and evolution decode at temperature 0 (§4.1, App. A.1). In self_evo_defense/evolve_baseline.py, the evolver call sets temperature = 0.3 for every non-GPT-5 model, which includes the main-configuration GLM-5.2 path (around line 2244).
  • Judge identity. The paper names GLM-5 as the safety and UA judge. The released dev.md (lines 12, 210) records judge=opus-4-6, and evolve.py's header says the pipeline measures safety/utility with an Opus-4.6 judge. Please state which judge produced Tables 1–3.
  • Undisclosed task exclusion. All three single-suite split files mark resource_exhaustion QUARANTINED. This removes 35 DPI, 26 IPI, and 1 MC task from both train and test, and through the ID mapping above also shrinks the LPA pool. The paper's split description (App. A.2) and the rest of the paper never mention this. Resource exhaustion is a category where a tool-call gate is directly relevant, so the exclusion changes the scope of the four-surface claim and should be disclosed with its rationale.
  • Missing inputs. All 609 task_*.md files are 0 bytes; tasks/skills_poison/ is empty and no SP split is provided, although Table 1 reports SP results on an 80/81 split; both evolver prompt templates (policy_evolver_prompt.md, gate_evolver_prompt.md) are 0 bytes (App. A.3 does print the prompt text); and no Llama Guard integration appears in the package although it is the strongest baseline in five of six settings. Evolution cost was also not logged (App. D). I cannot confirm that the released code is the version used for the experiments. As released, however, no headline number can be reproduced, and the temperature and judge discrepancies bear directly on the reported results. Questions
Questions for authors
  1. How do random routing and a budget-matched single editor compare with the trace router, and can you release router decision logs?
  2. How does HARD compare with Task Shield, MELON, DRIFT, and at least one other evolution method (e.g., GEPA on the policy, or SHE) under the same harness, and with a single-shot (non-iterative) policy/gate written from the same training failures?
  3. Which judge produced Tables 1–3, and which temperature did the evolver actually use (0 as in the paper, or 0.3 as in the code)?
  4. Why were all resource_exhaustion tasks quarantined, and how do results change if they are included?
  5. Can you report a cross-category transfer experiment, and at least three independent evolution runs on two or three settings?
  6. Can you release the task definitions, the SP split, the evolver templates, and the Llama Guard integration, and report the three-way blocked/completed breakdown for IPI?
  7. How does HARD behave with a validation-and-rollback step before accepting an update, and in a sequential multi-category evolution where earlier categories are re-tested after each stage?
Reviewer_otkJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method for automatically improving online defenses for LLM agents. It works by collecting successful attack trajectories and routes them to specialized editors that update natural-language security polics. The updated policies are deployed in subsequent evolution rounds and frozen before deployment. Experiments on different attack scenarios demostrate the effectiveness of the proposed method.

Critical strengths
  • The studied problem is interesting.

  • The experiments includes multiple attack surfaces and adaptive attacks.

  • This paper is well written.

Critical weaknesses
  • The contributions of the routing mechanism is not sufficiently established. Policy-only achieves a lower attack success rate than routed refinement on memory contamination. Comparisons with matched cost (e.g., tokens and number of llm calls) with simpler routing strategies would help show whether the proposed routing mechanism provides additional benefits.

  • The adaptive attack evaluation has limited coverage, and the attack strength might be difficult to assess. The used adaptive attacks operate through direct user requests, leaving adaptive attack beyond it unexplored, e.g., indirect injection, and memory/skill poisoning.

  • The novelty of this paper might need clearer justification relative to existing methods. The core procedure uses an LLM to analyze failures and revise external security policies or blocking rules. Although the paper discusses related methods such as Membrane and SESG, the experiments mainly compare against non-evolving defenses. Comparisons with representative existing evolving defenses would be helpful.

Questions for authors

see above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.