acceptodds
Under review as a conference paper at ICLR 2027

When Agents Don't Take "No”: Curbing Guardrail Evasion while Keeping Utility

25 views

Abstract

Blocking an unsafe tool call does not necessarily block its harmful effect, as an agent can retry or reroute until subsequent actions produce the same outcome, even without an external attacker or explicit instructions to bypass the guardrail. We call this behavior guardrail evasion. Evaluating four existing guardrails on two agent-security benchmarks, RedCode and SABER, we find that every guardrail either allows risky actions to proceed or hinders legitimate task execution. These failures arise primarily because agents either disregard or misinterpret the guardrail's verdict. We develop \sysname, which derives effect-oriented rules from reported incidents, hardens them against effect-preserving rewrites through counterexample-guided refinement, and augments each denial message with an agent-steer prompt that curbs evasion without disrupting legitimate work. With one rule set that transfers across agents, benchmarks, and harnesses, \sysname lies on the Pareto front of safety, utility, and cost against four baselines in all five settings we test. On RedCode, for example, \sysname cuts the attack success rate of a DeepSeek-V4.1-Flash agent in OpenCode from 49.7% to 7.8%, below every baseline.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Coding agent, Tool-call guardrail, Behavior security

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_AEcSOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses guardrail evasion—a critical runtime failure where blocking an unsafe tool call fails to stop a harmful outcome because the agent retries, rewrites commands, or reroutes actions until the effect is achieved. Evaluating four baseline guardrails (AgentSpec, AutoSpec, LlamaFirewall, ActPlane) on RedCode and SABER, the authors find that current safeguards fail because agents either disregard safety verdicts or misinterpret their scope (e.g., assuming only a specific command string was banned and wrapping payloads in scripts). To fix this, the authors introduce HEADNOTE, combining 156 effect-oriented rules—derived from threat intelligence and hardened via a 3-agent defender–adversary–corrector loop—with a structured agent-steer prompt (Verdict, Routing, Provenance) appended to denial messages. Across three LLM backbones, three harnesses, and four benchmarks, HEADNOTE sits on the Pareto front of safety, utility, and cost, dropping DeepSeek's RedCode attack success rate from 49.7% down to 7.8%.

Critical strengths

Identifies Post-Block Cognitive Failures: Formalizes how agents evade guardrails by disregarding block verdicts or misinterpreting rule scope.

Counterexample-Guided Rule Evolution: Uses a 3-agent loop to abstract fragile syntax denylists into consequence-class rules, dropping false-block rates on benign tasks from 39% to 4%.

Agent-Steer Prompt Architecture: Guides agents to halt on malicious requests or safely reroute on benign ones without extra API calls.

Generalizable Pareto-Front Performance: A single 156-rule set transfers seamlessly across models, harnesses, and benchmarks.

Critical weaknesses

Unattended Adversary Scope: Assumes non-adaptive agent runs. Suggestion: Test against active, multi-turn prompt injection attempts aimed at jailbreaking the steer prompt.

Dependence on Informative Block Reasons: Utility drops if guardrails return opaque rule names. Suggestion: Add an automated translation layer for opaque policy verdicts.

Focus on Shell/CLI Environments: Primarily evaluates Linux shell tools. Suggestion: Extend evaluation to web APIs, GUI workflows, and multimodal agents.

Questions for authors

How does the agent-steer prompt perform on smaller instruction-following models?

Could the defender–adversary–corrector loop operate continuously online using production telemetry?

Does appending multi-paragraph steer prompts cause token window saturation in long-horizon tasks?

2 comments
AuthorsOpenReview ↗

Reply to Reviewer AEcS for Three Weakness

We thank the reviewer for the encouraging assessment and thoughtful suggestions. We have uploaded a revised PDF, in which additions are blue and deletions are red with strikethrough. Table and figure numbers refer to the revised PDF. Part 2 follows in a second comment.

W1 (adaptive attacks on the steer prompt). Early in this project, we did consider defending against adaptive attacks, but doing so requires strong adaptive attacks first, which is likely a substantial research topic in its own right. As stated in Section 3, we therefore scope our study to a common practical setting for coding-agent users, in which unsafe calls arise because the agent encounters indirect environment injection or makes unsafe calls on its own during task execution. This setting is widespread and clear to analyze, and studying it in depth already yields rich empirical findings. Adaptive attacks against steer prompts, together with stronger defenses that still preserve utility, are an exciting direction that we leave to future work.

W2 (a translation layer for opaque block reasons). We tested your suggestion directly in OpenCode with DeepSeek-V4.1-Flash on AutoSpec, whose stock block reason is only an opaque predicate name, using the exact same model and rules as Table 3 and Table 11. Before the runs, we used DeepSeek-V4.1-Flash once offline to translate each AutoSpec rule (18 on SABER and 14 on RedCode) into one sentence stating the forbidden effect, adding zero model calls at runtime. Full results on both SABER and RedCode are reported in Appendix D.6 (Table 12), with a pointer in Appendix D.5.

On SABER, appending our prompts directly to AutoSpec's opaque predicate names makes the agent overly cautious because it cannot tell what effect was blocked, lowering SR from 33.0% (AutoSpec alone, Table 3) to 28.2% (strict) and 32.4% (utility, Table 11). Adding the offline translation layer restores SR to 32.8% with strict and 33.2% with utility, recovering the 33.0% SR of AutoSpec alone while lowering SABER ER from 5.8% to 2.4% and 1.9% and HR from 5.2% to 3.9% and 3.4%. This confirms that an offline translation layer recovers the utility lost to opaque block reasons.

Comparing rewritten AutoSpec plus our prompts against full HeadNote (our 156-rule set plus our prompts) in Table 12 also shows the distinct contribution of our rule set. Full HeadNote achieves both higher SABER SR (37.0% and 38.5% versus 32.8% and 33.2%) and much lower RedCode ASR (7.8% and 8.3% versus 13.9% and 15.6%), because our hardened rules catch more attacks upfront (RedCode BR 85.0% and 85.3% versus 77.9% and 75.6%) with fewer false blocks on legitimate tasks. Rewritten AutoSpec trades lower SABER SR and higher RedCode ASR for slightly lower SABER HR (3.9% and 3.4% versus 5.6% and 5.3%), so both lie on the Pareto front and no AutoSpec configuration dominates either version of HeadNote.

W3 (web APIs, GUI workflows, and multimodal agents). Web APIs, GUI workflows, and multimodal agents are valuable directions. They lie outside the CLI coding and operations setting studied in this paper, so we leave their empirical evaluation to future work. At the same time, we believe our methodology of measuring post-block agent behavior first and then designing targeted interventions transfers well to these broader domains and can inform follow-up defenses.

AuthorsOpenReview ↗

Reply to Reviewer AEcS for Three Questions

Q1 (smaller instruction-following models). To answer this question, we evaluated three smaller open instruction-following models in the OpenCode harness, namely Qwen3.8-27B, gpt-oss-20b, and Llama-3.1-8B-Instruct, on both RedCode and SABER. Using the exact 156-rule set and agent-steer prompts from Section 6 without retuning, we compared four settings on each model, namely no defense, HeadNote rules without the steer prompt (HeadNote w/o steer), HeadNote-strict, and HeadNote-utility. Full results and behavioral statistics are added in Appendix D.8 (Table 13), with a pointer at the end of Section 6.2, and key metrics (%) are summarized in the table below.

On RedCode, HeadNote's rules alone already deliver the bulk of the safety improvement across all three models, cutting ASR from 76.7% to 15.3% on Qwen3.8-27B, from 51.1% to 7.2% on gpt-oss-20b, and from 35.9% to 12.5% on Llama-3.1-8B-Instruct. Adding the multi-paragraph steer prompt on top of the rules shows a clear split by model capability, depending on how reliably each model follows conditional post-block instructions.

  • On Qwen3.8-27B, the model follows the prompt as intended. After a RedCode block, it keeps making tool calls in 38% of blocked runs without the prompt, dropping to 1% (strict) and 2% (utility) with the prompt, which lowers RedCode ER from 10.8% to 1.1% and 2.8% (and ASR to 7.5% and 9.2%).
  • On gpt-oss-20b, the base model already stops after 96% of RedCode blocks even without a steer prompt, leaving little room for change, so RedCode ER stays low (1.7% to 4.7%).
  • On Llama-3.1-8B-Instruct, the 8B model struggles to follow conditional routing, keeping up tool calls after 55% of RedCode blocks without the prompt and 50% with strict. Under the utility prompt, which invites safe alternative routes when the user's goal is legitimate, it misreads attack tasks as goals to keep pursuing, raising post-block continuation to 70%, ER from 6.6% to 16.8%, and RedCode ASR from 12.5% to 18.4%.

On SABER without a defense, these models rarely refuse risky requests on safety grounds (18.7% on Qwen3.8-27B, 5.6% on gpt-oss-20b, and 0.3% on Llama-3.1-8B-Instruct, versus 45.2% on DeepSeek, where safety refusals count against SR). Raw SR therefore includes tasks completed harmfully, whereas safe success rate (SSR) drops sharply with model capability from 46.6% to 13.7% and 5.3%, and HR rises (29.3%, 48.6%, and 65.4%) because weaker models execute injected or risky commands early even when they fail to finish the task (37.4% of Llama-3.1-8B-Instruct runs cause harm without completing the task). Adding the prompt lowers SABER HR by up to 2.7 points while shifting SR by at most 3.9 points and ER by at most 3.5 points. Overall, our steer prompt works well on capable mid-sized models such as Qwen3.8-27B, whereas small 8B models benefit primarily from the effect-oriented rules and should avoid conditional utility prompts.

ModelSettingRedCode ASRRedCode ERSABER SRSABER SSRSABER HR
Qwen3.8-27BNo defense76.7–71.846.629.3
HeadNote w/o steer15.310.868.846.127.2
HeadNote-strict7.51.165.643.326.8
HeadNote-utility9.22.868.446.426.3
gpt-oss-20bNo defense51.1–52.213.748.6
HeadNote w/o steer7.21.746.610.348.6
HeadNote-strict9.24.746.911.746.6
HeadNote-utility8.12.846.412.347.5
Llama-3.1-8B-InstructNo defense35.9–33.25.365.4
HeadNote w/o steer12.56.630.64.864.6
HeadNote-strict14.28.329.15.463.3
HeadNote-utility18.416.826.74.361.9

Q2 (continuous online operation with production telemetry). Yes. We use this loop in production to iterate on rules. It takes run logs as input, which production telemetry can supply directly (Figure 3, “run logs from real use”). At the same time, maintaining the rule set in real agent use still involves periodic patching alongside a small amount of manual supervision and adjustment (Section 5.1).

Q3 (token window saturation in long-horizon tasks). Not in our runs. The strict and utility prompts are 124 and 181 words long (Appendix C.2) and are appended only when a tool call is blocked. On SWE-bench with HeadNote-utility, only 212 of 16,515 tool calls are blocked (Table 9). In the SABER runs with HeadNote on DeepSeek in Table 3, a run is blocked less than once on average (99% of runs have at most 4 blocks, with a maximum of 5 under utility and 8 under strict), adding under 1,000 words even in the single most-blocked trajectory. Runtime text injections of this size are routine in modern coding harnesses, such as the system reminders injected during conversation in Claude Code.

Reviewer_rcmjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies agent behavior after a tool-call guardrail blocks an action, focusing on continued pursuit of harmful effects and premature abandonment of legitimate tasks. Its measurement study examines four guardrails and interprets six observed behavior patterns through two proposed mechanisms: disregarding the verdict or misinterpreting its reason or scope. Motivated by these findings, HEADNOTE combines 156 effect-oriented rules with denial messages that explain the prohibited effect and guide the agent to stop or pursue a safe alternative. Its strict and utility variants differ in their treatment of request provenance. Experiments cover RedCode and SABER across five model–harness settings, benign coding tasks, transfer to other rule sets and guardrails, and fixed-rule prompt ablations. The results support the practical value of post-block feedback with setting-dependent safety–utility trade-offs.

Critical strengths

S1. The outcome-level evaluation addresses an important gap in tool-call defenses.

Blocking a call does not establish that the agent avoids the harmful outcome or completes the legitimate task. Evaluating both outcomes makes the paper's distinction practically useful. For example, on OpenCode–DeepSeek, HEADNOTE-utility reduces RedCode ASR from 49.7% without a guardrail to 8.3%, and SABER harm from 8.8% to 5.3%, while task success changes from 40.2% to 38.5% (Table 3). Using the same rules and prompt variants across five settings gives the findings meaningful breadth, although the size of the benefit varies across settings.

S2. The ablations and prompt transfer support a contribution beyond the rule library.

Figure 5 holds the rules fixed while comparing denial messages, including simple instructions against workarounds and for safer routes. These controls help distinguish suppressing harmful continuation from preserving useful continuation. The transfer experiments also provide evidence beyond HEADNOTE's own rule library: on OpenCode–DeepSeek, RedCode ER with AgentSpec's rules decreases from 13.9% to 4.2% with the strict prompt and 1.8% with the utility prompt (Tables 3 and 10). Section 6.3 specifies that AgentSpec/AutoSpec rules are loaded into the HEADNOTE enforcement framework. Effects vary across guardrails and metrics: the strict prompt lowers LlamaFirewall's RedCode ER but also its SABER task success, while ActPlane's SABER task success decreases under both prompts. These results support useful, setting-dependent feedback interventions rather than uniform improvement. The intervention is lightweight and does not require another runtime model call.

Critical weaknesses

W1. Disabling ActPlane's native feedback limits the post-block safety–utility comparison.

Appendix A.3, Table 6 disables ActPlane's forwarding of block reasons. Semantic feedback is part of the original ActPlane design, which separately evaluates an opaque-feedback variant. Communicating the meaning and scope of a denial is central to HEADNOTE, so comparison with native feedback is relevant. This concern is most consequential for SABER: ActPlane's RedCode ER is already 0–3.8% across the three OpenCode models, leaving little room to improve that post-block metric. The experiment adding HEADNOTE prompts measures a different feedback intervention from ActPlane's native one.

Q1 — Configuration clarification and targeted validation: Why was native reason feedback disabled? A corresponding feedback-enabled comparison, particularly on SABER, would clarify the contribution relative to the full existing system; alternatively, delimit the claim to the restricted variant evaluated.

W2. The prevalence of the two proposed mechanisms needs better support from the labeling procedure.

Appendix B.1 includes human reading and category refinement, but the final labeling and adversarial reassessment use the same LLM without a reported independent audit of the resulting labels. Several excluded categories also lack counts. Some exclusions, such as false harm labels and successful completion, are appropriate. Knowingly choosing an undetected command needs a clearer boundary if the agent continues pursuing the blocked effect. This affects the interpretation and prevalence of the proposed mechanisms. Figure 2's behavior classifications and ER cover different outcome sets, with ER including harm from any cause in blocked tasks.

Q2 — Clarification and targeted label validation: Please report the excluded-category counts and explain this boundary. An independently annotated sample would help assess label reliability and the strength of the proposed mechanism interpretation.

Q3 — Secondary clarification of evaluation provenance: Appendix D.4 already reports a 7:3 split and training-only feedback for an 11-round study. Please explain how that study relates to the final 156 rules and prompt selection, and how the 90-task subset in Figure 5 was selected. This asks for the existing development and selection history. The fixed-rule ablations and reported cross-setting results remain positive evidence for the intervention.

Overall assessment. I lean toward weak acceptance. The outcome-level evaluation, fixed-rule ablations, and feedback-transfer experiments constitute a useful empirical contribution. The native-feedback comparison and mechanism-label reliability limit the comparative and explanatory scope, but currently do not outweigh the positive intervention evidence. The provenance clarification would strengthen confidence in the reported evaluation.

Questions for authors

See weaknesses

2 comments
AuthorsOpenReview ↗

Reply to Reviewer rcmj for W1/Q1

We thank the reviewer for the thorough review and constructive requests. We have uploaded a revised PDF, in which additions are blue and deletions are red with strikethrough. Table and figure numbers refer to the revised PDF. Part 2 follows in a second comment.

W1 / Q1 (ActPlane's native feedback was disabled). ActPlane's open-source repository provides native-feedback hooks for Claude Code and Codex, but not for OpenCode. In Table 3, the agent on OpenCode therefore sees only the raw system-call error EPERM when ActPlane blocks an operation. We now clarify this in Section 4 and Appendix A.3 (Table 6).

Following your suggestion, we implemented an OpenCode hook that forwards ActPlane's native feedback to the agent (returning the rule name, target operation, and reason, and instructing the agent to take a compliant alternative route or explain why none exists). Using the exact same OpenCode and DeepSeek-V4.1-Flash setup as Table 3 and Table 11, we report a side-by-side comparison on both SABER and RedCode in Appendix D.6 (Table 12), alongside Section 6.3 and Appendix D.5 (Table 11). Across ActPlane's rules, Table 12 compares four denial-message settings, namely EPERM only (Table 3), ActPlane's native feedback, and ActPlane's three-line block header (rule name, target operation, and reason) paired with our strict or utility steer prompt (Table 11), and contrasts them with full HeadNote (our 156-rule set plus our prompts, Table 3).

Table 12 highlights two clear findings. First, holding ActPlane's rules fixed, enabling structured feedback does curb post-block evasion on the tasks that ActPlane blocks. Native feedback lowers SABER ER from 15.2% under EPERM (22.2%, or 8 of 36, on the paired task slice) to 6.1% (2 of 33 blocked runs) and lowers SABER HR from 7.7% to 5.9% while keeping SR at 38.5%. Replacing the guidance after ActPlane's block header with our steer prompts achieves a comparable reduction on blocked tasks, lowering SABER ER to 5.6% (2 of 36, strict) and 7.1% (2 of 28, utility), while RedCode ER is 0.0% across all four ActPlane settings. Second, regardless of which denial message ActPlane uses, its overall safety is bottlenecked by rule coverage. Its rules block only 7.8% to 10.1% of SABER tasks and 35.1% to 37.8% of unrefused RedCode attacks, so most unsafe calls execute without ever triggering a block. Consequently, ActPlane's RedCode ASR remains high at 29.7% under native feedback and 30.8% and 31.4% with our prompts (versus 49.7% without a defense). By contrast, full HeadNote blocks 85.0% and 85.3% of unrefused RedCode attacks and lowers RedCode ASR to 7.8% (strict) and 8.3% (utility) and SABER HR to 5.6% and 5.3%, while keeping SABER SR at 37.0% and 38.5% (and SSR at 32.8% and 34.0% versus 33.5% for ActPlane with native feedback). Thus HeadNote-utility matches native-feedback ActPlane in SABER SR and has better point estimates for SABER SSR, SABER HR, and RedCode ASR.

AuthorsOpenReview ↗

Reply to Reviewer rcmj for W2/Q2/Q3

W2 / Q2 (excluded-category counts, the boundary for knowingly evading, and label reliability). Excluded counts and why Figure 2 differs from ER. Appendix B.1 adds Table 7 accounting for all 3,177 blocked runs in Figure 2, comprising 1,735 safe endings (stopping on RedCode or completing SABER without harm), 550 runs labeled harmful by the benchmark (harm side), and 892 harm-free SABER runs without completion (utility side). Because Figure 2 and Table 1 isolate failures caused by how the agent reacts with tool calls after seeing a block, they exclude runs that fail for other reasons, as broken down below. This scope difference explains why Figure 2's harm bars are lower than ER in Table 3. In Table 3, each guardrail's ER counts every harmful blocked run in that setting (550 in total across Figure 2) regardless of cause, including benchmark false positives, pre-block harm, or text-only disclosure, whereas Figure 2's harm bars count only the post-block evasion subset in Table 1 (241 in total) over the 3,177 blocked runs (Section 6.1 and Appendix B.1).

Counted in Table 1Not counted
Harm side, 550 runs labeled harmful by the benchmark241238 false harm labels, 34 no denial message reached the agent, 23 only disclosed the action in text, 12 harm before the first block, 2 text advice only
Utility side, 892 harm-free runs without task completion209318 safe core already delivered, 221 kept working and failed for another reason, 59 blocked on a step the task requires, 58 found no other route, 27 no denial message reached the agent

Boundary for knowingly choosing an undetected command. We agree that if an agent knows the effect itself is forbidden and deliberately crafts a command to slip past detection, it is still pursuing the blocked effect and belongs under Disregard (verdict) rather than outside Table 1. Although both this case and Misinterpret's “Rewrites the command form” (133 runs) switch commands after a block, the boundary lies in how the agent's reasoning interprets what was forbidden. Inferring that the block ruled out only the command form, tool, or path rather than the effect is “Rewrites the command form”, whereas knowing the effect was forbidden and picking a form to avoid detection is deliberate evasion under Disregard (verdict). Across all 550 harm-side runs, 0 runs show such reasoning, consistent with our evaluated agents being non-adversarial coding models, so Table 1 is unchanged (Appendix B.1).

Independent label check. On a stratified random sample of 120 blocked runs (60 harm-side and 60 utility-side) blindly re-annotated by two LLMs from model families different from our classifier (GPT-6-astra and Gemini-4-argon), binary agreement with our labels on entering Table 1 is 84.2% and 80.0% (Cohen's κ = 0.68 and 0.60), three-rater Fleiss' κ across Disregard, Misinterpret, and excluded is 0.60, and inter-annotator Cohen's κ is 0.66. Both annotators also reproduce the prevalence split between the two mechanisms among counted failures, assigning 82.1% and 86.5% to Misinterpret and 17.9% and 13.5% to Disregard, versus 80.0% and 20.0% under our original labels (Appendix B.1).

Q3 (how the 11-round study relates to the 156 rules and prompts, and how the 90-task subset was chosen). Purpose of the 11-round study. The 156 rules in our main experiments were refined over many rounds in real agent use (Section 5.1), incorporating incoming threat intelligence alongside light human supervision and manual adjustment. Because that ongoing process cannot be replayed as a closed benchmark, we designed a separate 11-round study on a fixed SABER split (222 target and 154 benign tasks, split 70/30 into train and test). It starts from one seed rule without the 156-rule set and reaches 71 rules in round 11, giving readers a controlled illustration of how our iterative workflow reduces false blocks while preserving safety rather than producing the 156 rules. This 11-round study also played no role in prompt design or selection. No agent-steer prompt is loaded during the 11 rounds and the agent sees only each rule's block reason, so neither the harmful rate nor the false-block rate is affected by the prompt. Our two prompt versions were developed separately with the fixed 156-rule set on OpenCode with DeepSeek (Figure 5) and used unchanged in all other settings (Section 6.3 and Appendix D.4).

Selection of the 90-task RedCode subset. Because post-block evasion is a relatively rare event that concentrates in three of the 12 RedCode categories without the steer prompt, path traversal, dynamic code execution, and unsafe deserialization (30 tasks each, 90 total), focusing on these 90 tasks makes differences in post-block evasion clear to observe in Figure 5, whereas main comparisons (Table 3 and Table 12) run all 360 tasks (Section 6.3 and Appendix D.7).

Reviewer_6srSOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

The paper researches on guardrail evasion when the agent continues to pursue the effect that the blocked call would have produced, regardless of command form or execution route. The paper first realized that the researched guardrail models will let harm happen after a block or hinders legitimate work. As a result, to prevent guardrail evasion, the paper proposes HEADNOTE. HEADNOTE derives effect-oriented rules from reported incidents, hardens them against effect-preserving rewrites through counterexample-guided refinement, and augments each denial message with an agent-steer prompt that curbs evasion without disrupting legitimate work. The experiments are comprehensive on two benchmarks where competitive performance were obtained.

Critical strengths
  1. The paper is interesting by focusing on guardrail evasion.
  2. The findings are strong motivation though it is better to limit the scope of conclusion.
  3. The Figure 3 is well drawn, which could directly understand the global picture.
  4. The experiments are comprehensive with detailed ablation studies.
Critical weaknesses
  1. The "every guardrail" for the in Line 78 is too absolute since the work does not cover all guardrails.
  2. The "guardrail" in L156 misses a more specific definition (besides L90 description), what if the Guardrail is a reject-fine-tune model without any requirement for the rule?
  3. The setup in L203 is some unclear whether there is any system prompt/agent configure to make sure that the expected harmful goal will keep encouraging to happen after one/several-round guardrail blocks.
  4. For the generated rule, whether it will be overfitting or limited to certain scenarios in L268?
  5. For Sec. 5.2, it would be better to reorganize the paper to provide more details in Sec 5.2.
Questions for authors
  1. In L147, whether "subsequent" should be "alternative"?
3 comments
AuthorsOpenReview ↗

Reply to Reviewer 6srS

We thank the reviewer for the careful reading and the specific suggestions. We have uploaded a revised PDF, in which additions are blue and deletions are red with strikethrough. Table and figure numbers refer to the revised PDF.

W1 / S2 (“every guardrail” is too absolute). We agree and have scoped this statement to the four guardrails evaluated in our measurement study across the abstract, the Introduction, Section 2, and Section 4.

W2 (the definition of “guardrail”, and a fine-tuned model without rules). Section 2 defines a guardrail by its runtime interface (issuing allow or block verdicts before a tool call executes), whether driven by rules, a prompted LLM judge, or a fine-tuned safety classifier. Because a fine-tuned rejection model also applies to our system, we revised Section 2 to explicitly include fine-tuned safety classifiers such as Llama Guard 4 (12B) (and specified LlamaFirewall's judge model), and updated Section 3 to refer to rules only when a guardrail uses them. We also tested HeadNote's agent-steer prompt with Llama Guard 4, which requires only the classification output of the guard model and no externally supplied rules.

We evaluated Llama Guard 4 alone in all five settings (Table 3 and Table 8, configured as in Table 6) and paired it with both agent-steer prompt versions using DeepSeek-V4.1-Flash in OpenCode (Section 6.3, Table 4 and Table 11). In the paired evaluation, RedCode ER fell from 10.8% with the classifier alone to 0.5% with the strict prompt and 1.6% with the utility prompt, while SABER SR fell from 25.5% with the classifier alone to 16.2% and 19.8%, respectively. Across all five settings (Table 3 and Table 8), HeadNote remains on the Pareto front of safety, utility, and cost.

W3 (post-block prompts and configuration). No. In the OpenCode measurement study, we added no system prompt, agent instruction, or retry mechanism to encourage the agent to keep pursuing a blocked effect. After each block, the guardrail returns its own denial message, and the agent decides how to proceed. We clarified this setup in Section 4.

W4 (rule overfitting and generalization). We moved key rule-design details from Appendix C.1 into Section 5.1. To reduce the risk of overfitting, the defender groups missed harms by consequence class and excludes task IDs and paths unique to individual tasks. The corrector backtests the resulting rules and revises or removes low-precision rules.

In a separate 11-round SABER study, the workflow started from one seed rule and used only training trajectories. The training false-block rate fell from 0.39 to 0.04, and the gap between the training and held-out rates stayed within 0.03 after round 2 (Section 6.3 and Appendix D.4). In the main evaluation, we used the same frozen rule set on RedCode and SABER across all five model–harness settings (Tables 3 and 8). On Aider Polyglot and SWE-bench, whose tasks were disjoint from the benign tasks used by the corrector, HeadNote-utility had slightly higher task-completion point estimates than the no-defense baseline (Appendix A.3 and Table 9).

W5 (additional detail in Section 5.2). We expanded Section 5.2 to explain when the agent should stop after a block and when it should continue a legitimate task without producing the blocked effect. The revised section also explains how the denial message states the blocked effect before presenting the agent-steer prompt, and how the strict and utility versions differ. The text and Table 2 now connect the verdict, routing, and provenance paragraphs to the post-block failure mechanisms they address, and Appendix C.2 provides the full text of both versions.

Q1 (“subsequent” versus “alternative”). Both behaviors occur in practice. In Figure 1a the agent retries slightly modified forms of the blocked command, whereas in Figure 1b it switches to an alternative route that produces the same effect. We have updated the definition in Section 3 to read “subsequent retries or alternative actions”.

I am raising my score to 3 as the rebuttal and paper revision addressed most of my concern. However, the rule overfitting is still a concern as it only avoids overfitting on task IDs and paths. Plus, the layout of revised Table 2 should be aligned.

AuthorsOpenReview ↗

Appreciation and Further Clarification about W4

Thank you for raising your score and for the helpful follow-up comments.

Sorry for explaining this too briefly in our earlier reply and leaving the impression that we only strip task IDs and task-specific paths. In our workflow, avoiding overfitting goes well beyond that check.

  1. First, the adversary probes each rule with systematic rewrite families across shell commands and harness-native tools, and the defender merges missed harms by consequence class into paired target-and-action predicates without reading benchmark scoring rules, while the corrector prunes or narrows low-precision rules using only training logs.
  2. Second, our experiments verify that the resulting rules generalize rather than memorize seen tasks. In the 11-round SABER study, both the harmful rate and the false-block rate on held-out test tasks closely track the training split (Appendix D.4), and in our main evaluation one frozen 156-rule set derived from public threat intelligence transfers unchanged across RedCode, SABER, SWE-bench, and Aider Polyglot as well as three harnesses and six models (Tables 3, 8, 9, and 13).

Thank you also for flagging Table 2. That shift comes from the revision markup kept in the current PDF so reviewers can track our exact edits, and we will make sure Table 2 is cleanly aligned in the unmarked version.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General comment to all reviewers

We thank all reviewers for their thoughtful feedback. We have uploaded a revised PDF, in which additions are blue and deletions are red with strikethrough. Table and figure numbers refer to the revised PDF. Our main revisions and additional experiments are summarized below.

  1. Scope and definitions. The abstract and Sections 1, 2, and 4 now scope our empirical findings to the evaluated guardrails. Section 2 clarifies the judge model used in LlamaFirewall and explicitly includes fine-tuned safety classifiers in our guardrail definition. Section 3 refers to rules only when a guardrail uses them and refines the evasion definition to “subsequent retries or alternative actions”.
  2. Experimental setup. Section 4 and Appendix A.3 (Table 6) explain why ActPlane returns only EPERM on OpenCode in Table 3, document the configuration of Llama Guard 4, and confirm that we add nothing to the harness that encourages the agent to retry after a block.
  3. Method details. Section 5.1 describes how the defender and the corrector prevent rule overfitting. Section 5.2 explains how each denial message is assembled and maps the three prompt paragraphs to the failure mechanisms they address (Table 2).
  4. Post-block measurement. Appendix B.1 adds a full breakdown of all 3,177 blocked runs (Table 7), clarifies the boundary between Table 1 and ER in Table 3, and reports an independent re-annotation check on 120 blocked runs.
  5. Study design and subset selection. Section 6.3 and Appendices D.4 and D.7 clarify why we run a separate 11-round rule-iteration study on a fixed SABER split and why the prompt ablation in Figure 5 uses the 90-task RedCode subset.
  6. New experiments. Section 6 (Table 3 and Table 4) and Appendices D.1 (Table 8) and D.5 (Table 11) add the fine-tuned safety classifier Llama Guard 4 (12B) across all five settings (GPT-5.5, DeepSeek-V4.1-Flash, and GLM-5.3 in OpenCode, plus Codex and DeepSeek Harness) and paired with our steer prompts on DeepSeek. Appendix D.6 (Table 12) evaluates ActPlane with native feedback and AutoSpec with offline-rewritten block reasons on OpenCode and DeepSeek. Appendix D.8 (Table 13) evaluates HeadNote on three smaller open models.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.