- Rating
- 4
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper addresses guardrail evasion—a critical runtime failure where blocking an unsafe tool call fails to stop a harmful outcome because the agent retries, rewrites commands, or reroutes actions until the effect is achieved. Evaluating four baseline guardrails (AgentSpec, AutoSpec, LlamaFirewall, ActPlane) on RedCode and SABER, the authors find that current safeguards fail because agents either disregard safety verdicts or misinterpret their scope (e.g., assuming only a specific command string was banned and wrapping payloads in scripts). To fix this, the authors introduce HEADNOTE, combining 156 effect-oriented rules—derived from threat intelligence and hardened via a 3-agent defender–adversary–corrector loop—with a structured agent-steer prompt (Verdict, Routing, Provenance) appended to denial messages. Across three LLM backbones, three harnesses, and four benchmarks, HEADNOTE sits on the Pareto front of safety, utility, and cost, dropping DeepSeek's RedCode attack success rate from 49.7% down to 7.8%.
Critical strengths
Identifies Post-Block Cognitive Failures: Formalizes how agents evade guardrails by disregarding block verdicts or misinterpreting rule scope.
Counterexample-Guided Rule Evolution: Uses a 3-agent loop to abstract fragile syntax denylists into consequence-class rules, dropping false-block rates on benign tasks from 39% to 4%.
Agent-Steer Prompt Architecture: Guides agents to halt on malicious requests or safely reroute on benign ones without extra API calls.
Generalizable Pareto-Front Performance: A single 156-rule set transfers seamlessly across models, harnesses, and benchmarks.
Critical weaknesses
Unattended Adversary Scope: Assumes non-adaptive agent runs. Suggestion: Test against active, multi-turn prompt injection attempts aimed at jailbreaking the steer prompt.
Dependence on Informative Block Reasons: Utility drops if guardrails return opaque rule names. Suggestion: Add an automated translation layer for opaque policy verdicts.
Focus on Shell/CLI Environments: Primarily evaluates Linux shell tools. Suggestion: Extend evaluation to web APIs, GUI workflows, and multimodal agents.
Questions for authors
How does the agent-steer prompt perform on smaller instruction-following models?
Could the defender–adversary–corrector loop operate continuously online using production telemetry?
Does appending multi-paragraph steer prompts cause token window saturation in long-horizon tasks?
2 comments
Reply to Reviewer AEcS for Three Questions
Q1 (smaller instruction-following models). To answer this question, we evaluated three smaller open instruction-following models in the OpenCode harness, namely Qwen3.8-27B, gpt-oss-20b, and Llama-3.1-8B-Instruct, on both RedCode and SABER. Using the exact 156-rule set and agent-steer prompts from Section 6 without retuning, we compared four settings on each model, namely no defense, HeadNote rules without the steer prompt (HeadNote w/o steer), HeadNote-strict, and HeadNote-utility. Full results and behavioral statistics are added in Appendix D.8 (Table 13), with a pointer at the end of Section 6.2, and key metrics (%) are summarized in the table below.
On RedCode, HeadNote's rules alone already deliver the bulk of the safety improvement across all three models, cutting ASR from 76.7% to 15.3% on Qwen3.8-27B, from 51.1% to 7.2% on gpt-oss-20b, and from 35.9% to 12.5% on Llama-3.1-8B-Instruct. Adding the multi-paragraph steer prompt on top of the rules shows a clear split by model capability, depending on how reliably each model follows conditional post-block instructions.
- On Qwen3.8-27B, the model follows the prompt as intended. After a RedCode block, it keeps making tool calls in 38% of blocked runs without the prompt, dropping to 1% (strict) and 2% (utility) with the prompt, which lowers RedCode ER from 10.8% to 1.1% and 2.8% (and ASR to 7.5% and 9.2%).
- On gpt-oss-20b, the base model already stops after 96% of RedCode blocks even without a steer prompt, leaving little room for change, so RedCode ER stays low (1.7% to 4.7%).
- On Llama-3.1-8B-Instruct, the 8B model struggles to follow conditional routing, keeping up tool calls after 55% of RedCode blocks without the prompt and 50% with strict. Under the utility prompt, which invites safe alternative routes when the user's goal is legitimate, it misreads attack tasks as goals to keep pursuing, raising post-block continuation to 70%, ER from 6.6% to 16.8%, and RedCode ASR from 12.5% to 18.4%.
On SABER without a defense, these models rarely refuse risky requests on safety grounds (18.7% on Qwen3.8-27B, 5.6% on gpt-oss-20b, and 0.3% on Llama-3.1-8B-Instruct, versus 45.2% on DeepSeek, where safety refusals count against SR). Raw SR therefore includes tasks completed harmfully, whereas safe success rate (SSR) drops sharply with model capability from 46.6% to 13.7% and 5.3%, and HR rises (29.3%, 48.6%, and 65.4%) because weaker models execute injected or risky commands early even when they fail to finish the task (37.4% of Llama-3.1-8B-Instruct runs cause harm without completing the task). Adding the prompt lowers SABER HR by up to 2.7 points while shifting SR by at most 3.9 points and ER by at most 3.5 points. Overall, our steer prompt works well on capable mid-sized models such as Qwen3.8-27B, whereas small 8B models benefit primarily from the effect-oriented rules and should avoid conditional utility prompts.
| Model | Setting | RedCode ASR | RedCode ER | SABER SR | SABER SSR | SABER HR |
|---|---|---|---|---|---|---|
| Qwen3.8-27B | No defense | 76.7 | – | 71.8 | 46.6 | 29.3 |
| HeadNote w/o steer | 15.3 | 10.8 | 68.8 | 46.1 | 27.2 | |
| HeadNote-strict | 7.5 | 1.1 | 65.6 | 43.3 | 26.8 | |
| HeadNote-utility | 9.2 | 2.8 | 68.4 | 46.4 | 26.3 | |
| gpt-oss-20b | No defense | 51.1 | – | 52.2 | 13.7 | 48.6 |
| HeadNote w/o steer | 7.2 | 1.7 | 46.6 | 10.3 | 48.6 | |
| HeadNote-strict | 9.2 | 4.7 | 46.9 | 11.7 | 46.6 | |
| HeadNote-utility | 8.1 | 2.8 | 46.4 | 12.3 | 47.5 | |
| Llama-3.1-8B-Instruct | No defense | 35.9 | – | 33.2 | 5.3 | 65.4 |
| HeadNote w/o steer | 12.5 | 6.6 | 30.6 | 4.8 | 64.6 | |
| HeadNote-strict | 14.2 | 8.3 | 29.1 | 5.4 | 63.3 | |
| HeadNote-utility | 18.4 | 16.8 | 26.7 | 4.3 | 61.9 |
Q2 (continuous online operation with production telemetry). Yes. We use this loop in production to iterate on rules. It takes run logs as input, which production telemetry can supply directly (Figure 3, “run logs from real use”). At the same time, maintaining the rule set in real agent use still involves periodic patching alongside a small amount of manual supervision and adjustment (Section 5.1).
Q3 (token window saturation in long-horizon tasks). Not in our runs. The strict and utility prompts are 124 and 181 words long (Appendix C.2) and are appended only when a tool call is blocked. On SWE-bench with HeadNote-utility, only 212 of 16,515 tool calls are blocked (Table 9). In the SABER runs with HeadNote on DeepSeek in Table 3, a run is blocked less than once on average (99% of runs have at most 4 blocks, with a maximum of 5 under utility and 8 under strict), adding under 1,000 words even in the single most-blocked trajectory. Runtime text injections of this size are routine in modern coding harnesses, such as the system reminders injected during conversation in Claude Code.
Reply to Reviewer AEcS for Three Weakness
We thank the reviewer for the encouraging assessment and thoughtful suggestions. We have uploaded a revised PDF, in which additions are blue and deletions are red with strikethrough. Table and figure numbers refer to the revised PDF. Part 2 follows in a second comment.
W1 (adaptive attacks on the steer prompt). Early in this project, we did consider defending against adaptive attacks, but doing so requires strong adaptive attacks first, which is likely a substantial research topic in its own right. As stated in Section 3, we therefore scope our study to a common practical setting for coding-agent users, in which unsafe calls arise because the agent encounters indirect environment injection or makes unsafe calls on its own during task execution. This setting is widespread and clear to analyze, and studying it in depth already yields rich empirical findings. Adaptive attacks against steer prompts, together with stronger defenses that still preserve utility, are an exciting direction that we leave to future work.
W2 (a translation layer for opaque block reasons). We tested your suggestion directly in OpenCode with DeepSeek-V4.1-Flash on AutoSpec, whose stock block reason is only an opaque predicate name, using the exact same model and rules as Table 3 and Table 11. Before the runs, we used DeepSeek-V4.1-Flash once offline to translate each AutoSpec rule (18 on SABER and 14 on RedCode) into one sentence stating the forbidden effect, adding zero model calls at runtime. Full results on both SABER and RedCode are reported in Appendix D.6 (Table 12), with a pointer in Appendix D.5.
On SABER, appending our prompts directly to AutoSpec's opaque predicate names makes the agent overly cautious because it cannot tell what effect was blocked, lowering SR from 33.0% (AutoSpec alone, Table 3) to 28.2% (strict) and 32.4% (utility, Table 11). Adding the offline translation layer restores SR to 32.8% with strict and 33.2% with utility, recovering the 33.0% SR of AutoSpec alone while lowering SABER ER from 5.8% to 2.4% and 1.9% and HR from 5.2% to 3.9% and 3.4%. This confirms that an offline translation layer recovers the utility lost to opaque block reasons.
Comparing rewritten AutoSpec plus our prompts against full HeadNote (our 156-rule set plus our prompts) in Table 12 also shows the distinct contribution of our rule set. Full HeadNote achieves both higher SABER SR (37.0% and 38.5% versus 32.8% and 33.2%) and much lower RedCode ASR (7.8% and 8.3% versus 13.9% and 15.6%), because our hardened rules catch more attacks upfront (RedCode BR 85.0% and 85.3% versus 77.9% and 75.6%) with fewer false blocks on legitimate tasks. Rewritten AutoSpec trades lower SABER SR and higher RedCode ASR for slightly lower SABER HR (3.9% and 3.4% versus 5.6% and 5.3%), so both lie on the Pareto front and no AutoSpec configuration dominates either version of HeadNote.
W3 (web APIs, GUI workflows, and multimodal agents). Web APIs, GUI workflows, and multimodal agents are valuable directions. They lie outside the CLI coding and operations setting studied in this paper, so we leave their empirical evaluation to future work. At the same time, we believe our methodology of measuring post-block agent behavior first and then designing targeted interventions transfers well to these broader domains and can inform follow-up defenses.