acceptodds
Under review as a conference paper at ICLR 2027

TRACE: Enforcing Tool-Use Policies Beyond Model Guardrails

Abstract

Tool-using agents turn model outputs into actions that read from or change external systems, then feed the results back to the model. Model-side guardrails can discourage unsafe tool calls, but applications need a separate policy governing which calls may execute. We present TRACE, a policy-enforcement engine at the application's tool-call boundary. Before a tool call executes, TRACE checks it against a deployer-defined policy and returns ALLOW, ASK, or DENY. It records each decision separately from what the application reports as executed. We evaluate TRACE on 200 SWE-bench Pro tasks, testing each task under a protective policy and an allow-everything policy while holding the rest of the setup fixed. When the agent is instructed to manipulate grading, TRACE's protective policy cuts successful reward-hacking attempts from 50 to 14. Under ordinary task instructions, the policy has a small observed cost to legitimate work: the agent successfully completes 126 tasks, compared with 129 under the allow-everything policy. TRACE also supports an LLM as the decider for tool calls. We demonstrate this capability by replaying 5,650 tool calls with Claude Sonnet 5 as the decider. It denies 89 of 91 calls labeled as carrying attack payloads, compared with 75 for a basic rule-based decider, while denying 4 of 4,537 calls from honest runs. TRACE can combine both deciders while preserving every deterministic policy denial.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI agent safety, tool-use policy enforcement, agent security evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_J9ZfOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

TRACE is an external policy engine that controls tool calls made by LLM agents. Before execution, it checks call arguments, target resources, and reported execution history, then returns ALLOW, ASK, or DENY. It supports deterministic rules, an LLM judge, or both, with the more restrictive decision taking priority. It also records policy decisions separately from execution reports to help investigate surviving attacks. On 200 SWE-bench Pro tasks, successful reward hacking falls from 50 to 14, while legitimate task completions change from 129 to 126. Additional experiments cover simulated services, browser-policy revisions, and an offline comparison of rule-based and LLM judges on 5,650 recorded tool calls.

Critical strengths
  1. The main comparison keeps tools and budgets fixed and uses two disjoint task samples.
  2. Removing tampering and rerunning the grader provides a clear check of whether a passing patch depends on reward hacking.
  3. The analysis of surviving attacks explains concrete weaknesses in file-path and shell-command rules.
Critical weaknesses
  1. The added value over existing systems is unclear. The main experiment compares a protective policy with allow-everything. This shows that the policy helps, but does not establish what TRACE adds beyond a simpler implementation of the same restrictions. Matched comparisons with a simple denylist, Progent, or AgentSpec are missing. The Cedar comparison shows agreement on the browser panel, but provides little evidence of an advantage for TRACE.
  2. The evaluation relies heavily on known attack details. The SWE-bench attack prompt names grading files and techniques that the protective policy explicitly targets. The AgentDojo policies also contain known attacker addresses and identifiers. Testing different tasks under these same attack assumptions provides limited evidence of generalization. The surviving attacks already show that agents can place payloads in permitted source files or use shell commands that the rules miss.
  3. The LLM evaluation does not establish effectiveness during live execution. The replay uses fixed history, leaving four history-dependent rules inactive. It also cannot measure how agents respond to denials or recover to complete their tasks. The hybrid produces 1368 ASK decisions out of 5650 calls, compared with 56 for the LLM alone. Since ASK can require user approval or become DENY, denial rates alone do not capture the practical interruption cost.
  4. The font used in the paper is weird. It makes the paper looks like a draft and hard to read.
Questions for authors

See weaknesses.

1 comment

Thank you for the direct response and for fixing the font. I will keep my rating.

The response agrees with my three main concerns but does not resolve them. The revision itself states that the rules add no new mechanism and that the main experiment measures a path denylist against a scripted attack, which file permissions would also enforce. The policies still rely on known attack details, and the blind-authoring test is left for future work. The LLM decider is still evaluated only in offline replay, and under a host that treats ASK as DENY the combined mode would block 23% of honest calls.

The measurement is careful, but the paper does not yet show a contribution beyond it.

Reviewer_9rDUOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a policy-enforcement framework that intercepts agent tool calls and applies rule-based and LLM-based decisions. It separates policy decisions from execution records. The authors evaluate its effectiveness against reward hacking and prompt-injection attacks and the results show substantial reductions in attack success with limited impacts on task completion.

Critical strengths
  1. Enforcing policies at the tool-call boundary is a meaningful approach to agent security. The separation of policy decisions and execution records also supports useful failure analysis.
  2. The controlled comparison on swe-bench pro demonstrates a substantial reduction in successful reward hacking, with a limited small decrease in task completion.
Critical weaknesses
  1. The designs in this work, including policy enforcement, resource-based authorization, and runtime monitoring have been extensively explored in prior work such as Progent, and AgentSpec. TRACE primarily integrates these established mechanisms, but the paper lacks direct end-to-end comparisons demonstrating that it provides substantial advantages over existing work. This weakens the novelty and contribution claims.
  2. The presentation and writting of this work could also be improved.
Questions for authors

What's the key difference and advantage of TRACE compared with existing policy-enforcement methods like Progent, and AgentSpec? Can the authors provide controlled comparisons under equivalent policies and tool interfaces?

Reviewer_8iEKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents TRACE, an external policy-enforcement engine situated at the application's tool-call boundary to prevent unsafe or policy-violating agent actions when internal model guardrails fail. TRACE intercepts proposed tool calls, extracts target resources (e.g., file paths, network destinations, MCP channels), evaluates them against deployer-specified rules, trusted context, and reported execution history, and returns ALLOW, ASK, or DENY verdicts. Crucially, TRACE records policy decisions prior to execution and maintains them separately from host-reported execution traces to facilitate tamper-evident post-hoc auditing. In an empirical evaluation on 200 SWE-bench Pro software engineering tasks under an adversarial prompt commanding the agent to hack test grading, TRACE’s protective policy reduces successful reward-hacking attempts from 50 to 14, while preserving legitimate task completion (126 under protective policy vs. 129 under allow-everything). The paper also validates TRACE’s support for LLM-based deciders (Claude Sonnet 5), which denied 89 of 91 recorded attack payload calls compared to 75 for basic rules, while using deterministic rules as an un-overrideable policy floor.

Critical strengths

Situating authorization logic strictly outside the acting agent’s prompt and runtime context is the correct architectural paradigm for agent security. Decoupling pre-execution decision logs from post-execution host reports provides a solid mechanism for auditing mediation discrepancies and identifying unhooked execution channels.

Evaluating against 200 SWE-bench Pro tasks under explicit reward-hacking incentives grounds the defense in a realistic developer-agent threat model. The reversion-regrade methodology,re-evaluating patches after surgically removing the tampering payload on clean environment images,guarantees that the reported metric reflects genuine, load-bearing security enforcement rather than grading artifacts.

Critical weaknesses

The forensic analysis of surviving attacks in Section 5.3 reveals that 6 of 11 successful bypasses occurred because the agent embedded test-tampering payloads inside legitimate project source files (e.g., Ansible modules). Because the agent must legitimately modify source code to solve tasks, path-based access control rules cannot distinguish authorized functional patches from embedded grading exploits without deep semantic code inspection.

In fully autonomous or headless agent workflows, the ASK verdict must either halt execution or fall back to pre-configured defaults (such as mapping to DENY in Codex or default-ALLOW in unattended runs). The paper does not thoroughly evaluate the human-in-the-loop review overhead, potential alert fatigue, or the systemic failure modes introduced when hosts fail to handle ASK closed-fail.

Questions for authors

When an adversarial agent injects malicious tampering payloads directly into legitimate, task-editable source code files, what runtime mechanisms can TRACE integrate beyond static file-path regular expressions to detect payload delivery?

In LLM-decider mode, the observed median latency is 1.865 seconds per call. How does this latency and token cost scale in real-time, interactive multi-agent deployments involving dozens of tool calls per turn?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.