acceptodds
Under review as a conference paper at ICLR 2027

PACT: Provenance-Aware Contract Testing — What Do LLM-Agent Task-Contract Checkers Actually Certify?

Abstract

LLM agents now act on behalf of users: sending messages, initiating payments, and booking travel. To keep them in check, developers run contract checkers: programs that verify the agent only acted on destinations named in the task text. A clean certificate is usually taken to mean the agent did only what the task authorized. But the checker only matches strings: it can pass an action the task never authorized, and flag one the task did. We measure that gap on 7,598 public AgentDojo episodes. We first compute exact risk certificates for such a checker. We then test what its flags mean: independent LLM judges blindly review 400 stratified trace-task pairs and decide whether each sampled flag is a truly unauthorized action, under a predeclared rubric. These two steps — certify the label, then measure its meaning — form an audit framework we call Provenance-Aware Contract Testing (PACT). On the attack side, the checker is largely sound: about three-quarters of sampled attack-side flags hold up as genuinely unauthorized actions, and a predeclared extension covering what the main audit cannot see shows that almost no attacks are missed. Yet the dominant failure is over-strictness: 57.5% of sampled benign-side flags are actions the task did authorize (the exact rate depends on the coding rubric). A provenance-aware resolver, which tracks where each destination string came from rather than whether it appeared anywhere, authorizes 2.6x fewer semantic violations than the breadth baselines and eliminates all measured false rejections. The frame-level semantic violation rate is estimated under measured worst-case bounds, not certified. The rubric is human-validated on every checkable boundary unit, with zero residual disagreement, and in a prospective test on unseen tasks, the compiled resolver falls short of its predeclared threshold. Certificates should be read as statements about what the checker actually verified, not as proof of what the task authorized; we measure exactly what separates the two.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, agent safety, contract checking, sampled-audit certificates, selective labels, semantic validity, provenance tracking, AgentDojo

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_nsmsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PACT, a framework for auditing the “contract checkers” used to keep LLM agents safe. These checkers flag an agent action, such as sending a message or making a payment, when its destination doesn’t appear word-for-word in the task text. The paper argues that a clean checker result is often misread as “the agent did only what the task authorized,” while in fact the checker only verifies string matching. The paper identifies several ways the checker’s verdict and the true authorization disagree. The authors contribute a blinded LLM-judge audit protocol, a provenance-aware resolver that authorizes destinations by where they came from, and exact risk certificates for the checker’s literal label.

Critical strengths

The paper asks what a “clean” checker result actually guarantees, and it shows clearly that checker verdicts and true authorization can disagree. This matters because developers increasingly rely on these checkers to judge whether agents are safe.

The audit uses predeclared rubrics, blinded adjudication, exact statistical bounds, and an honest separation of what is certified from what is only measured. Because of this rigor, the findings are trustworthy and reproducible.

The paper finds that the checker is mostly sound on attacks but over-strict on benign tasks, and that a provenance-aware resolver reduces both kinds of error. This gives practitioners actionable guidance on how to design better checkers.

Critical weaknesses

The paper mixes several research questions. It combines (1) analyzing how and why current checkers and verifiers are wrong with (2) designing a new auditing system (the resolver and the risk certificates), and it also brings in prompt injection, which is a separate, well-studied research line. Each of these already has a substantial body of work, so putting them together makes the paper unfocused and makes it hard to tell what the core contribution is or how it improves on prior work in any single line.

The related work is not positioned in recent LLM-auditing research. The related work relies mainly on classical statistics references from the 1930s–1960s (e.g., Clopper–Pearson, Horvitz–Thompson, Hoeffding) and engages little with recent work on LLM-based verifiers and agent evaluation (e.g., AgentRewardBench, Agent-as-a-Judge, guardrail models like Llama Guard). Without this comparison, it is unclear how PACT differs from or improves on existing methods for auditing LLM agents, which weakens the paper’s claim of novelty.

I would suggest that:

  1. Split the paper’s message more clearly: either focus on auditing checker validity, or on the resolver and certificate design.
  2. Evaluate on at least one more agent benchmark beyond AgentDojo to show the error patterns generalize.
  3. Cover a wider range of errors, such as content errors where the destination is right but the payload is wrong, and multi-step leaks, which the destination-only contract can’t see.
Questions for authors

The paper does three things: analyzes checker errors, designs a resolver and certificates, and analyzes prompt-injection defenses. If you had to pick one main contribution, which is it, and how does it advance beyond prior work in that specific line?

How do the literal checker and the provenance-aware resolver compare with recent LLM-based judges or guardrail models (e.g., AgentRewardBench judges, Llama Guard) on the same 400 adjudicated pairs? If an LLM judge already catches most of these errors, what extra value do PACT’s certificates provide?

The rubric was written by the same team that built the checkers, and most adjudication is done by LLM sessions from a single model stack. How confident are you that the 57.5% over-strictness rate is not shaped by the rubric itself? Human validation covers only the 135 checkable units, out of 615 boundary units.

The resolver has hand-written bindings for only four task families, and its prospective test on unseen tasks was MARGINAL. What would it take to extend it to new families or other benchmarks, and how much manual effort would each new family need?

Reviewer_nEKvOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents PACT, a framework for evaluating whether LLM-agent task-contract checkers correctly identify unauthorized actions. The authors analyze 7,598 AgentDojo episodes and use LLM-based adjudication on 400 stratified trace-task pairs to compare literal contract violations with semantic authorization. They also propose a provenance-aware destination resolver and evaluate risk certification methods. The results suggest that literal checkers are often overly restrictive on benign tasks, while the proposed resolver reduces semantic false authorizations compared with broader string-matching baselines.

Critical strengths
  • Relevant problem formulation: The distinction between literal contract compliance and semantic task authorization is important for evaluating the security of LLM agents. The paper provides empirical evidence that mechanically verified contracts do not necessarily reflect user intent.
Critical weaknesses
  • Limited reliability of semantic ground truth (Sections 3.1–3.2, 6.3): The central findings depend on LLM-generated annotations rather than independently established human ground truth. Although the authors report high inter-session agreement and a human-validation study, the latter covers only 135 mechanically checkable boundary units. The larger annotation corpus remains without comprehensive human validation.
  • Unclear methodological contribution and limited generalizability (Sections 4 and 9): The provenance-aware resolver is evaluated on only four manually specified task families. More importantly, the authors acknowledge that the primary improvement comes from a conservative default-deny policy rather than the proposed provenance bindings or guards, whose individual ablations each change only one verdict. Existing provenance-grounded agentic frameworks emphasize evidence tracing and verifiable reasoning [1]. The authors should better explain how their provenance-aware resolver validates authorization decisions beyond the proposed destination-matching rules. The prospective evaluation on unseen tasks also fails to meet its predefined threshold. These findings substantially weaken the claimed contribution beyond a restricted set of hand-crafted rules.
  • Limited interpretability of experimental results (Sections 3–5): The evaluation reports numerous aggregate statistics without sufficiently explaining the underlying failure mechanisms. For example, the reported 2.6x reduction in semantic false authorizations is based on 59 violations from a selected subset, rather than an independent representative evaluation. Although examples are provided, a clearer breakdown of failure categories and their relationship to individual resolver components is needed to establish what the proposed approach actually improves.

[1] Mukherjee, K. and Kantarcioglu, M. (2025) ‘LLM-driven Provenance Forensics for Threat Investigation and Detection’. arXiv preprint, arXiv:2508.21323. Available at: https://arxiv.org/abs/2508.21323

Questions for authors
  1. Can the authors provide independent human validation on a representative sample of the semantically ambiguous cases, rather than relying primarily on LLM adjudication?
  2. Given that the provenance bindings and guards each change only one verdict in the ablation study, what evidence demonstrates that provenance-aware resolution provides a meaningful improvement beyond a simpler default-deny baseline?
Reviewer_RcRTOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper audits literal-destination contract checkers on AgentDojo traces. It gives exact finite-frame certificates for the literal label, then uses LLM adjudicators (400 stratified pairs) to estimate how well those flags match true authorization. It also proposes a provenance-aware resolver as a case study. The main finding is that the checker rarely misses attacks (recall ≈0.96) but is over-strict on benign tasks: 57.5% of sampled benign flags are actions the task did authorize.

Critical strengths
  • The paper asks a useful question: what a checker’s certificate actually certifies versus what users assume it means.
  • It shows concretely that literal checkers rarely miss attacks but are often over-strict on benign tasks, and that relaxing them lets attacker-injected destinations through.
  • It shows that contract-level and injection-level metrics can disagree, which matters for how prompt-injection defenses are evaluated.
Critical weaknesses
  • The semantic labels come from LLM adjudicators of a single model family, using a rubric written by the checker authors. Humans validated only the easy, mechanically checkable units.
  • The benign over-strictness finding depends on how empty-task units are coded, and the paper never explains why so many tasks are empty.
  • The resolver’s gain comes mostly from default-deny rather than provenance, it was developed on the evaluation traces, and it misses its own threshold on unseen tasks.
Questions for authors
  • Can you report human agreement on a sample of units that need real semantic judgment, not just the empty-task boundary cases?
  • Why do so many sampled tasks have empty task text, and does the benign over-strictness finding hold without them?
  • How does a plain default-deny rule, with no bindings or guards, compare to the resolver against the semantic labels?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.