acceptodds
Under review as a conference paper at ICLR 2027

Proof-Carrying Agent Contracts (PACT): Governed Release for Stateful Agent Execution

Abstract

Tool-using agents are usually judged by task completion, even though the side effects that produce a successful result must also satisfy authority, budget, interface, and recovery constraints at execution time. Proof-Carrying Agent Contracts (PACT) treats this commit boundary as a separate control problem. A typed contract binds an action graph to scoped authority, budget rules, admissible interfaces, declared recovery paths, proof obligations, and accountability evidence. At commit time, a trusted gate rechecks live-dependent obligations and records the decision in an attested release record. Under explicit assumptions on policy coverage, local-verifier soundness, and trusted commit binding, released actions are policy-admissible; structural refinement can only shrink the release set; and dependency-complete cache reuse is equivalent to recomputing reused checks. VeraContract-Bench adds deployment constraints to 180 autonomous software-engineering task instances and evaluates matched runs at both action and task levels. With three matched seeds for stochastic methods, PACT attains 63.5% task success with 1.8% unsafe-release rate, compared with 64.1% and 19.3% for the matched ReAct backbone. Governed success reaches 49.3%, versus 37.2% for AgentSpec-style runtime rules; recovery reaches 79.4%, and release-gate P99 latency is 64 ms. These results separate successful task completion from deployable execution at the commit boundary.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
LLM agents, agent safety, proof-carrying contracts, governed release, commit-time verification, accountability, autonomous software engineering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_WZyPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Proof-Carrying Agent Contracts (PACT), a runtime framework that separates an agent’s proposed actions from authorization to execute them. PACT represents execution constraints in typed contracts, checks applicable obligations against current state before releasing side effects, and records authorization evidence and declared recovery paths. Its formal analysis establishes conditional release soundness, monotonicity under contract refinement, and correctness of cached verification. Evaluation uses VeraContract-Bench, which adds deployment constraints to 180 SWE-bench-derived instances. The authors report reduced unsafe releases with similar task success, supported by component ablations and measurements of recovery, verification latency, and operator diagnosis.

Critical strengths

The paper makes a useful distinction between task completion and admissible execution: an agent can produce a correct result through unauthorized actions. Evaluating both outcomes captures deployment failures that task-success metrics alone would miss.

The reported results suggest that PACT reduces unsafe releases without a substantial loss in task success. Ablations separately examine live revalidation, recovery handling, and proof reuse, providing evidence about the roles of individual components, although they do not resolve my concerns about novelty and baseline adequacy.

Critical weaknesses

The technical contribution beyond existing runtime enforcement is unclear. PACT combines policy checks, approval handling, recovery declarations, and evidence recording, but the paper does not establish which capability requires its proposed contract framework. The formal guarantees assume complete policy coverage, sound verifiers, and trusted execution binding, rather than demonstrate how these difficult conditions are achieved. This limits the case for a substantive advance over existing approaches.

The evaluation does not convincingly isolate the framework’s practical advantage. Although a runtime-judge baseline is included, its specification is insufficient to assess whether it provides comparable policy coverage and access to current state. A simpler rule-based router with an independent verifier for ambiguous actions remains a plausible alternative. Without a comparison that matches these factors, the reported gains could reflect more comprehensive checks rather than the proposed architecture, leaving its additional complexity insufficiently justified.

Questions for authors

Could you clarify how the runtime-judge and AgentSpec-style baselines are implemented, particularly their policy coverage, access to current state, and handling of approval expiry and policy changes? Are these factors matched to PACT, so that the reported gains isolate the contribution of the contract framework?

What specific capability does PACT provide beyond deterministic policy checks combined with an independent verifier for ambiguous actions and commit-time state revalidation? Please identify a concrete failure case and explain which proposed mechanism addresses it.

What backbone model, decoding settings, and hardware were used, and what are the paired confidence intervals for the main performance differences? Please also clarify whether the latency comparison includes contract compilation, verification, and repair costs. These are requests to clarify the current evidence, rather than suggestions for future work.

1 comment
AuthorsOpenReview ↗

Matched baseline, recovery dependencies, and cost

Thank you. We evaluated your strengthened rule-router alternative. PACT retains substantial gains with separate verification and commit-time revalidation.

1. Strengthened baseline and parity. All methods share source tasks, repository snapshots, task-card policies, backbone, and budgets (40 tool calls, 1800 s). The runtime judge uses a rule-plus-model decision over policy text and the action record; AgentSpec-style applies deterministic rules before tool execution (Appendix E).

The strengthened baseline adds a separate runtime-verifier invocation for judgment-dependent actions, live-state access, approval-expiry and policy-epoch checks, and commit-time revalidation. It retains AgentSpec-style action blocking and agent-driven recovery under the same task-card eligibility/scoring rules: recovery improves from 54.8% to 58.2%. PACT adds the compiled action graph, graph-linked compensation preconditions, proof reuse, and Ω\Omega.

Task/UAR/VRR/Recovery/GSR are 62.9/5.6/81.3/58.2/40.6% for the strengthened baseline and 63.5/1.8/95.1/79.4/49.3% for PACT-full. RCR is 93.8% for the strengthened baseline and 98.2% for PACT-full (Table 2). Paired differences (PACT minus baseline), in percentage points with 95% intervals:

  • UAR: -3.8 [-5.4,-2.2].
  • Recovery: +21.2 [+16.4,+26.0].
  • GSR: +8.7 [+5.2,+12.3].
  • Task: +0.6 [-2.1,+3.3].

PACT retains safety, recovery, and governed-completion gains with similar observed task utility. Table 2 reports this comparison and intervals; Table 1 in Section 6 documents capability matching.

2. What the contract adds. PACT compiles task-specific cross-action dependencies into typed release obligations. The compiler instantiates G=(V,Efwd,Ecomp)G=(V,E^{\mathrm{fwd}},E^{\mathrm{comp}}), checks mandatory-obligation closure, and validates recovery/evidence references. The gate checks obligations in current state and binds the verdict to βt=(da,dC,ds,ρt)\beta_t=(d_a,d_C,d_s,\rho_t) (Eq. 9).

Appendix G, Case 2 records an LLMCompiler-style parallel refactor that removes a test file required by a later staging action's compensation path. Current-state access reveals the file's absence; the compiled graph represents its role as a prerequisite of that compensation path and staging release. PACT checks this graph-linked prerequisite at the staging commit (Eqs. 7, 29), alongside Pattern fan-out constraints, and withholds the effect when it fails. The evaluated action-local baseline has no compiled compensation dependency linking that resource to the later release. This illustrates the additional obligation that PACT constructs and enforces.

One-at-a-time ablations retain the remaining PACT configuration: removing compensation edges lowers recovery from 79.4% to 38.5% and GSR from 49.3% to 22.8%; removing critical-subgraph checks raises UAR from 1.8% to 4.4%; removing bounded safe downgrade lowers Task from 63.5% to 58.7%; removing Ω\Omega lowers blame localization from 87.4% to 71.9% (Tables 6, 8).

Eq. 13 defines complete dependency tuples. Under tuple equality, deterministic verification, unchanged verifier versions, and valid cache signatures, Theorem 2 equates reuse with recomputation. Across 600 trials mutating absence predicates, expiry, policy epochs, registry/verifier versions, and action/contract digests, cached-path decisions matched full recomputation in 600/600 cases, with correct invalidation of affected entries.

3. Settings and statistics. All methods use claude-sonnet-4-5-20250929 via an OpenAI-compatible chat-completions endpoint. Proposals: max_tokens 4096 and temperature 0.0 as the only sampling parameter sent; matched harness labels {17,42,101}. Full PACT compilation uses temperature 0.0, requests strict JSON-schema responses, and has k=5k=5 total attempts. Local hosts: 4 A100-SXM4-80GB GPUs, dual EPYC 7763 CPUs, 512 GB RAM.

Intervals use 10,000 hierarchical bootstrap replicates with source-issue clusters and nested matched harness runs; primary tests use Holm control. Against original AgentSpec-style, paired 95% intervals are delta UAR -5.3 [-6.8,-3.9], recovery +24.6 [+19.8,+29.4], and GSR +12.1 [+8.4,+15.9].

4. Full latency accounting. First-attempt compilation P50/P99 is 2.9/7.4 s; repair instances average 1.4 steps, with per-step P50 of 1.6 s. Measured compilation and repair contribute under 3% of run wall-clock within the shared 1800 s budget. Verification-path P50/P95/P99 is 612/938/1617 ms cold and 246/378/634 ms with proof reuse, a warm runtime, and speculative preparation. The combined configuration reduces path P99 by 60.8% (Table 9).

Final synchronous gate P99 is 64 ms, versus AgentSpec-style's 96 ms and the rule-verifier's 118 ms (Table 2). Preverified stable obligations reduce repeated work; live-dependent checks remain at release. Section 7 and Appendix I.3 define compilation/repair, verification-path, and synchronous-gate timing boundaries. We would appreciate reconsideration of your assessment and score.

Reviewer_L28vOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PACT introduces typed contracts and proof-carrying release for stateful agent actions. A deterministic gate checks obligations at commit time, while an untrusted model proposes actions. The design includes state/policy binding, recovery declarations, accountability records, and dependency-sensitive certificate caching. Evaluation reports improved governed completion and reduced unsafe releases with limited loss of task completion.

Critical strengths
  1. The separation between proposal and trusted release is clear and well motivated.
  2. The treatment of negative dependencies, expiration, and verifier versions in cache validity is thoughtful.
  3. The manuscript distinguishes declared recovery obligations from guarantees of eventual successful recovery, and reports several safety and utility metrics.
  4. It's quite timely problem!
Critical weaknesses
  1. The trusted execution boundary needs concrete evidence.
  • Atomic binding between a checked action and its actual effect is a central assumption. Please explain how the implementation mediates shell/code execution, remote API effects, concurrent state changes, and revocation between verification and commit.
  1. Metric aggregation is insufficiently specified.
  • Under the described 20 instances per family and three runs, family task-success rates such as 68.4% and 67.8% in Table 8 do not correspond to a simple average of 60 binary outcomes. Can you also provide raw numerators, denominators, exclusions, and aggregation formulas?
  1. Reproducibility and statistical evidence are incomplete in the manuscript.
  • The paper describes bootstrap and multiple-comparison procedures, yet the principal comparisons are presented as point estimates. Concrete model/decoding settings, task IDs, hardware, budgets, and uncertainty intervals should be well-written.
Questions for authors

What are the exact observation counts and aggregation rules behind Table 8?

Can you provide the full run manifest and confidence intervals for safety/utility differences?

It seems to me that policy coverage remains the dominant unresolved failure mode. Residual unsafe releases are attributed to policy omissions. Could you please distinguish what the closure check guarantees structurally from semantic coverage, show evidence supporting each residual-error attribution?

Which side effects are actually mediated atomically, and which remain assumptions about external services?

How does the closure test relate to the policy omissions responsible for the remaining unsafe releases?

1 comment
AuthorsOpenReview ↗

Counts, binding, and coverage

Thank you for the precise questions.

1. Original Table 8 (revised Table 10): aggregation and counts. Each family has 20 tasks and three matched harness repeats (60 scheduled runs). Family Task averages valid-run proportions; benchmark Task macro-averages nine families (Eq. 28):

Taskm,f=1003∑s=13km,f,sNm,f,svalid,Taskm=19∑f=19Taskm,f.\mathrm{Task}_{m,f} = \frac{100}{3} \sum_{s=1}^3 \frac{k_{m,f,s}}{N^{\mathrm{valid}}_{m,f,s}}, \qquad \mathrm{Task}_m = \frac{1}{9} \sum_{f=1}^9 \mathrm{Task}_{m,f}.

Here km,f,sk_{m,f,s} counts successes and Nm,f,svalidN^{\mathrm{valid}}_{m,f,s} counts valid runs. Only external setup faults (container timeouts, mount failures) are excluded; method, compilation, and tool failures remain scored. Family A: ReAct (13/19,13/19,13/19) gives 68.42% (68.4%); PACT (13/19,13/20,14/20) gives 67.81% (67.8%). Nine-family averaging yields benchmark Task of 64.1% and 63.5%. GSR uses Eq. 27's joint indicator over valid runs. Table 11 gives the terminal action counts.

Action metrics pool terminal candidates (Appendix D). UR/UB: unsafe released/blocked; SR/SB: safe released/blocked:

FamilyURUBSRSBUAR%VRR%
A26123430.896.8
B45516032.493.2
C24515921.295.7
D2329222.194.1
E2130366843.093.5
F22561149641.496.2
G78525832.692.4
H24720920.995.9
I48341820.995.4
Total6612723694251.895.1

The partition gives 66+1272=133866+1272=1338 unsafe proposals and 66+1272+3694+25=505766+1272+3694+25=5057 candidates, with 3760 releases:

\mathrm{VRR}=1272/1338\approx95.1\%,\quad \mathrm{RCR}=4966/5057\approx98.2\%.$$ Safe-candidate blocking is 25/3719(0.67%). ReAct's UR/UB/SR/SB = 1250/0/5220/0 yields UAR 19.3% and RCR 80.7%. **2. Mediated effects and commit.** Shell/code runs in copy-on-write staging; only the gate can write the canonical worktree. An exclusive commit lock protects live digest/epoch revalidation and staged-effect promotion. Signed evidence binds $\beta_t=(d_a,d_C,d_s,\rho_t)$ (Eq. 9). Changes to represented dependencies or the policy/registry epoch invalidate affected verdicts and require revalidation; revocations update that epoch. Package installs use a registry proxy pinned to the content-addressed artifact. Direct sockets are blocked. An egress proxy checks API request bindings, live allowlists, budgets, and epochs before transmission and attaches an idempotency token. The mediated commit event is request release; downstream service behavior follows Appendix J's external-service assumptions. Participating writers share synchronization; subprocesses inherit sandbox restrictions (Appendix I). Binding tests used 500 injections (100 per adapter class), covering parameter substitution, concurrent writes, revocation, approval expiry, and bypass attempts; no erroneous release or missed block was observed. Removing revalidation raises UAR from 1.8% to 6.9%; Appendix G, Case 3 shows rejection after an epoch change during suspension. **3. Closure, coverage, and attribution.** Closure checks mandatory families, typed cross-field/skill references, and recovery declarations; LCR@5 measures schema validity and closure within five attempts. Coverage concerns whether the predicates capture the governing policy. An Iface record can pass structural checks while omitting a forbidden-destination predicate. For all 66 residual unsafe releases, executed bindings matched signed records and verifier outcomes agreed with declared predicates. Comparison against the task-card oracle identified omitted or overly broad policy clauses, classifying these cases as coverage omissions. Table 12 reports interface 0.6, economy 0.4, memory 0.3, approval 0.3, and skill 0.2 percentage points of released actions. **4. Settings and paired intervals.** Backbone: claude-sonnet-4-5-20250929 via an OpenAI-compatible chat-completions endpoint. Proposals: max_tokens 4096, temperature 0.0 as the only sampling parameter sent; matched harness labels {17,42,101}. Full PACT compilation: temperature 0.0, strict JSON-schema responses requested, $k=5$ total attempts. Budgets: 40 tool calls, 1800 s. Local hosts: 4 A100-SXM4-80GB GPUs, dual EPYC 7763 CPUs, 512 GB RAM, Ubuntu 22.04, Docker 26.1. Paired 95% percentile intervals use 10,000 hierarchical bootstrap replicates with source-issue clusters and nested matched harness runs. Each replicate retains the metric's aggregation rules; primary tests use Holm control. Against AgentSpec-style, delta UAR is -5.3 points [-6.8,-3.9] and delta GSR +12.1 [+8.4,+15.9]; against Runtime judge, -7.6 [-9.8,-5.5] and +17.9 [+14.1,+21.6]; against ReAct, delta Task -0.6 [-3.1,+1.9]. Table 5 reports these intervals. The full run manifest and case-level audit records will be supplied as auxiliary supplementary materials. We would appreciate reconsideration of your assessment.
Reviewer_H617OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed PACT, a contract-based framework that verifies LLM agents’ actions against authorization, budget, safety, and recovery requirements before execution. Its key contributions include typed agent contracts, formal guarantees for policy-compliant execution, and VeraContract-Bench for evaluating execution safety. Experiments on 180 software-engineering tasks show that PACT reduces unsafe releases while maintaining comparable task success. Ablation studies further validate the effectiveness of its key components.

Critical strengths
  1. PACT addresses the important problem of ensuring policy-compliant agent execution. It substantially reduces unsafe releases while maintaining comparable task success, demonstrating a favorable safety–utility trade-off.

  2. The paper combines typed contracts, formal guarantees, and runtime verification into a coherent framework. Extensive baseline comparisons and ablation studies substantiate the effectiveness of individual components.

Critical weaknesses
  1. PACT combines established techniques such as runtime policy enforcement, proof-carrying verification, and transaction-based recovery. The paper does not sufficiently distinguish its fundamental technical innovations from closely related frameworks, raising concerns about its methodological novelty.

  2. Evaluation is restricted to software-engineering tasks with explicitly defined policies and constraints. Moreover, the formal guarantees rely heavily on complete policy coverage and sound verification, which may be difficult to ensure in real-world agent systems. This limits the practical significance of the theoretical results and the evidence for broader applicability.

Questions for authors

Please see the weaknesses above, and respond to them. Thank you!

1 comment
AuthorsOpenReview ↗

Graph-linked release contracts

Thank you for recognizing PACT's safety-utility trade-off. PACT's technical advance is a typed, graph-linked release object that unifies action authority, cross-action state dependencies, recovery prerequisites, and accountability evidence. Its obligations are constructed before execution, maintained as state changes, and checked together at each mediated, contract-covered commit.

W1. Technical contribution and evidence.

(1) State-bound authorization and correct proof reuse. The binding βt=(da,dC,ds,ρt)\beta_t=(d_a,d_C,d_s,\rho_t) identifies the exact release object (Eq. 9). For each verifier, Eq. 13 defines a complete dependency tuple χj,t\chi_{j,t}, including negative/absence predicates, expiry-sensitive facts, and verifier versions. Reuse requires χj,t=χj,t′\chi_{j,t}=\chi_{j,t'}, deterministic verification, an unchanged verifier version, and a valid cache signature. Under these conditions, Theorem 2 equates cached verification with full recomputation.

Quality metrics remain unchanged when proof reuse is removed (Table 6). Verification-path P99 falls from 1617 ms cold to 887 ms with reuse and a warm runtime, then to 634 ms with speculative preparation (Table 9): a 60.8% reduction for the combined configuration. Across 600 dependency-mutation trials, cached-path decisions matched full recomputation in 600/600 cases, with correct invalidation of affected entries.

(2) Construction of graph-linked release obligations. Within a typed task-family skeleton, the compiler constructs G=(V,Efwd,Ecomp)G=(V,E^{\mathrm{fwd}},E^{\mathrm{comp}}), instantiates mandatory obligations, and validates cross-field references. LCR@5 reaches 97.6%, versus 68.5% for direct JSON generation (Table 7). Recovery declarations and their represented prerequisites become explicit release obligations. In Appendix G, Case 2, a parallel refactor removes a test file required by a later staging action's compensation path. PACT checks that graph-linked prerequisite in current state and withholds the staging effect; Section 7, Eq. 29 makes this release check explicit.

The one-at-a-time ablations have distinct signatures: removing live revalidation raises UAR from 1.8% to 6.9%; removing compensation edges lowers recovery from 79.4% to 38.5% and GSR from 49.3% to 22.8%; removing bounded safe downgrade lowers Task from 63.5% to 58.7%. The accountability map Ω\Omega connects failed obligations to evidence and adjudication routes, reducing operator diagnosis from 13.8 to 5.9 minutes and raising blame localization from 43.6% to 87.4% against audit-only evidence (Table 8).

(3) Advantage over the strengthened alternative. We evaluated AgentSpec-style rules plus a separate runtime-verifier invocation for judgment-dependent actions and commit-time revalidation. Task/UAR/VRR/Recovery/GSR are 62.9/5.6/81.3/58.2/40.6%, compared with PACT-full's 63.5/1.8/95.1/79.4/49.3%. Paired differences (PACT minus baseline) are -3.8 points in UAR, 95% CI [-5.4,-2.2], and +8.7 points in GSR [+5.2,+12.3]. The advantage persists with separate verification and fresh-state checks present in the comparison.

Section 2 distinguishes PACT from AgentSpec/Progent, ABC/PCAA, PCC, and transaction/recovery systems by their verification objects and enforcement focus. Table 1 documents matched policy/state access, verifier roles, and commit-time revalidation, alongside graph-linked recovery and dependency-valid evidence.

W2. Practical assurance and applicability beyond SWE.

The formal assumptions identify three auditable interfaces: policy coverage, verifier decisions, and check-to-commit binding. Under these assumptions, Theorem 1 establishes compositional release soundness. For prel>0p_{\mathrm{rel}}>0, Corollary 1 aligns their failure probabilities with UAR's released-action denominator:

Pr⁡[unsafe∣release]≤min⁡{1,(ϵcov+ϵver+ϵbind)/prel}.\Pr[\mathrm{unsafe}\mid\mathrm{release}]\leq\min\{1,(\epsilon_{\mathrm{cov}}+\epsilon_{\mathrm{ver}}+\epsilon_{\mathrm{bind}})/p_{\mathrm{rel}}\}.

Structural closure validates mandatory-obligation instantiation. The residual audit compares the task-card oracle, compiled predicates, verifier outcomes, and executed binding; all 66 residual unsafe releases are classified as coverage omissions across the five Table 12 classes. Proposition 1 guarantees release-set inclusion under valid refinement at a fixed epoch on the common action domain. Binding stress tests used 500 injections (100 per adapter class) with no erroneous release observed.

For cross-domain evaluation, we augmented 60 retail/airline τ\tau-bench tasks with policy task cards and action-level safety oracles. Two matched harness runs per task give 120 episodes per method. Action-level UAR falls from ReAct's 16.7% to PACT's 2.5%; episode-level Task is 70/120 (58.3%) and 69/120 (57.5%), respectively. The revised paper reports the cross-domain protocol/results in Appendix K and Table 15, and the strengthened comparison with intervals in Table 2. We would appreciate reconsideration of your assessment and score in light of this evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.