- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this paper, authors propose, FormalClaw, which turns an agent hand-off into a contract containing local obligations and evidence requirements. Each obligation is assigned to a trace check, executable monitor, SMT solver, or proof kernel. Failed checks produce clause-level feedback for another execution attempt. The paper evaluates this interface in human-agent delegation, heterogeneous multi-agent communication, and iterative refinement, with additional safety and verification-tier studies.
Critical strengths
-
The paper addresses a critical challenge in LLM-based agent systems, which is particularly relevant as agents increasingly perform long-horizon tasks involving tool use and delegation.
-
The proposed four-tier verification architecture provides a practical compromise between informal natural-language supervision and expensive end-to-end formal verification, making the framework applicable to a broader range of agent tasks.
-
The paper evaluates the proposed framework in human-agent delegation, heterogeneous multi-agent communication, and iterative agent refinement. The experiments cover multiple model families and sizes and provide evidence of the framework’s general applicability.
Critical weaknesses
-
The effectiveness of FormalClaw heavily depends on the correctness and completeness of the automatically generated contracts. If the LLM produces incomplete or incorrect specifications, the verifier may accept unreliable outputs. The paper does not sufficiently investigate how specification-generation errors affect overall reliability.
-
Most experiments are conducted on benchmark environments with relatively well-defined tasks and evaluation criteria. It remain unclear how effectively the framework can handle highly ambiguous, dynamic, or open-ended real-world agent interactions where constructing reliable verification contracts is more challenging.
-
The ablation studies show relatively modest improvements from the higher verification tiers, and so I am concerned about their necessity and cost-effectiveness.
Questions for authors
Please help answer my concerns in the weakness part. Thank you!
2 comments
Response to Reviewer CUWw [2/2]
W2: How does FormalClaw handle ambiguous, dynamic, or open-ended real-world tasks?
We agree this is the main open problem and list it as the primary limitation. Two new studies move closer to it.
- ClawEval (300 open-ended tasks graded by an LLM judge) and ClawArena (multi-session, stateful scenarios) were run with no benchmark feedback at all and an equal budget. Contracts written from the task text give a selective retry trigger. It stays within the CI of the best retry baseline while using ~40% fewer attempts (1.7–1.8 vs. 2.3–3.0) and less wall time.
- On AgentDojo, contracts written by GPT-5.4 from the task text alone lower ASR by 14–48 points for the Qwen executors. They block 7–9% of benign tasks, against 40–47% for Progent.
W3: Higher tiers give modest gains; are they necessary and cost-effective?
Usage. On SCAM the contracts contain 102 T1, 2 T2, 28 T3, and 0 T4 obligations, so most obligations use the cheap tiers by design. Tier assignment is per obligation, and a higher tier is invoked only when a property requires it.
Matched T3 versus T4 on SCAM. Re-checking the same obligations with T4 gives the same verdicts (+0.002 [−0.038, 0.045] for 4B, +0.003 [−0.050, 0.076] for 9B). SCAM's properties are single-agent and finite-trace, so the modest increments in Table 4 are expected there.
Where higher tiers are necessary. Properties that span hand-offs need them; an example is "no secret egress across a delegation chain".
| Checker | Detects cross-hand-off egress (depth 1–16) | Cost per check | Encoding size |
|---|---|---|---|
| Local check (T2) | No (balanced accuracy 0.5 at every depth) | — | — |
| Fixed-horizon SMT | Only within its horizon | — | — |
| Adaptive SMT (T3) | Yes (1.0) | 19–310 ms | 6 → 182 KB |
| Kernel-checked proof (T4) | Yes (1.0) | ~1.3 s | 0.5 → 2 KB instances of one reusable theorem |
T4 also carries a soundness theorem linking the executable checker to the declarative property. In a mutation test, it rejected all 6 mutated checkers that accepted the attack.
The revision therefore presents the tiers as a cost/assurance menu: T1/T2 by default, T3 for finite-trace data-flow properties, and T4 when a reusable, kernel-checked semantics is worth ~1 s per check.
We hope these responses address the reviewer's concerns and welcome further questions.
Response to Reviewer CUWw [1/2]
We thank the reviewer for the constructive review. We ran new experiments on each of the three concerns. (Wi denotes weakness i.)
W1: FormalClaw depends on the correctness and completeness of generated contracts; how do specification errors affect reliability?
We measured this directly in four ways.
1. Contracts without explicit constraints. In the main SCAM runs, contracts include SCAM's per-task safety constraints as T1 obligations, as a user stating their constraints would. We then had GPT-5.4 write each contract from the task text alone (SCAM native scorer, 30 tasks, acc. = acceptance rate):
Incomplete contracts reduce the benefit but do not remove it: derived-only contracts still improve safety by 20–42 points over Direct, and explicit constraints add a further 18–24.
2. Coverage audit. Of SCAM's 46 per-task constraints, derived-only contracts forbid the same tool call in 7 (15.2%), constrain the tool in 26 (56.5%; for example, as a secret-egress sink or through argument checks), and miss 13 (28.3%). The missed constraints account for the gap in the table above.
3. Under-specified obligations. An incomplete contract shows up as a run the gate accepts but the benchmark scores unsafe:
Entries are unsafe/accepted.
4. Over-strict obligations. A wrong or over-strict obligation shows up as a false block. On AgentDojo, with contracts written from the task text, false blocks are 7–9% (9/97, 8/97, 7/97, 7/97 for the four executors).
In short, the verifier faithfully enforces what the contract says, and contract coverage is the dominant source of error. The revision states the guarantee precisely: a passing verdict certifies the encoded obligations over the abstracted trace, not that the contract captures the user's intent. It also lists contract coverage as the main limitation and reports acceptance, false-block, and accepted-but-unsafe rates next to every safety number.