acceptodds
Under review as a conference paper at ICLR 2027

AgentLTL: A Trace-Verification Framework for Measuring, Enforcing, and Training Procedural Compliance in Tool-Using LLM Agents

Abstract

Tool-using LLM agents are typically evaluated by final-answer correctness or by LLM judges. Neither captures how an answer was produced, yet in safety-critical settings the procedure is itself part of correctness. In this paper, we introduce AgentLTL, a specification language derived from first-order linear temporal logic over finite traces (FO-LTL), which expresses procedural rules over agent traces and yields a deterministic, judge-free compliance score. A single specification serves two purposes. In harnessing, constraints either score completed traces offline or gate tool calls online, by checking each prefix before execution. In finetuning, the same compliance signal acts as a dense reward. We evaluate both on our novel dedicated benchmark ProcFlow covering ordering, branching, iteration, and grounding, and on other several established tool-calling benchmarks. Online harnessing raises procedural compliance across models without degrading task performance, and finetuning with compliance-aware rewards improves held-out performance on ProcFlow while also increasing accuracy over the base model on every external benchmark we evaluate. Together, these results show that compliance is a transferable measure of procedural structure: it generalizes beyond the procedures seen during training and can be leveraged directly for efficient runtime enforcement.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
LLM agents, tool-using agents, formal methods, temporal logic, runtime verification, AI safety, procedural compliance, process supervision, neurosymbolic AI, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ZeTYOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper discusses how task completion is difference than "compliance", ie if an agent followed the correct procedure to get to its final answer or not. The authors propose a specification language based on first order LTL to encode compliance constraints. Given a task and its compliance constraints, the authors define a compliance score of an agent's tool call trace on this task as the fraction of satisfied constraints. The constraints are created with an autoformalization pipeline, beginning from the natural language task description, then an LLM creates a procedural graph, and from this the constraints are deterministically compiled. The authors then use this for two purposes. First, online harnessing of an agent trying to complete a task, with a guard blocking tool calls that would violate constraints. Second, the compliance score is used as a finetuning signal. The authors conduct experiments on their own specialized benchmark and a few others, and show that compliance differs from accuracy, their harnessing can improve compliance with small accuracy loss, and finetuning can transfer across benchmarks.

Critical strengths
  • The authors' specialized benchmark is generated programmatically (task descriptions, procedural graphs, constraints, correct solutions) with no LLM involved. This gives a good finetuning training setting, and the authors show that it can improve accuracy on other benchmarks.

  • The differentiation between compliance and accuracy is clear.

  • The constraint language is expressive and can capture temporal and provenance dependencies, while also being deterministically compilable from a graph.

  • The authors experiment with different finetuning rewards and harnessing enforcement settings, which is nice.

Critical weaknesses
  • The procedural graph is task specific, meaning that for each new task (ie benchmark question) one has to autoformalize a new one. This can be costly and impractical especially for online harnessing, since the autoformalization approach uses a multi-stage LLM pipeline. There is no latency or cost analysis for the autoformalization step. It would be important to report the time and cost (in tokens and LLM calls) of the method, and also of the compared baseline.

  • The compliance score is based on the same constraints that the online harness is enforcing, which may confound the results especially in the benchmarks where the constraints are LLM generated. As the authors mention, there is no guarantee that they are correct, but the paper does not analyse the robustness of the produced specifications. This is especially important when the task is not well specified ie if the natural language is a bit ambiguous or vague. I can think of several ways to improve this part of the paper. First, you could try multiple independent iterations of your autoformalization pipeline per task, to see how much the constraints vary per iteration. Second, you only tried one LLM for this, you should try at least one more LLM to see how much the results are translator-dependent. Third, try using the constraints of one autoformalization instance for the harness and the constraints from a different autoformalization instance for the evaluation.

  • The comparison with related work is not clear enough, mainly regarding runtime guards and NL-to-LTL, and the experiments lack meaningful baselines. The paper claims their novelty related to runtime guards is that "AgentLTL instead scores completed traces..." but it is also used for harnessing, so the authors should include a more detailed comparison with the other runtime guards they cite, especially Agent-C and AgentSpec, and potentially also [1,2]. The paper is not limited to harnessing but it is one of the main contributions so it should be properly differentiated. Further, while the authors cite some runtime guards, they only compare empirically with a single constraint-free guard and none of the more related ones. It would add to the paper to compare with more baselines in their experiments or explain why they are not applicable. Regarding NL-to-LTL, it would help to compare what types of constraints or tasks can be expressed by prior work and what does your language additionally cover or how it differs, otherwise why not use existing translation techniques? Finally, there is a recent benchmark for testing trace-level compliance using LLM-generated constraint-based rules that might be relevant to you on revision [3].

  • The compliance score being the fraction of satisfied constraints does not seem to be a very discriminative metric, or the choice of tasks are very easy for the models evaluated. The base models are very good to begin with (0.8+), and there is not much separation between different models and harnesses. The tables report no statistical uncertainty so it is unclear whether these differences are due to sampling noise. I suggest (i) reporting standard error in every table, (ii) do not average different models and benchmarks together in the main paper, just pick a few if there are space constraints, (iii) consider a binary compliance score where success means all constraints were satisfied, this makes more sense for evaluation, but keep your fine-grained fraction score for finetuning. I also suggest trying either weaker models or creating more difficult ProcFlow tasks.

[1] Hoover et al. "Dynaguard: A dynamic guardian model with user-defined policies", ICLR 2026

[2] Shi et al. "Progent: Securing AI Agents with Privilege Control", arXiv 2025

[3] Anand et al. "MANTRA: Synthesizing SMT-Validated Compliance Benchmarks for Tool-Using LLM Agents", arXiv 2026

Questions for authors
  • See weaknesses: cost analysis, autoformalization robustness, related work comparison, statistical uncertainty.

  • You say that sometimes the benchmark graders disagree with your specifications. How do you know if the graders are incorrect, and not your autoformalization was incorrect? Did you check? Can you include specific examples in the paper?

  • Can you report compliance scores on the other benchmarks after finetuning? Currently you only report accuracy. It would help to add compliance to substantiate claims of transferability.

  • how do you create the "finite domain of constants" mentioned in line 200? In the appendix you mention a domain extractor but can you explain how it works? Does it use an LLM?

  • How did you translate tau bench tasks into specifications? Since it includes user interactions, how do you use these user messages and any info that they give?

  • Is the FO-LTL language more expressive than the DAG? If yes, what kind of procedures or constraints cannot be expressed by your DAG? Could you give some insight on the limitations of its expressivity?

  • Several citations are incomplete (lacking venues or years). Please fix. For example tau^2 bench, GinSign, the LTLf citation.

  • You say compliance has limited effect on accuracy, but each of the enforcement settings reduces accuracy in some cases. Do you have any thoughts on how to choose between enforcement settings in new domains?

  • What is the difference between "at_call" and "on evidence"? Is there a particular benefit to separating these, since both are enforceable?

  • minor: you claim in 240 to have step-wise reward, but in your explanations the reward is only given at the end, not during the trace after each step, so please reword.

  • minor: define intervention-to-failure ratio in the main (line 372)

3 comments

Thank you for a thorough review and concrete suggestions.

W1: Cost. We measured cost with Qwen3.6-27B on 30 tasks per benchmark.

Induction of the procedure graph runs once per task, offline, and the result is reused across all models and settings:

BenchmarkLLM callsPrompt tokensCompletion tokensTime (median)
BFCL21.215,7411,55128.6 s
τ-bench16.024,6131,60322.3 s
WorkBench17.18,4251,02317.3 s
MedAgentBench11.87,8901,05316.3 s

Enforcement makes no LLM calls beyond the agent's own retries after a block. One check takes 0.1–0.15 ms on BFCL and MedAgentBench, and 3.8 ms on average on τ-bench.

The Reviewed setting makes 3–9 LLM calls per episode. On BFCL, this is about 18k prompt tokens and 28 s per episode. This is similar to the cost of inducing one BFCL specification, but the Reviewed setting incurs this cost again for every episode, model and run, while induction is paid once per task.

W2: Robustness and circularity. Please see the general response. We will add a robustness study across repeated runs and different LLMs. Note that accuracy is always measured by each benchmark's own grader, which never sees the specification. Circularity therefore affects the external compliance numbers, not accuracy.

W3: Related work and baselines. We will expand Section 2. Agent-C defines temporal constraints in a DSL, checks them with SMT, and uses constrained decoding to replace a non-compliant tool call with a compliant one. AgentSpec enforces rules made of triggers, predicates and enforcement actions, written by hand or generated by an LLM. Progent restricts which tool calls an agent may make, using JSON-based privilege policies. DynaGuard is a guardian model that judges whether a conversation complies with user-defined policies. MANTRA generates trace-level compliance checks from procedural manuals and validates them against an independently generated world model using SMT; it is a benchmark-generation method, not an enforcement or training method.

The prefix-gating algorithm itself is standard and close to these runtime systems. Our claim concerns one specification shared across offline scoring, online gating and training. This includes constraints that cannot be enforced before execution, such as required calls, which we score offline and use as a reward. To our knowledge, none of the cited systems uses its specification as a training signal for the agent. The procedure graph is a supporting component: its role is to keep the LLM out of formula generation when we obtain specifications for existing benchmarks. We will also fix the sentence "instead scores completed traces," which is inaccurate given our own harness.

We did not run Agent-C or AgentSpec as baselines because each requires specifications written in its own language. A comparison would therefore measure the specifications as much as the mechanisms. We will state this in Section 4.2.

W4: Metric and uncertainty. We agree with all three suggestions. We will report binary full-compliance rates, standard errors over the three ProcFlow trials, and bootstrap confidence intervals on the external benchmarks. We will show per-model results in the main text instead of averages. The fraction score is compressed because easy constraints, such as literals stated in the prompt, make up much of the count; the binary metric addresses this.

Two existing results show that the signal discriminates when the specification is reliable. ProcFlow already separates models widely: accuracy ranges from 14.2% (Qwen3-4B) to 99.2% (DeepSeek), and compliance from .839 to .995 (Tables 11–12). On ProcFlow, a constraint is 9.5× more likely to be violated on a wrong episode than on a correct one, and only 1.26% of constraints are violated on correct episodes (Table 13). One family is an exception: source is violated only on correct episodes (3.69%) and never blocks; we will discuss it in the appendix.

Q2: Grader vs. induction errors. We did not verify this, and our text overstates it. Please see the general response. We will include examples in the appendix.

Q3: Compliance after finetuning. We will rescore the finetuned models' traces and report compliance alongside accuracy.

Q4: Domain of constants. The domain is extracted deterministically from the trace (tool names, arguments and returned values seen during execution), the constants of the specification and, where needed, the tool schemas. Quantification can be restricted with extractor functions, for example to the outputs of one tool. These extractors are written by hand. No LLM is involved. We will make this explicit in Section 3.2 and Appendix A.

(to be continued)

(continued)

Q5: τ-bench. Each τ-bench task has a hidden instruction that the simulated user follows, for example: user id mia_li_3668, direct flight New York to Seattle on May 20, after 11am, cheapest, economy, 3 bags, no insurance. Induction reads this instruction, the domain policy (the agent's system prompt) and the tool schemas. It does not see the conversation, the gold actions or the reward; a test in our repository fails if it does. Using the instruction is allowed because the user states these goals to the agent during the conversation. The agent, however, learns them progressively, whereas the specification holds them from the start.

The model splits the instruction into steps, for example "search direct flights from NY to Seattle" and "book in economy with 3 bags." Code then builds the graph and the constraints: orderings (look up the user before booking), stated values (cabin, bags; checked to appear in the instruction), provenance (the booked flight id came from a search), and counts (book once). Compliance is computed on the agent's tool calls only; user messages are never scored. The simulated user is the same model as the agent.

Two limitations. Steps like "tell the user X" have no tool call, so they produce no constraints; 15 of 165 tasks have no constraints at all. These are excluded from the harness experiments, which is why the harness uses 150 τ-bench tasks and finetuning uses 165. Also, policy rules such as "confirm before booking" are mostly not yet turned into constraints. We will state both in the paper.

Q6: DAG vs. FO-LTLf. Yes, the language is more expressive than what we compile. The compiler emits presence, argument values, call counts, ordering, prohibition and provenance. The language can also express bounded response (WithinSteps), pairing obligations (every opened file is eventually closed), and conditions scoped by Until. Since the graph is acyclic, loops are captured only through call counts, so a rule like "repeat until the total reaches the target" is approximated, not enforced exactly. We will add this to the limitations.

Q7: Citations. We will fix them.

Q8: Choosing an enforcement mode. Each constraint has its own severity: log, warn or block (Section 3.4). The right mode depends on the task's constraints. Constraints on values stated in the prompt or fixed by the schema can block; constraints based on inferred dependencies should only warn. If one mode must be used for everything with induced specifications, we recommend block-and-warn, since it allows overrides. Per-constraint modes are supported but not evaluated in the paper, and we will say so.

Q9: at_call vs. on_evidence. An at_call constraint can be checked from the proposed call and earlier calls, for example a fixed argument value or a call-count limit. An on_evidence constraint needs an earlier result, for example "tool_c's argument must come from tool_a_list's output." The harness only checks it once that result exists, so it never blocks on something it cannot yet decide. We will clarify this in Section 3.4.

Q10–Q11. Agreed. We will write "per-constraint partial credit" instead of "step-wise reward," and define the intervention-to-failure ratio in the main text.

Thank you for your detailed response that addressed some of my questions. I will wait for the robustness results, the uncertainty estimates and compliance scores of finetuned models. However, I still have remaining concerns that I believe require non trivial effort to address, which I describe below.

The cost measurements and clarifications are helpful, but I suggest adding uncertainty estimates to the costs instead of only reporting averages or medians. Also, report the cost/latency of the reviewed baseline on all the benchmarks instead of just BFCL. I do agree that for offline evaluation, there is clear benefit to reusing the specification, but for online harnessing, each new task would require generating the specification and the cost/latency/performance is similar to the reviewed baseline in BFCL. It should be mentioned in the limitations that the amortization benefit of your method depends on reusing the same task and this limits its cost advantage in deployment. Further, you clarified that on tau-bench the specification generation uses the simulated user’s full hidden instruction, which is impossible in deployment. I appreciate your commitment to acknowledge this limitation, but I do find it important and think that the exact scope of the online harnessing setting should be made more clear in the paper.

Regarding specification correctness and interpretation of compliance on the benchmarks, I understand that it is not an issue on ProcFlow, but its tasks are abstract and do not represent realistic usecases. In the external benchmarks the issue remains and you acknowledge that disagreements could reflect either grader errors or specification errors. The planned robustness study is a useful first step and I will wait for its results, and for future versions consider evaluating compliance using independently constructed checks such as cross-model experiments or the MANTRA benchmark. Further, while releasing the specifications is helpful for transparency, the paper should include an assessment of their correctness rather than leave this to reviewers. Even a small audit of a representative sample would give some insight.

Regarding novelty and related work, you claim as your contribution that you use a specification for offline scoring, online gating and training. In this case, I believe it is important to compare against methods using alternative specification languages, since it is part of your contribution. AgentC, AgentSpec, Progent and DynaGuard all seem very relevant methods, and I strongly disagree that a comparison that depends on the quality of different specifications would be uninformative. Additionally, my question about how your specification language and translation pipeline differ qualitatively from prior NL-to-LTL approaches remains. Could existing specification languages also support offline/online scoring and training with minor adaptations? And does your translation express constraints that existing NL-to-LTL approaches cannot?

Reviewer_ckazOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents AgentLTL, a fragment of first-order LTL over finite tool-call traces. A compiler turns a procedure DAG into weighted constraints. The satisfied fraction of the weight gives a judge-free compliance score. The same constraints score traces offline, block safety violations online, and act as a GRPO reward. The authors build ProcFlow, a set of 12 synthetic arithmetic templates. They use Qwen3.6-27B to induce specifications for BFCL, τ-bench, WorkBench, and MedAgentBench. Online blocking raises mean compliance from 0.885 to about 0.94 across six models. GRPO on Qwen3-4B with 300 examples raises held-out ProcFlow accuracy and gives small gains on the four external benchmarks.

Critical strengths
  1. The design keeps the LLM out of formula writing. The model fills node-local fields, and deterministic code emits the edges and formulas. Each constraint is therefore inspectable and traces back to a node or an edge. The safety and liveness split decides which constraints can block a call. These properties make the framework a sound and reusable tool for agent audits.

  2. The failure-class analysis gives a clear negative result. Online blocking cannot repair missing calls, and presence constraints block 3 times in 10,787 ProcFlow episodes. GRPO recovers 74 to 89 pp on refusable and mixed failures. This result shows where enforcement stops and where training must start.

Critical weaknesses
  1. The induced specifications have no human validation, and the paper’s own data suggest many errors. -> On the external benchmarks, constraints fire 1.9x more often on wrong traces than on correct ones, and 5.9% of constraints fire on correct traces. -> 84% of blocks on solved episodes come from constraints that the correct trace violates. Solved pairs with a block lose 41.4 pp. -> The authors read this as lax graders, but specification error gives the same pattern. -> The accuracy claim uses the best of three settings per cell. Soft-block loses 7.5 pp on τ-bench, and 14.7 pp for DeepSeek on τ-bench.

  2. The finetuning results do not isolate the compliance reward.

-> The gold-trace reward g matches or beats the compliance reward c on four of five ProcFlow splits, and it transfers to a similar degree. -> The main model c+a+g is the weakest GRPO variant, at 57.2% on Full against 85.6 to 86.9% for the others. -> The GRPO versus SFT gap compares RL to imitation. It gives no evidence for compliance as a reward. -> One 4B model and small external gains weaken the transfer claim. Outside WorkBench the gains are 1 to 3 pp, and the τ-bench standard deviation is 3.5.

Questions for authors

What is the human-judged precision of the induced constraints on a sample of external tasks? What accuracy does GRPO with the answer reward alone give, and does the addition of c improve on it? Why does c+a+g score 57.2% on Full while c+a, c, and g reach 85.6 to 86.9%? Which enforcement setting do you recommend before you see the results, and what is its accuracy change on each benchmark? How much of the +22 pp MedAgentBench gain comes from repair messages that state schema-prescribed values?

4 comments

Thank you for a precise review. Several points are correct, and we will revise the paper.

W1: Induced specifications (and Q1). We agree that Table 17 and the 84.3% figure fit specification errors as well as lax graders, and we will remove our interpretation favoring graders. We also acknowledge the figure you cite: solved pairs with a block lose 41.4 points, against 6.3 without one. We have no human-judged precision to report: we could not run a human audit within the rebuttal period, and we will list this as a limitation. Please see the general response for what limits the concern (released, traceable specifications; reliability that tracks value provenance) and for the robustness study we will add.

W1 / Q4: Best-of-three reporting and recommended setting. We agree that reporting the best of three settings per cell is selection after seeing results, and we will remove those sentences from Section 5.1. We will report block-and-warn as the default. This choice is made after seeing the results, and it is also the setting with the best mean accuracy change on the established benchmarks (+5.7). We justify it by a criterion that does not depend on the results: it is the only setting that lets the agent override a block, which bounds the damage from a wrong constraint. All settings will be reported in full.

For block-and-warn, the average accuracy changes are +0.5 (BFCL), −2.1 (τ-bench), +0.9 (WorkBench), +23.5 (MedAgentBench) and +2.9 (ProcFlow). It still breaks solved episodes: among baseline-correct instances with at least one block, it fails 23.9% on BFCL, 39.6% on τ-bench, 18.6% on WorkBench and 20.3% on MedAgentBench (Table 15). We will also report the soft-block losses you note (−7.5 on τ-bench on average, −14.7 for DeepSeek), and correct the abstract's claim of "without degrading task performance."

We should also have been clearer that the framework does not need one mode per run. Each constraint has its own severity: log, warn or block (Section 3.4). The right mode depends on the task's constraints, not the benchmark. Constraints on values stated in the prompt or fixed by the tool schema can block, while constraints based on inferred dependencies should only warn. In the paper we applied each mode uniformly to compare them. Per-constraint modes are supported but not evaluated, and we will say so.

Q5: MedAgentBench and repair messages. Fair point. On MedAgentBench, most bind constraints check values fixed by the tool schema. The agent already sees this schema, so no hidden information is added, but the repair message does point the agent to these values. The gain is best read as enforcing the schema, not as a general procedural effect, and we will say so in Section 6. We will report how many MedAgentBench fixes come from bind blocks on schema values.

W2: Finetuning (and Q2, Q3). We agree that GRPO vs. SFT does not isolate the compliance reward. We are running GRPO with the answer reward alone, and will post c+a vs. a in a follow-up comment.

On c+a+g scoring 57.2% on Full: the existing tables locate the failure. c+a+g is the only finetuned model whose argument-binding pass rate falls below the base model (.796 vs. .974, Table 20), so its errors are wrong literal arguments rather than wrong procedures. They concentrate on three templates: nested_loops+branching (0.0%), fan_out+loop (16.7%) and nested_loops (20.0%) (Table 18). We will no longer present c+a+g as the main model. We will present c+a as the main model and c+a+g as a variant that is best only on WorkBench (36.0%).

On transfer, we will restate the claims per benchmark:

  • WorkBench is the clearest case: all four GRPO variants improve over the base model (+3.2 to +9.2), while all three SFT models fall below it.
  • MedAgentBench: GRPO gains are small (+1.3 to +2.7), while SFT loses 7.7 to 12.8 points.
  • BFCL: GRPO gains are consistent (+2.8 to +3.5), but SFT3 is the best model there (28.6%), so we do not attribute this gain to the compliance reward.
  • τ-bench: differences are within noise (base sd 3.5), as you note.

The standard deviations in Table 21 reflect pipeline nondeterminism only, since replicates share a seed; they are not sampling uncertainty over instances. We will add bootstrap confidence intervals over instances. We will also state that transfer evidence comes from one 4B model, and replace the abstract's claim of gains "on every external benchmark" (see the general response).

We hope these revisions address your main concerns.

Our follow-up comment under the general response reports the answer-only GRPO control (your Q2): adding the compliance term improves over answer-only GRPO on all nine evaluations, and answer-only GRPO falls below the base model on all four external benchmarks. On c+a+g (your Q3), we do not yet know the cause and are investigating; we will present c+a as the main model.

Reviewer_ckazOpenReview ↗

Response.

Thankyou for the response. I will maintain my score, because the induced specifications still have no measured precision, and you agree that the 84.3% figure fits specification errors as well as lax graders, so it remains unclear whether the compliance scores, blocks and GRPO reward track the procedure or errors in the specifications.

We thank you for your comment. We would just like to clarify that the GRPO training does not use induced specifications, but deterministic ones. Induction happens only for NL to LTL translation. In the case of ProcFlow, we started from the procedure graph, not from a natural language specification. The finetuning results are not concerned by possible errors in specification. At training time, constraints are deterministic and not induced, and evaluation time we do not use our harness but only the finetuned models. We hope this helps clarify and localize the possible error sources.

Reviewer_d1UMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces AGENT LTL, a trace-verification framework for evaluating procedural compliance in tool-using LLM agents. It represents natural-language procedures as structured graphs and compiles them into temporal-logic constraints over tool usage, including call ordering, argument binding, branching, call counts, and value provenance. The resulting compliance score is used for offline evaluation, online interception of non-compliant tool calls, and reinforcement-learning-based finetuning. The authors also introduce ProcFlow, a synthetic benchmark covering diverse procedural patterns, and evaluate the framework across six models and four existing tool-use benchmarks. The experiments suggest that online enforcement generally increases measured compliance, while compliance-aware GRPO training improves performance on held-out procedures and external benchmarks.

Critical strengths

The paper addresses an important limitation of outcome-based agent evaluation by explicitly measuring whether an agent follows the required procedure, rather than considering only the correctness of its final answer. Its central idea of using a unified executable specification throughout the agent lifecycle is conceptually appealing. The procedure-graph representation provides a structured and auditable approach to compiling natural-language requirements into multiple constraints. The empirical evaluation is broad and informative, covering multiple models and benchmarks and offering detailed analyses by constraint type and failure mode. In particular, the paper clearly characterizes both the capabilities and limitations of runtime enforcement, including its effectiveness in preventing inadmissible actions and its inability to recover calls that an agent never proposes.

Critical weaknesses

The main weakness lies in validating the induced procedural specifications. On external benchmarks, specifications are generated using Qwen3.6-27B and checked automatically for structural consistency, but they are not compared with human-validated reference procedures. Rejecting some task-correct traces is expected because the framework deliberately imposes requirements beyond final-answer correctness. However, without independent semantic validation, it remains unclear whether these rejections reflect genuine procedural violations or errors in specification induction. This ambiguity limits the interpretation of the reported compliance gains.

Questions for authors

The paper makes a thoughtful and promising attempt to formalize procedural compliance in tool-using agents. Several questions remain for further discussion:

  1. How do the enforcement results change when individual constraint families—such as binding, ordering, or call-count constraints—are removed from the runtime specification? Reporting task accuracy, intervention frequency, and compliance against the original full specification could identify which constraints drive the gains and which contribute to the accuracy trade-off.

  2. Could the framework support iterative specification refinement, using intervention outcomes, audits by a stronger model, and selective human feedback to identify and revise inaccurate constraints over time?

  3. How would an answer-correctness-only GRPO baseline perform? Given that gold-trace-only training outperforms compliance-only training on the unseen-pattern split, this comparison could better isolate the contribution of the compliance reward and assess the robustness of the reported gains.

2 comments

Thank you for the positive review and useful questions.

W1: Specification validation. Please see the general response. We agree that without independent validation, we cannot separate real violations from induction errors. We will revise the discussion and add the robustness study.

Q1: Removing constraint families at runtime. This experiment is running; results will be posted in a follow-up comment during the discussion period. Existing counts already show which families block. On ProcFlow, call-count limits (bound) cause 95% of block-and-warn blocks, and bind, order and source never block (Table 13). On the external benchmarks, bind causes about half of all blocks, and presence constraints never block (Table 17). We will report accuracy, interventions and compliance with bind, bound and order each removed.

Q2: Iterative refinement. Yes, the design supports this. Procedure graphs are stored and compilation is deterministic, so a corrected graph can be recompiled and re-scored without new model calls. Constraints that block episodes the grader accepts are natural candidates for review. A suspect constraint can also be lowered from blocking to warning instead of removed. Refinement using grader outcomes must use a separate split from evaluation to avoid circularity. We will add this to the discussion.

Q3: Answer-only GRPO. This is running, and we will post results in a follow-up comment. One point holds independently of that control. The compliance reward c scores a trace against the specification, so it accepts any valid execution, whereas the gold-trace reward g rewards proximity to one reference trace. That c comes close to g (85.6 vs. 86.9 on Full, 87.2 vs. 94.9 on Unseen) while rewarding only procedural structure is the property we want from a compliance signal. We will state clearly that g matches or exceeds c on most splits.

We hope these revisions address your concerns.

Our follow-up comment under the general response reports the answer-only GRPO baseline (your Q3). Adding the compliance term to the answer reward improves accuracy on all nine evaluations, and answer-only GRPO falls below the base model on all four external benchmarks. The constraint-family ablation (your Q1) is still running.

Comments on the paper

2 comments

We thank the reviewers for their careful reviews. Two concerns are shared, so we address them here.

What the concerns do not affect. Three parts of the paper are independent of the induced specifications:

  • ProcFlow specifications are built by code, with no LLM. All ProcFlow results use specifications that are correct by construction.
  • Finetuning uses only ProcFlow. Transfer is measured with each benchmark's own unmodified grader, so induced specifications play no role in training or in evaluating the finetuned models.
  • Induction never sees gold traces, answers or graders. Specification errors can lower accuracy under enforcement, but cannot leak answers. (On τ-bench, induction reads the user instruction that the simulated user follows; see our answer to Reviewer ZeTY, Q5.)

(1) Validity of induced specifications. We agree this is the main limitation of our external-benchmark harness results, i.e. the external rows of Table 1. We also agree that we over-interpreted one statistic. When 84.3% of blocks on solved episodes come from constraints that the correct trace also violates, this can mean either a lax grader or a wrong specification. The paper cannot tell these apart. We will remove the sentence "indicating that answer-based graders accept traces that do not follow the procedure", revise Sections 5.1 and 6, and harmonize the figure (the main text says 85%, the appendix 84.3%).

We also state the cost plainly: solved (instance, setting) pairs with at least one block lose 41.4 points, against 6.3 for pairs without one. This is what a wrong or over-strict constraint costs under enforcement.

Three elements limit this concern and make it auditable:

  • The LLM never writes a formula or an edge. It fills node-local fields, each checkable against the task text or a tool schema; code derives all relations and formulas. Induction errors are therefore local and traceable.
  • All induced specifications are released in the anonymous repository, and each constraint records the node or edge that produced it. Reviewers can inspect them directly.
  • Our existing data show that constraint reliability tracks the provenance of the asserted value. On MedAgentBench, where bind values are prescribed by the tool schema, a constraint is 9.9× more likely to be violated on a wrong baseline than on a correct one (1.2–1.8× on the other three benchmarks), and only 49.9% of blocks on solved episodes come from a constraint that the correct trace violates (81–94% elsewhere). This supports setting severity per constraint: block on schema- or prompt-stated values, warn on inferred dependencies.

We will add a robustness study: we run induction several times per task and with different LLMs, measure how much the specifications vary, and report how this affects compliance and accuracy. We could not run a human audit within the rebuttal period, and we will list this as a limitation.

(2) Isolating the compliance reward. We agree that GRPO vs. SFT also compares RL with imitation, so it does not isolate the compliance reward. We are running the requested controls, including GRPO with the answer reward alone, and will post the results in a follow-up comment during the discussion period.

AuthorsOpenReview ↗

Follow-up: isolating the compliance reward.

As promised, we report GRPO with the answer reward alone (a) and with answer plus gold-trace similarity (a+g), trained with the same configuration as all other variants.

Benchmark (n)Baseaa+ggcc+ac+a+g
Unaugmented (83)4.840.153.268.769.973.567.5
Augmented (100)12.022.740.761.058.067.046.0
Unseen (39)7.764.187.294.987.289.746.2
Diverse-tool (39)23.151.348.759.059.056.461.5
Full ProcFlow (360)13.431.976.186.985.686.957.2
BFCL (800)24.723.827.527.528.227.827.5
τ-bench (165)24.122.726.025.027.328.025.0
WorkBench (568)26.825.029.230.430.630.036.0
MedAgentBench (300)34.932.732.736.637.636.237.1
  1. The compliance term adds to the answer reward on every evaluation. c+a improves over a on all nine: by 25 to 55 points on the Unaugmented, Augmented, Unseen and Full splits, by 5.1 on Diverse-tool, and by 3.5 to 5.3 points on each external benchmark.
  2. Transfer requires a procedure-level reward, not RL alone. Answer-only GRPO falls below the base model on all four external benchmarks. Every variant that includes c or g improves on the base model on all four.
  3. Compliance combines with the answer reward better than gold-trace similarity does. c+a outperforms a+g on all nine evaluations, while needing only a specification instead of a reference trajectory per instance. Several margins are small (BFCL +0.3, WorkBench +0.8, Unseen +2.5 with n=39), so we claim consistency of direction, not significance.

g alone matches c+a on Full and exceeds it on Unseen. Compliance therefore matches trajectory supervision rather than surpassing it; its advantage is that it does not need gold traces.

We will present c+a as the main model. c+a+g remains an outlier: it scores 57.2% on Full, below both a+g (76.1%) and c+a (86.9%), although each of these rewards works on its own. We do not yet know why, and we are investigating. We will report it in the appendix as an unexplained result.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.