acceptodds
Under review as a conference paper at ICLR 2027

SymPolicyBench: Evaluating Symbolic Rule Extraction from LLM-Agent Prompts

Abstract

Enterprise agents are governed by prose instructions, and enforcing that governance independently of the agent’s own reasoning requires re-expressing it as a symbolic policy artifact—structured constraints over tools, parameters, and call counts that another component can read and apply, with a declared naturallanguage form for requirements no structured kind carries faithfully. Systems that synthesize such artifacts are now being built, but the artifact itself is rarely the unit of evaluation: agent-safety benchmarks score a trajectory under a guardrail that is fixed or absent, while synthesis work that does score policies scores them in one domain against one hand-authored target, so no two systems can be compared. We release SymPolicyBench, a corpus of 32 enterprise workflows across 8 domains with 543 golden intents traced to the clauses they derive from and 907 labelled decision points carrying untrusted conversational history. Ground truth is stated as intents rather than rules, so matching is many-to-many and a submission is never penalized for decomposing a requirement differently than the annotator did. Each policy is frozen and scored twice—statically against the intents it should encode, and behaviourally by simulating that exact policy, omissions included, on decision points the scorer sees unlabelled. Across 7 LLM-driven generators under one pinned judge, strict coverage spans 51.7–81.5% while balanced accuracy spans only 7.0 points and inverts at the top, the most faithful configuration obstructing the most legitimate work. Rescoring every frozen submission with a second-family judge and regenerating two configurations establishes which conclusions travel: the fidelity and false-allow orderings reproduce across judges (ρ = 0.82 and ρ = 0.96), and the fidelity–utility trade-off itself holds under a fixed scorer.

Primary area:
datasets and benchmarks
Keywords:
Agentic AI, Neurosymbolic AI, Agent Governance

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_zXxHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This work presents a benchmark for measuring how well LLMs generate symbolic policies from enterprise agent instructions written in prose, so that other systems can readily interpret them. The authors point out that existing benchmarks either evaluate only trajectories with the guardrail fixed or absent, or compare against a hand-crafted policy in a single domain, which has made comparison across synthesis systems difficult. The benchmark consists of 32 LLM-synthesized workflows across 8 domains, comprising system prompts, trust models, tool descriptions, 543 golden intents, and 907 labelled test cases. Generated policies are frozen and then evaluated from two perspectives, static fidelity and behavioural consequence: the former is measured by strict coverage and rule precision, and the latter by the false-allow rate (FAR) and false-block rate (FBR). The authors evaluate 7 models and point to a fidelity–utility trade-off, noting that models with high static fidelity also show high FBR. They further analyze that high FAR is mostly due to requirements missing from the policy, whereas high FBR stems less from rule generation itself than from when the rules are applied.

Critical strengths
  • By clearly identifying what existing benchmarks do not address, this paper targets LLMs' policy-generation capability itself, and its contribution lies in providing, for the first time, a public multi-domain shared resource that enables comparison across systems.
  • It measures LLM capability separately in terms of static fidelity and behavioural consequence, reporting missing rules as false allows and rule application as false blocks. In other words, it splits LLMs' policy-generation capability into two axes, coverage and utility, and reports that there is a trade-off between the two.
  • It explicitly states that a change of judge can affect behavioural consequence, and honestly reports, metric by metric, which are reliable (FAR) and which can fluctuate (FBR, BA).
Critical weaknesses

W1. I checked the supplementary material for the trust model, but could not find any information about it anywhere in the paper. Even if it is a commonly used term in the security field, please define its meaning in the paper. It also needs to be explained how it differs from the prompt and why the two are separated. The submitted policy in the figure is presented only as an abbreviated checklist, so the actual rule structure defined in Section 3 (governed target, description, and the four constraint kinds) cannot be seen anywhere in the main text. Including one rule example each for a structured constraint and a natural-language constraint, taken from an actual submission, would greatly help readers understand the contract.

Related to the above, there is no example of the actual reference prompt, so the explanation of how policies are generated is also insufficient. Overall, it is difficult to picture what the benchmark looks like. Examples of the model’s input and output are also lacking. Some content appears to be summarized only in Table 2 without being introduced in the main text, such as the concepts of system spread and run shift. To be precise, the numbers and concepts are described in the main text, but their names (system spread, etc.) and how they are computed appear only in the caption.

W2. The benchmark is described as fully synthetic, but the model used to synthesize it is not specified. The entire dataset is synthetic and there seems to be no human verification at all, which concerns me. Without an end-to-end evaluation, I wonder whether it is difficult to determine if these policies actually affect the safety of the system.

W3. For FBR, the results vary too much depending on the judge (especially for Opus). Since models tend to rate their own outputs more highly (self-preference bias), I think a non-overlapping judge should be used, or this issue should otherwise be addressed.

Minor:

The paper states that ST-WebAgentBench is the closest work in agent safety evaluation, but the explanation of why it is the closest is too brief. It should clarify in what respects the two are similar. Beyond the harmfulness-oriented works mentioned earlier, the paper should include more related work that measures policies. Figure 1 contains both the generation process and the two evaluation views in a single figure, making its information density excessively high. Although the evaluation boundary is marked, the generation stage and the evaluation stage are not visually separated enough, which makes it hard for first-time readers to follow the flow.

Note to authors and AC: The weaknesses have been condensed and grouped for clarity. This is an editorial change only — the content and my overall evaluation remain the same as in the original version.

Questions for authors

Q1. Are there no other reference benchmarks that measure policy guardrails? Is ST-WebAgentBench the only related work? There are also works such as SHIELDAGENT-BENCH; are they not related to this paper?

Q2. Is the trust model a concept that is commonly used explicitly? If so, is there a need to set up such a trust model under realistic conditions? I also wonder whether the trust model makes the problem easier, since it directly provides the constraints in natural language.

Q3. Are there any research results or measurements showing that creating policies actually makes agents safer?

Q4. Even if the system policy is wrong, are there cases where the model fills in the content with its own reasoning power?

Q5. If the benchmark was synthesized with one particular model, wouldn't that model have an advantage in scoring because of its similar distribution? I am also unsure whether the benchmark will remain applicable when next-generation models come out.

Q6. Regarding generation, the paper states that "temperature zero is not a determinism guarantee." I wonder why the authors did not run generation multiple times and average the results. Does the fact that the generator's outputs differed even at temperature 0 mean that reasoning was used? If so, the quality could vary depending on the reasoning effort used. Please also specify whether the judge used reasoning and what its reasoning effort was.

Minor

M1. Please make sure that no text is cut off in Figure 2.

M2. In particular, the paper should specify what kind of correlation coefficient ρ is. It appears to be Spearman's ρ; please confirm.

Reviewer_Vw7VOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces SymPolicyBench, which is a benchmark for evaluating how effectively LLMs translate natural language enterprise agent instructions into structured security policies. The benchmark consists of 32 model generated workflows across eight domains, with 543 annotated policy requirements and 907 behavioral test cases. The authors evaluate seven LLM configurations using both static evaluation of policy correctness and behavioral evaluation of whether generated policies appropriately allow or block actions. The results indicate a trade-off between policy fidelity and practical utility, showing that higher requirement coverage can potentially reduce security violations but may also lead to incorrectly blocking legitimate actions.

Critical strengths
  • The paper addresses an important and timely problem in AI safety, focusing on the challenge of translating natural-language instructions into structured security policies that can be independently evaluated and enforced.
  • The experiments evaluate a diverse set of seven model configurations from different model families. This provides a relatively broad empirical comparison and strengthens the analysis by examining whether the observed trends are consistent across different models.
Critical weaknesses
  • The paper is very hard to parse, due to overly convoluted sentences, forward references and excessive use of references to the appendix. This makes the overall content significantly less accessible to the reader.
  • The workflows and benchmark corpus are model generated, as the authors themselves state in the limitation section. Human validation or inter-human agreement of the workflows would strengthen the work.
  • The overall takeaway is somewhat unclear to me. The benchmark problems are model generated and do not appear to be validated against real-world enterprise workflows. This significantly limits the signal from the experiments, since LLM generated reasoning tasks have previously shown to be easier compared to human authored tasks [1]. Furthermore, while the authors report various metrics such as FAR, FBR and BA, these metrics are not sufficiently motivated and remain difficult to interpret. Most of the analysis following the experiments seems to be descriptive, while concrete, actionable insights to improve such workflows seem to be missing.
  • The token budget of the workflows is not ablated or tested. In Appendix B.4 the authors instead report that all models are capped at 32k output tokens.
  • The related work section seems to be missing highly relevant literature such as for example [2,3,4]. A broader discussion regarding the novelty of the proposed framework relative to these existing works is missing and one of most critical gaps in the manuscript.

[1] What Has Been Lost with Synthetic Evaluation?, Gill et al., 2025, https://arxiv.org/abs/2505.22830

[2] AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents, Wang et al., 2025, https://arxiv.org/abs/2503.18666

[3] Towards Enforcing Company Policy Adherence in Agentic Workflows, Zwerdling et al. 2025, https://arxiv.org/abs/2507.16459

[4] Extraction of Machine Enforceable ABAC Policies from Natural Language Text using LLM Knowledge Distillation, Yang et al., 2025, https://doi.org/10.1145/3734436.3734447

Questions for authors
  • Since the specifications, golden intents, and behavioral test cases are LLM-generated, have the authors conducted any independent human expert validation? How can the correctness of the benchmark labels beyond agreement between LLMs be established?
  • How does SymPolicyBench differ from recent work such as AgentSpec? What limitations of these works does the paper currently address?
Reviewer_qK5nOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper is concerned with evaluating LLMs in how well they can produce a security policy for agent tool calls from just a natural language description of a workflow. For this purpose it provides a synthetic (LLM-generated) dataset of workflows, each with (1) the natural language description of which actions are acceptable and which aren't in the style of a system prompt, (2) the tools available to the agent, (3) a list of "golden intents" that describe each individual requirement one-by-one, and (4) a list of cases in which a certain tool call must be judged, alongside the correct allow/block decision and its rationale. A policy for a workflow is a set of rules, which have to follow a schema defined by the authors.

The authors test 7 different LLMs on how well they can generate a policy from the just the system prompt and the tool specifications (1 and 2). The generated rules are tested statically on how well they match the ground-truth rules defined by the "golden intents" and behaviorally by the accuracy of the allow/block decisions for the given cases. The first is done with an LLM judge. The allow/block decision are derived from the policy through pattern matching if a rule has a structured form and again by an LLM if the rule is stated in natural language.

The dataset comprises 32 different workflows and 7 different LLMs are used to generate policies for each. The authors make the following findings. (RQ1) Between models the false allow rates and false block rates of the generated policies do not correlate. The model with the lowest false allow rate has the highest false block rate. This is surprising as one would expect a more capable model to make fewer mistakes in general. (RQ2) The intents that the models most struggled with were determined by their kind. In the dataset, each "golden intent"/requirement has a "binding kind" classification and an "expressibility label" associated with it. The expressibility label predicts how well the requirement can be translated into structured checks. The authors find whether the models were able to encode a certain requirement correlated much more with "binding kind" than with the "expressibility label". (RQ3) The false allow decisions are explained by missing or incomplete rules, but the false block decisions are not explained by superfluous/spurious rules. Instead false block decisions seem to stem from misspecified translation of a real requirement. (RQ4) Changing the LLM judge or rephrasing the input to the policy generator changes the results in different ways.

Critical strengths

The paper is comprehensive in its approach. I believe the authors went to great lengths to make the dataset cover different fields, different tools and requirements. They attempt to capture real scenarios and test 7 different LLMs for generating the policies. The experiments are structured with the goal of not giving any model an advantage or disadvantage. Lots of precise data on the experiments, model failures and ablations with a different judge and rewritten prompts are reported.

Critical weaknesses

The paper is barely readable because it is too dense and uses too many specialized words. The hallmarks of LLM-generated text are everywhere in this document. Every sentence is information-rich and careful not to overstate anything, but the structure makes understanding the setup for the experiments very difficult to follow. Sentences like “Cross-tabulation settles the confound.” and “Runs fail at two stages, and the protocol fixes three dispositions.” are examples of what I mean. The first is unnecessarily technical and doesn’t contribute anything. The second uses the words “run”, “stage”, “protocol” and “disposition” when none of these terms are properly clear in what they refer to. Only after making a table of the different parts of the dataset and the generated policies did I notice that “requirement” and “intent” basically refer to the same thing, namely ground truth, whereas “rule” is entirely different and refers to parts of the generated policies that are evaluated.

Some parts of the experimental setup seem flawed. The authors themselves notice that there is a “prose escape hatch” in that natural language rules can be part of a policy. In this case the responsibility of interpreting the rule shifts from the model generating the policy to the model executing the policy. The authors state that none of the models used this option to simply copy the policy prompt into its own rule and have the judge decide based on all of the context given, but the option exists as far as I can see. Moreover, some rules are basically unenforceable in the given setup. To quote the paper: “completeness constraints are hardest despite being conceptually simple, because a policy of per-action gates has no natural place to say an action must eventually occur”. Since a safety policy is only invoked during tool calls, the model has no option to enforce these completeness rules and they cannot be tested by example cases. A benchmark that cannot be solved, is fundamentally flawed.

Questions for authors

None of these questions/comments will materially affect my assessment. They came up during my reading and I suggest authors address them in a future draft.

  • How does the system disincentivize the policy generator from copying the prompt verbatim into a single natural language rule? How would such a policy fare in terms of fidelity/precision/FAR/FBR?
  • The text names the main judge as "openai/gpt-5.4" but the secondary judge is simply called "Judge B". What is judge B?
  • A value of ρ\rho is given when comparing rankings and judgements between different LLM judges, but the text did not say what exactly it measures and how it is defined.
  • The table show that a total of 216 valid policies have been generated (out of a possible 8*4*7 = 224), but later on the text mentions "197 frozen submissions of all 7 generators". Why is there a mismatch between these numbers?
  • Many of the reference cite arXiv versions of papers, when they have been published in journals or conferences. As one example, see https://proceedings.iclr.cc/paper_files/paper/2025/hash/5750f91d8fb9d5c02bd8ad2c3b44456b-Abstract-Conference.html
Reviewer_SP9FOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SymPolicyBench, a benchmark for evaluating synthesized policy artifacts: list of rules into which the prose instructions of enterprise agents are translated, where each rule carries constraints (tool binding, parameter constraint, call-count constraint, and natural-language constraint). The corpus comprises 32 workflows (four from each of eight enterprise domains), with 294 tool definitions, 543 golden intents, and 907 linked behavioral cases (623 attack cases for which block is correct, and 284 benign cases for which allow is correct). Ground truth is expressed as intents rather than rules, so the matching between intents and rules is many-to-many, and a policy is not penalized for decomposing a requirement differently from the annotator. A policy generator receives only the agent specification and produces one policy per workflow. The policy is then frozen and scored by an LLM judge in two ways. In the static evaluation, each golden intent is labelled COVERED, PARTIAL, or MISSING, and each rule is labelled JUSTIFIED, BONUS (defensible but not required), or SPURIOUS. From these labels, strict coverage (the fraction of golden intents that are COVERED with the correct value) and rule precision (JUSTIFIED rules over JUSTIFIED plus SPURIOUS rules) are computed. In the behavioral evaluation, the judge, without access to the intents or labels, simulates the allow/block decisions of the frozen policy on each case, and false-allow rate, false-block rate, and balanced accuracy are reported. The authors claim their main contributions are the linked multi-domain corpus, the dual-view scoring protocol with its per-metric reporting rule, and the diagnostic analyses.

Critical strengths
  • To the best of my knowledge, prior policy synthesis systems have so far been validated only within their own policy languages and self-contained evaluation settings, which makes them difficult to compare with one another. This paper describes ground truth as intents rather than rules and matches intents to rules many-to-many, so a policy is not penalized for how it decomposes a requirement. This is a meaningful step toward comparing systems with different policy representations under a common standard, although this paper does not evaluate existing synthesis systems with different policy representations. The paper also freezes each generated policy and scores it by static fidelity and behavioral simulation, with separate information access for each. Intent-case links connect the two results, which makes it possible to trace a behavioral failure back to a specific static defect. For example, the authors show that most false allows whose cases are annotated with an expected firing intent (397 of 435, 91.3%) occurred where the same policy did not strictly cover that intent. As a public resource, I think it is a good point that this benchmark jointly provides cross-domain agent specifications, decomposition-independent intents, and linked decision-level tests, which the authors state did not previously exist.
Critical weaknesses
  1. In this work, the specifications, intents, and cases in the corpus are all LLM-generated, quality review and adjudication are also LLM-based. The authors state that there is no human audit or inter-annotator agreement. Cross-judge agreement (92.5% verdict, 87.6% strict coverage) shows consistency between two LLMs, not validity against ground truth, and errors may be correlated. Moreover, both judges (A: openai/gpt-5.4, B: anthropic/claude-opus-5) are also among the seven generators whose policies are being scored, and the paper does not test for self-preference. A headline finding is judge-conditional: Claude Opus 5 ranks seventh on balanced accuracy under judge A but second under judge B (Section 6, RQ1), leaving open whether this reflects judge leniency, self-preference, or a real effect. The behavioral score is also a semantic judgement rather than the outcome of executing a policy, and 55.4% of submitted rules are natural-language, so it may measure the judge’s interpretation more than the policy’s behavior. Since this determines whether the static and behavioral scores can be read as measures of synthesis quality at all, it weighs heavily in my assessment.

  2. The paper motivates the benchmark by noting that no shared multi-domain resource exists on which synthesis systems can be compared (Abstract, Section 2), and states that it accepts submissions from any system meeting its contract (Section 6). However, the only systems evaluated are seven LLMs driven by one shared prompt, so the results show differences between models, not between synthesis methods. They do not show whether the benchmark can distinguish synthesis methods with different designs. The authors state that none of the three synthesis systems in Section 2 is publicly runnable on arbitrary agent specifications and that their input contracts differ (Section 6), but they give no system-specific justification and attempt no simplified re-implementation. Even if full reproduction is difficult, this gap could be addressed by adding one or two simplified implementations of the core designs of prior work on top of the same underlying model, for example a generator–critic loop in the style of Mondl et al., or per-request policy generation in the style of Conseca. Holding the model fixed and varying only the design would separate model effects from method effects, and would give follow-up work a precedent. It would also help to provide adapter guidelines for systems that use other policy representations (e.g., Cedar) onto the authors' schema (Section 3), together with a discussion of the information lost in that conversion, so that external submissions become easier. Comparing synthesis systems is the stated motivation of the benchmark, yet this is left untested. As a result, the central motivation is not yet supported, which weighs on my recommendation.

Questions for authors

Please see Critical Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.