acceptodds
Under review as a conference paper at ICLR 2027

Green Because They Leaked: Test-Gated Policy Updates Don't Certify Safety

Abstract

Automated judges—compiled rule systems, fine-tuned classifiers, or prompted language models—increasingly gate what AI agents are allowed to do. When the policy changes, the judge must be updated, and the standard safeguard is borrowed from continuous integration: admit an update only if it passes a suite of test cases. We show that this green checkmark means far less than it appears to. First, the standard way of measuring the safeguard leaks. When gating tests and evaluation cases share underlying evidence states, the gate is credited with catching errors on cases it has already screened: under such a protocol, our test-gated updater appears to buy monotone safety, reaching 100% error catch at large test budgets; under a corrected state-disjoint protocol, error catch saturates at 21% from budget one, and we explicitly retract the earlier result. Second, with leakage removed, safety is bounded by coverage, not budget. The effective budget is the number of unique evidence states that exist—about three per revision on our held-out split, zero for a third of revisions—and every admitted erroneous update lies on a criterion with no test coverage; neither test quality, gate strictness, nor prompted model scale repairs the gap. We then address both problems: the leakage, by the state-disjoint protocol itself, released with validators; the coverage, by a proven converse to our bound—checking the candidate against the archived old rule on the full typed state space outside the amendment's scope certifies zero protected-side errors, with no labels required—realizable by schema-based test generation, whose empirical validation is left open. Until then, inventory-drawn test suites make update gating an auditable screen, not a safety certificate. We release the benchmark, partitions, and audits.

Primary area:
datasets and benchmarks
Keywords:
LLM-as-a-judge, Model editing, Regression-gated updates, Evaluation leakage, Agent safety

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fJ4TOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies whether test-gated updates to automated policy judges provide meaningful safety assurance. It introduces PolicyPatch, which uses an LLM to propose typed rule updates and accepts them only after schema validation and regression testing, alongside a constructed benchmark of policy amendments. Its central finding is that splitting by textual cases permits underlying evidence states to overlap between gating and evaluation, inflating apparent safety: the previously reported 100% error catch falls to approximately 21% under a state-disjoint protocol. The corrected experiments reveal limited test coverage and errors concentrated on criteria with no available gating states. Validation includes budget sweeps, baseline comparisons, and noise and expressivity stress tests. The paper also formalizes update-specific metrics and outlines conditional certification through exhaustive checking, although the proposed coverage-generation mechanisms remain empirically unvalidated.

Critical strengths

The useful contribution is the concrete demonstration that evidence-state reuse can substantially inflate the measured effectiveness of an update gate. The reversal after correcting the evaluation protocol provides a practical warning for systems evaluated using paraphrases or templated cases. The proposed state-disjoint protocol offers an actionable way to avoid this specific source of misleading assurance. The paper also provides useful reporting practices for policy updates. Separating success on affected cases from harmful changes on protected cases exposes failures that aggregate accuracy can conceal, while reporting unique-state capacity makes testing limitations explicit.

Critical weaknesses

The scientific novelty is limited. The central principles, i.e., that passing a finite test suite does not establish correctness on untested inputs, that semantically overlapping evaluation examples inflate performance, and that updates require both edit success and locality, are established ideas from software testing, evaluation methodology, and model editing. Propositions 1 and 2 largely formalize direct consequences of the testing setup and the assumed correctness of the old rule and amendment scope. Applying these principles to policy-judge updates is relevant, but the paper does not sufficiently establish a distinct conceptual or technical advance. Its empirical findings therefore need to carry most of the contribution. The empirical evidence is too narrow to make that contribution sufficiently compelling. The main held-out splits contain only three and six criterion IDs, and the amendments and cases are constructed within a heavily assisted, closed-world pipeline. The budget saturation result is closely tied to an inventory containing very few unique states, with zero-capacity criteria accounting for much of the observed failure. Some ablations consequently yield outcomes that are structurally expected, while the unmatched model comparisons do not isolate the effect of scale. The external contract pool has no reported outcomes, and the proposed mechanisms for overcoming the coverage ceiling are unvalidated. These limitations leave the broader significance of the findings insufficiently established.

Questions for authors
  • For the zero-capacity criteria, how many states are forced into evaluation by the singleton-stratum constraint? Is zero gating capacity unavoidable under the stated constraints, or could another valid state-disjoint partition provide coverage? This would help distinguish an inherent inventory limitation from an effect of the partitioning design.

  • Table 2 states that aggregation conventions differ between the legacy and corrected results. Can the existing outputs be summarized using a common aggregation convention, with legacy performance separated by whether evaluation states overlap gating states? This would clarify how much of the reported reversal is attributable specifically to state reuse rather than changes in evaluation composition or weighting.

Reviewer_UYqROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the reliability of automated policy adjudicators after rule revisions, focusing on whether test-based gating mechanisms can provide reliable safety guarantees for policy updates. The authors point out that when gating tests and evaluation examples share underlying evidence states, evaluation results may be inflated due to data leakage. Once this leakage is eliminated, the effectiveness of test-based gating is primarily limited by test coverage rather than the testing budget. Therefore, passing a finite test suite does not imply that a policy update is safe.

Critical strengths
  1. Valuable research problem: The paper distinguishes between edit success within the intended update scope and locality outside that scope, introducing corresponding evaluation metrics that provide an operationally testable definition of whether an erroneous update is prevented. This evaluation perspective aligns with the practical needs of dynamic rule-based systems and addresses a meaningful research problem.

  2. Coherent paper organization: The paper follows a clear progression from the policy-update evaluation contract to the benchmark and systems, leakage auditing, coverage limits, and ablation studies. The overall structure is logically organized and easy to follow.

Critical weaknesses
  1. Insufficient validation of key claims: Although the paper constructs 45,360 textual examples, many of them share the same underlying evidence states. The core experiments involve only three criterion IDs in the operator split and six in the locked split, with just 16 and 35 affected evaluation representatives, respectively. Furthermore, the experiments primarily rely on manually constructed policy revisions. Although the paper establishes a CUAD-based external candidate pool and discusses potential solutions based on state generation and exhaustive state-space checking, it does not provide experimental results for these approaches. Consequently, the evidence supporting the paper's key claims remains limited.

  2. Insufficiently validated test-coverage findings: Appendix B specifies that the unique evidence state within each stratum must be assigned to the evaluation set and cannot be used for gating tests. This rule may leave some policy revisions with valid evidence states but no available gating tests. The paper reports that 14 out of 42 revisions in the operator split and 14 out of 84 revisions in the locked split receive no test coverage. However, it does not examine whether this phenomenon persists under alternative state-partitioning strategies. Therefore, the robustness of the reported test-coverage findings remains uncertain.

  3. Insufficient experimental controls: Table 2 compares results obtained with and without data leakage, but the two settings use different aggregation procedures, making it difficult to determine how much of the decline in error-catching rate is attributable specifically to leakage removal. In addition, the paper primarily compares different types of adjudicators, without sufficiently evaluating alternative gating strategies under the same rule compiler. These limitations weaken the experimental evidence supporting the main conclusions.

Questions for authors

Q1: To what extent do the differences in aggregation procedures described in the note to Table 2 account for the decrease in the error-catching rate from 100% to 21.11%?

Q2: In Table 6, the confidence interval for PolicyPatch's accuracy advantage over gpt-oss in the operator split includes zero. Why does the main text nevertheless make the general claim that the accuracy advantage persists after resampling?

Reviewer_ohx9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies updates to an automated judge, a program that decides whether an AI agent’s actions comply with a policy. When the policy changes, the judge must implement the new requirements while preserving decisions unaffected by the change. A common safeguard is to accept an update only if it passes a set of gating tests. The paper examines how reliably this safeguard detects incorrect updates. The paper reports two main findings: 1) overlap between "gating tests" and "evaluation cases" makes the gate appear more effective at detecting incorrect updates than it actually is; and 2) insufficient test coverage allows incorrect updates to pass, and increasing the test budget provides little benefit when additional relevant situations are unavailable.

The paper also introduces PolicyDeltaBench, metrics for evaluating updates, and a protocol that separates gating and evaluation cases by underlying situation. The paper observes that checking agreement with the old rule on every input outside the amendment’s scope guarantees preservation of behavior there.

Critical strengths
  • The authors carefully identify and correct evaluation leakage. They quantify the overlap, withdraw their earlier claim, and introduce a protocol that prevents the same underlying situations from appearing in both gating tests and evaluation.
  • The analysis of coverage is useful. The paper measures the distinct situations available for testing and identifies where incorrect updates pass through the gate. This helps explain why increasing the requested test budget provides little benefit in their benchmark.
  • PolicyDeltaBench provides a controlled setting for studying policy updates, including substantive changes, meaning-preserving edits, and executable ground truth. This supports evaluation of both intended changes and unintended effects on unrelated decisions.
Critical weaknesses
  • The paper is extremely difficult to read. The text is heavily compressed, introduces terminology before adequately explaining it, and often combines several claims in one sentence. Some word choices obscure the intended meaning. For example, phrases such as “the standard way of measuring the safeguard leaks” and “buy monotone safety” require multiple readings to interpret. A running example connecting the policy change, gating tests, and evaluation would help. In its current form, substantial effort is needed simply to decipher the setup and claims.
  • The paper does not clearly establish how widely the studied pipeline and evaluation practice are used. The leakage finding concerns the authors’ own earlier evaluation, while its prevalence elsewhere remains a hypothesis. Referring to this as the “standard way” of measuring the safeguard therefore seems stronger than the evidence supports.
  • The benchmark contains very few distinct situations per criterion, and some criteria have no gating tests. It is therefore difficult to separate limitations of test-gated updating from limitations of the benchmark and its partitioning. The experiments show that requesting more tests from this particular collection provides little benefit, but do not establish whether the same finding would hold with a richer collection or additional generated tests.
  • Corrupting existing tests does not establish that better test design or selection would be ineffective. The claim about test quality should be narrowed to reflect what the experiments actually examine.
  • The certification proposition makes a straightforward observation: exhaustive agreement with the old rule outside the amendment’s scope preserves behavior there. Its practical usefulness depends on a faithful schema, a correctly specified amendment scope, and the feasibility of checking the complete state space. The proposed checking procedure remains empirically unvalidated, so the claim that the paper addresses the coverage problem needs qualification.
Questions for authors
  1. Can you provide examples of existing systems or studies that use the evaluation practice criticized here, beyond the earlier version of your own work?
  2. How much does the coverage finding depend on the benchmark construction and partitioning? Would the same pattern persist with a richer collection of distinct evidence states and nonzero gating coverage for every criterion?
  3. Can you demonstrate the proposed certification procedure on the benchmark and report the number of states checked, runtime, and how the amendment’s scope is established?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.