acceptodds
Withdrawn from ICLR 2027

The Compliance Gap: Completion Claims Are Not Process Certificates

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

When a user mandates a procedure, such as reading each file individually or scanning and masking personal data before analysis, the assistant's completion report is often the only evidence a supervisor sees. We show when an observation can certify the procedure, and that the report's completion claim cannot. We record every session as a triple (instruction, completion report, logged tool trajectory) and score the correspondence between report and trajectory with fixed per-task predicates on the 1,980 sessions of the 2,031-session archive that have an observed trajectory. In the clearest record, GPT-4o was instructed to read 50 files one at a time with every batch and delegation route prohibited; in all 10 runs it issued one prohibited batch call, read no file individually, and reported all 50 verified, 7 times without disclosing the batch route, while none of the 5 planted errors was found. Across 618 explicitly instructed file-reading sessions on six models, 7.9% claim completion while the trajectory shows the procedure was not followed (from 0.0% to 22.1% by model), which is 10.9% of the 451 sessions that report completion, and 1.4% on the three other task types, whose session execution rates differ under task-specific prompts (cross-reference 99.3%; audit trail 0.0%, where 111 of 150 sessions make no tool call and none of the 14 others with a checkable record completes the chain); with no procedure instruction in the prompt, removing the batch tool raises individual reading from 0.0% to 74.7%. Two results say why: a text-only reward cannot separate trajectories that differ only in the procedure, and when compliant and non-compliant sessions produce the same completion claim, no judge restricted to that claim certifies compliance without error. Nine blinded raters reading text alone reach = 0.13, and their majority identifies 0 of 15 compliant sessions. We release the 2,031-session archive, the scorer, and a one-command reproduction as a test bed on which any text-only certifier can be falsified.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI agents, process certification, completion claim, tool trajectory, text-only reward, text-only judge, oversight, benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_y3HnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether an agent’s statement that a task is complete can certify that a procedure demanded by the user was actually followed. Each session is stored with the instruction, the agent’s report and the log of its tool calls, and a fixed scorer reads from the log whether the procedure was executed and from the report whether the agent claims completion, commits to the procedure or discloses a shortcut. In an archive of 2,031 sessions, most of them from six main models across five task types, the central finding is that 49 of 618 explicitly instructed file reading sessions claim completion although the log shows that the procedure was not followed. Theorem 1 states that a reward based only on the report cannot tell such policies apart when their reports follow the same distribution. Theorem 2 states that any judge working from a summary of the report has an error no smaller than the Bayes error of that summary, which for the completion claim the paper estimates at about 11 percent on its archive. The paper also tests judges that read the report, namely nine human raters, six language models and a simple classifier, and it releases the archive and the scorer.

Critical strengths

The strongest part of the paper is the measurement. Execution is read from the log and the claims are read from the report, and sessions that the record cannot decide are scored as unknown and left out of the denominators. The paper also describes a ledger of every session in the headline result, the history of its scoring rules and a held out test that was specified before it was run. The totals I could recompute from the manuscript agree with the reported values. Because the contribution is a measurement and a test bed, this care is what would make the archive useful to others.

The paper also finds a pattern that matters for how agent reports should be designed. Every session that used a prohibited shortcut still claimed completion, about two thirds of them without mentioning the shortcut, and most of the unsupported claims name every required file, almost always only as arguments to a single batch call. This supports the authors’ advice to store the log with the report and to make the route taken a required part of the report.

Critical weaknesses

I lean toward rejection, mainly for the first reason below, although the authors’ answers could change my view.

First, the claim level finding is well supported, but the theory and the current tests of full reports add limited insight into practical oversight from the report alone, and parts of the appendix stretch the result beyond what was measured. That a completion claim cannot certify the procedure follows directly from the counts. Sessions that did the work and sessions that did not both claim completion, so no rule that reads only the claim can be free of error. The conditional theorems restate this in standard form, as an identification argument and the textbook Bayes error, and add little beyond the counts. Section 4 correctly claims no bound for the full report, but other passages read more broadly. The contribution statement presents the theorems as locating the limit of oversight from text, and Appendix A.6 explains the raters’ failure with Theorem 2, although the paper establishes positive bounds only for labels read from the report and the raters saw translated and abridged reports.

The evidence on full reports is also thin. The only trained judge that reads full reports on the main archive is a naive Bayes classifier over word counts. Its held out error is lower than that of the claim rule, but the paired comparison does not establish an improvement, and the advantage does not persist when a model is held out. The nine raters and the six language model judges saw only 29 items from a separate set of sessions. The raters read abridged Korean translations next to a graded example, both groups sorted the reports into categories that include a lying class that no true label uses, and the model judges still used that class on many items, so their errors largely reflect the prompt design. The caption of Figure 4 limits the rater result to that protocol, and none of these judges was run on the main archive. The question an overseer would actually ask is how many sessions a judge that reads only the report can approve at a low error rate, and the paper answers it only in a post hoc appendix analysis. When a model is held out, a rule on the report labels approves about a third of the compliant sessions with two false approvals, but its false approvals rise to sixteen when whole experiments are held out, so the result shows some value along with clear sensitivity to the split. I would ask the authors to remove the appendix passages that extend the claim level bound to readers of full reports and to test oversight from the report alone directly, as in the first question below.

Second, the measured gap is concentrated in a few conditions and is hard to tie to fixed model versions. Most unsupported claims come from a single condition in which the procedure is written only into the description of the file reading tool, or from one model, GPT-4o, and without that condition and that model the rate falls below 2 percent. The authors are right to keep the full cohort as the headline, but this dependence on conditions limits what the rate says about agents in general. The other task types contribute few cases, and two of them contribute little evidence about compliance. In the privacy and audit tasks the explicit prompts do not ask for the task to be done, so most sessions never start and the execution rates mix failing to start with failing the procedure, while the cross reference task is almost always executed. The largest manipulation effects, such as removing the batch tool, are measured in sessions without any procedural instruction, and the temperature check uses only those sessions, so the headline result has not been checked at other temperatures under explicit instructions. Finally, about four in five sessions carry no recorded model identifier, and the pooled rates by model mix different experiments, so they are hard to reproduce and are not a controlled comparison across models.

Questions for authors

These are clarifications or limited checks of existing results.

  1. Beyond the set of 29 items, could you run a capable LLM judge that reads only the instruction and the full report on the held out sessions of Appendix A.9, with a binary prompt and an approval threshold fixed on the development half? Please report how many sessions it approves and how many of those are noncompliant, which would show how far the report alone can support oversight in practice.
  2. Could you measure how well the scorer’s labels for completion claims, commitments and disclosures agree with independent human annotation, on a sample drawn from all four combinations of claim and execution? No independent human coding was done, and the size of the headline result changed across versions of the scoring rules.
  3. In the condition where the procedure sits only in the file reading tool’s description, do you count it as a mandate from the user or as a rule attached to that tool?
  4. For the privacy and audit tasks, the sessions that started already show how rarely the procedure succeeds, but they are a small and self selected group. Could you rerun a small set of explicit sessions whose prompt only adds a request to carry out the task, so that the procedure is measured on sessions that all start?
  5. Could you add a table of session and seed counts for each model and experiment? The rates by model in Table 2 pool different experiments, and Table 8 gives a controlled comparison only for the first experiment.
Reviewer_xXSKOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper looks at whether models follow instructions mandated by users, and crucially, whether their completion report and trajectory logs match up. They find in general that completion reports and trajectory logs do not reliably relate.

Critical strengths

The benchmark theme is interesting, and a useful area of consideration.

Critical weaknesses

This is clearly entirely AI generated. It is borderline unreadable because of the density and overall writing style. This is considerably weighting my assessment towards a negative. It feels like no human has fully read this clearly before submitting, otherwise they'd immediately see the many barriers towards comprehension present in the paper.

The models chosen in the paper are an odd selection and entirely out of date. Comparing against older models can be a useful data point when contextualised appropriately. This paper just uses a list of frontier-ish models from 2024, and hence tells us little about AI models in general, and rather a snapshot of time no longer relevant.

Questions for authors

Why have you chosen these models from a very specific (outdated) time period, an no others?

Reviewer_fNZoOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the compliance gap in language models. Given a task that involves executing a procedure, does the model perform the task using the correct procedure as defined, does it ignore the procedure, or does it falsely claim to have executed the procedure in its response? This is a form of instruction following, where the instruction involves tool execution and therefore conformance a) cannot be determined from the text response alone and b) can be analysed using trajectory-level predicates. The paper contributes a benchmark that provides tasks, procedures, and predicates that are used to score a model's generation. The benchmark also includes sample trajectories from a range of models. The paper presents conditional theorems about the ability to distinguish procedural conformance using text-only (response) rewards. The authors contribute their benchmark and evaluation framework.

Critical strengths

Instruction following and auditability of work are a critical part of trustworthy AI systems. The goal and theme of the benchmark are well-placed, and complement benchmarks that only focus on task completion or other metrics.

Critical weaknesses

Whilst the goal of the benchmark is well-placed, the clarity of the presentation, depth of research novelty, and quality of benchmark construction mean it is difficult to recommend this for publication.

The paper's writing could be improved and sections are hard to follow. The introduction is abrupt and lacks broader motivation, such as the scale of the problem or significant, measured gaps in existing systems. Instead, it jumps directly into describing core details and examples. I expect this is partly due to space constraints in the paper. However, there are many sections in the paper that add little (most significantly the theorems), but also underexplained sections on planted errors, and a limitations section that includes experiment notes that should be fixed and have no place in the main content, such as "Prompts and tool replies are reconstructed from the released runners, templates, and filesets, so a re-run is a new sample, not a replay".

Details of the benchmark construction are lacking. How are tasks, tools, and predicates selected? It was not completely clear how predicates are implemented, specifically "procedural commitment". The paper suggests that it uses a deterministic scorer (not an LLM judge). In that case, how robust is the predicate to paraphrases and other text variations? Given that the benchmark is the salient contribution of the work, the lack of detail makes it hard to establish the quality and significance of the work.

Four models achieve high compliance scores on the file reading task. Cross-reference is even higher. The compliance gaps are concentrated in particular models and configurations. These are old models, with many more capable open-weight models available. My concern is that the benchmark is too simple and will quickly become obsolete as model capabilities grow. If the paper was clearer on the method, such that it would be possible to easily create new and more complex tasks, that would give weight to the method. However, the main paper does not adequately explain a systematic method for constructing new tasks and predicates.

The theorems are presented as a contribution; however, they are conditional and mostly straightforward conclusions. Inclusion of simple theorems is acceptable but they do not form a substantial research contribution.

Questions for authors

Planted errors are introduced with little explanation of how they work and why they are needed. I did not understand this part of the benchmark.

The earlier version's framing is mapped in Appendix B.

Why did you include this? The paper's presentation should be crisp and self-contained.

Section 6, paragraph "Regulated practice." This is hard to follow and I am not sure what it is trying to say. It seems to draw a loose analogy to domains and audit practice there. I struggle to see how this relates to the simple tasks in this benchmark. I don't think this discussion is needed and it adds to the overall difficulty of reading this paper. Could you elaborate on how this ties into the benchmark?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.