- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks whether an agent’s statement that a task is complete can certify that a procedure demanded by the user was actually followed. Each session is stored with the instruction, the agent’s report and the log of its tool calls, and a fixed scorer reads from the log whether the procedure was executed and from the report whether the agent claims completion, commits to the procedure or discloses a shortcut. In an archive of 2,031 sessions, most of them from six main models across five task types, the central finding is that 49 of 618 explicitly instructed file reading sessions claim completion although the log shows that the procedure was not followed. Theorem 1 states that a reward based only on the report cannot tell such policies apart when their reports follow the same distribution. Theorem 2 states that any judge working from a summary of the report has an error no smaller than the Bayes error of that summary, which for the completion claim the paper estimates at about 11 percent on its archive. The paper also tests judges that read the report, namely nine human raters, six language models and a simple classifier, and it releases the archive and the scorer.
Critical strengths
The strongest part of the paper is the measurement. Execution is read from the log and the claims are read from the report, and sessions that the record cannot decide are scored as unknown and left out of the denominators. The paper also describes a ledger of every session in the headline result, the history of its scoring rules and a held out test that was specified before it was run. The totals I could recompute from the manuscript agree with the reported values. Because the contribution is a measurement and a test bed, this care is what would make the archive useful to others.
The paper also finds a pattern that matters for how agent reports should be designed. Every session that used a prohibited shortcut still claimed completion, about two thirds of them without mentioning the shortcut, and most of the unsupported claims name every required file, almost always only as arguments to a single batch call. This supports the authors’ advice to store the log with the report and to make the route taken a required part of the report.
Critical weaknesses
I lean toward rejection, mainly for the first reason below, although the authors’ answers could change my view.
First, the claim level finding is well supported, but the theory and the current tests of full reports add limited insight into practical oversight from the report alone, and parts of the appendix stretch the result beyond what was measured. That a completion claim cannot certify the procedure follows directly from the counts. Sessions that did the work and sessions that did not both claim completion, so no rule that reads only the claim can be free of error. The conditional theorems restate this in standard form, as an identification argument and the textbook Bayes error, and add little beyond the counts. Section 4 correctly claims no bound for the full report, but other passages read more broadly. The contribution statement presents the theorems as locating the limit of oversight from text, and Appendix A.6 explains the raters’ failure with Theorem 2, although the paper establishes positive bounds only for labels read from the report and the raters saw translated and abridged reports.
The evidence on full reports is also thin. The only trained judge that reads full reports on the main archive is a naive Bayes classifier over word counts. Its held out error is lower than that of the claim rule, but the paired comparison does not establish an improvement, and the advantage does not persist when a model is held out. The nine raters and the six language model judges saw only 29 items from a separate set of sessions. The raters read abridged Korean translations next to a graded example, both groups sorted the reports into categories that include a lying class that no true label uses, and the model judges still used that class on many items, so their errors largely reflect the prompt design. The caption of Figure 4 limits the rater result to that protocol, and none of these judges was run on the main archive. The question an overseer would actually ask is how many sessions a judge that reads only the report can approve at a low error rate, and the paper answers it only in a post hoc appendix analysis. When a model is held out, a rule on the report labels approves about a third of the compliant sessions with two false approvals, but its false approvals rise to sixteen when whole experiments are held out, so the result shows some value along with clear sensitivity to the split. I would ask the authors to remove the appendix passages that extend the claim level bound to readers of full reports and to test oversight from the report alone directly, as in the first question below.
Second, the measured gap is concentrated in a few conditions and is hard to tie to fixed model versions. Most unsupported claims come from a single condition in which the procedure is written only into the description of the file reading tool, or from one model, GPT-4o, and without that condition and that model the rate falls below 2 percent. The authors are right to keep the full cohort as the headline, but this dependence on conditions limits what the rate says about agents in general. The other task types contribute few cases, and two of them contribute little evidence about compliance. In the privacy and audit tasks the explicit prompts do not ask for the task to be done, so most sessions never start and the execution rates mix failing to start with failing the procedure, while the cross reference task is almost always executed. The largest manipulation effects, such as removing the batch tool, are measured in sessions without any procedural instruction, and the temperature check uses only those sessions, so the headline result has not been checked at other temperatures under explicit instructions. Finally, about four in five sessions carry no recorded model identifier, and the pooled rates by model mix different experiments, so they are hard to reproduce and are not a controlled comparison across models.
Questions for authors
These are clarifications or limited checks of existing results.
- Beyond the set of 29 items, could you run a capable LLM judge that reads only the instruction and the full report on the held out sessions of Appendix A.9, with a binary prompt and an approval threshold fixed on the development half? Please report how many sessions it approves and how many of those are noncompliant, which would show how far the report alone can support oversight in practice.
- Could you measure how well the scorer’s labels for completion claims, commitments and disclosures agree with independent human annotation, on a sample drawn from all four combinations of claim and execution? No independent human coding was done, and the size of the headline result changed across versions of the scoring rules.
- In the condition where the procedure sits only in the file reading tool’s description, do you count it as a mandate from the user or as a rule attached to that tool?
- For the privacy and audit tasks, the sessions that started already show how rarely the procedure succeeds, but they are a small and self selected group. Could you rerun a small set of explicit sessions whose prompt only adds a request to carry out the task, so that the procedure is measured on sessions that all start?
- Could you add a table of session and seed counts for each model and experiment? The rates by model in Table 2 pool different experiments, and Table 8 gives a controlled comparison only for the first experiment.