acceptodds
Under review as a conference paper at ICLR 2027

ControlBench: Did the Approval Actually Happen? Process-Grounded Auditing of Human Control over LLM Agents

Abstract

AI agents now issue refunds, send messages, and modify files on a user's behalf, often after the user types a quick "ok". But did the agent wait for that "ok" before acting? A chat transcript cannot answer this: it records what was said, not the order in which things happened, so an agent that acts first and collects approval afterwards looks exactly like one that waited. We introduce **ControlBench**, a benchmark of 1,800 simulated human-agent sessions ( 8,300 executed actions, two simulators) in which every executed action carries a ground-truth label of whether approval preceded it, plus released real-world agent trajectories audited under the same rule. An LLM judge must then answer one question per action: was it approved before it ran? The answer has two parts. The first is the *record*. Judged from transcripts alone, every model we test errs in both directions, the most approval-optimistic judges approving never-approved actions more often than not; given the platform's event stream, the stronger judges' false approvals all but vanish. The second is the *reader*. Even with complete records, judges fail to apply the rule when an approval arrives late, targets a stale version, or has been revoked, falsely approving most revoked-approval executions; such cases never occur in the released sessions, so ordinary accuracy hides them, while a simple deterministic checker over the same records is exact. Gating actions on event-grounded verdicts rather than transcripts would have blocked nearly every unsafe execution in our logs while costing less. Reliable oversight of AI agents therefore needs both: records of what the system did, and a reader that executes the approval rule. We will release the benchmark, its approval semantics, and all per-item predictions.

Primary area:
datasets and benchmarks
Keywords:
LLM agents, agent oversight, approval auditing, benchmark, event logs, process grounding, human control, LLM-as-judge, agent safety, dialogue systems

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xUNrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors study whether an executed agent action was approved before it ran, turning this into a per-action ground-truth label computed from the platform's process event stream by a state machine implementing the frozen Approval-Coverage Semantics v1. ControlBench has 1,800 simulated human-agent sessions with about 8,300 executed actions, generated with LLM operators by a Qwen2.5-7B primary simulator and a harder Mistral-7B replication. Open-weight LLM judges from 1.5B to 72B, plus two hosted API judges on small subsets, see the transcript, the process event stream, or both. The authors report an "observation gap", where judges given transcripts alone err in both directions, while the process event stream cuts mid/large format-competent judges' conditional false-approval rate (FAR) to 0.2-1.3% on the primary corpus. They also report a "semantic-execution gap": on same-transcript counterfactual pairs, judges given the event stream stay at chance on approval order and stale versions, but a deterministic replay reader always gets them right. The benchmark's hard split, a retrospective analysis of gating on event-grounded verdicts, tau-bench logs, and public SWE-agent/OpenHands runs round out the evaluation.

Critical strengths
  • Executable ground truth, plus a counterfactual battery isolating rule application. Each per-action label is computed by a frozen specification, tested for conformance. Each same-transcript pair varies only in event order, binding, or version. This separates reading the record from applying the rule: binding is mostly readable for two judges, but order and version stay at chance, and in the stage-wise probe, Qwen3-4B finds the relevant events but misjudges precedence, while Llama-3.1-8B fails earlier at extraction. This result is the most informative in the paper, and the main reason ControlBench could serve to certify LLM readers.
  • Broad controls around the main comparison. Taken together, cross-simulator replication, a family-disjoint judge, trained probes, learned sequence-model baselines, verdict-event deletion, plus natural-prevalence accuracy make it unlikely that the gain from events is an artifact of one simulator or judge.
Critical weaknesses
  • The observation gap mostly measures if the answer-bearing event is in the prompt. Each label is defined over recorded approve and execute events, and the replay reader implements the semantics used to generate the labels; the paper itself calls decision-event sufficiency "partly definitional" and replay agreement "definitional rather than empirical". Removing the verdict-bearing events drops events-only accuracy to chance.
  • The external anchor scores a different construct. On tau-bench the labels measure action-authorization coverage, where an exact user request counts as covered, and the transcript hides the executed call. So the high external false-approval rate partly reflects a judge not knowing which action it audits.
  • Under the procedural variant, records help Qwen3-4B and gemma-2-9b but raise FAR for the Llama-3.1-8B and 72B judges, and the SWE-agent/OpenHands trajectories contain no verdict events. So the external evidence is mixed across judges and, for the coding agents, cannot test the rule at all.
  • The title's human-control claim rests on LLM-simulated operators whose text and events are decoupled by construction.
  • The claim about the semantic-execution gap is stated more generally than the judges tested. Judges are open-weight models up to an AWQ-quantized Qwen2.5-72B, plus two hosted API judges tested on 20 pairs per cue; neither a reasoning model nor a judge with code execution, the obvious remedy for rule execution, is evaluated. The order gap also depends on the prompt, since the post-hoc P3 condition closes it for gemma-2-9b.
  • The headline FAR range is computed across a "format-competent" panel excluding Yi-1.5-34B. This model seems to meet the paper's own verdict-parse failure threshold, but its FAR rises with events.
Questions for authors
  1. Does a reasoning model, or a judge that can write and run a check over the event log, close the order gap on the 200-pair and 300-pair batteries?
  2. For the revoked-approval result cited in the abstract, how many uncovered revocation items is each judge shown in total? And the FAR of each?
  3. Was the "format-competent" panel fixed before the E1 results were in?
  4. A transcript with realistic platform timestamps, rather than only appended reply latency, could show how much of the observation gap is due to the text-only condition hiding information that an auditor would normally see.
  5. Prior work shows LLM judges failing to apply temporal rules over event traces where formal monitors succeed (Alamdari et al.; FormalJudge by Zhou et al.) and same-surface authorization pairs where an LLM judge fails yet a structural check succeeds ("Same Action, Different Justification"). What does the counterfactual battery show that this earlier work doesn't?
1 comment
AuthorsOpenReview ↗

Follow-up on the Judge Panel and False Approval Rates

Dear Reviewer xUNr,

Thank you for flagging the judge selection and metric denominators. Table 4 now reports all seven judges, including Yi. Adding events improves overall accuracy for all seven judges and lowers the false approval rate for five. Table 8 now computes the prefix false approval rate over negative examples.

Do these changes address your concerns about how to interpret the main comparison?

Best regards,

The authors

Reviewer_QaBPOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a benchmark consisting of simulated human–agent sessions that contain actions together with a ground-truth label denoting a user-approval. It further presents an eroding of control-state inference. The paper presents an extensive evaluation of the benchmark on several open-weight LLMs. The results indicate that raw logs yield high false-approval rates. The proposed text+control state abstraction yields much lower false-approval rates.

Critical strengths

Tackles a real problem with today’s conversational LLM-based agents. Auditing control signals can be an uncertain process, especially with another LLM as a judge.

Exhaustive empirical evaluation on multiple open-weight LLMs. The benchmark contribution is potentially useful. Giving an ability to generate and replay diverse tasks and domains. The results indicate that there is a variety of behaviors and that it is not trivial for the current LLMs/judges. The evaluation shows that text+event stream gain in detection accuracy.

Critical weaknesses

The state encoding is fairly straightforward contribution. The innovativeness of the contribution of the event semantics and control state is not high. The approach sounds more like an engineering effort to improve the ability of the judges to properly order events than a new research contribution.

The approach provides no guarantees. Technical appendix C offers only the theorem statement, without proof. The main paper does not give a precise definition of the deterministic replay. Other important abstractions such as Approval-Coverage Semantics are described only in the appendix D. Moreover this mixes the description of the technique and the preliminary evaluation results, making it difficult to understand the result.

The writing should improve, as the attempt to put as many research results in the paper reduces the readers’ ability to comprehend and properly judge the contribution. The main paper is not self-contained in the current state. The page limit is forcing many important details to the appendix. It is hard to fully judge the construction of the benchmark and its generalizability to other tasks, e.g., if the future LLMs start being trained on the current traces.

Even though the evaluation in the appendix is quite detailed, the main text brings up in many cases only the number summary, without a clear insight. Going for 9 Evaluation questions, without clearly stating their importance after a point hurts more the paper than it helps. Consider rewriting this section after Evaluation Question 3 to emphasize the most important results, with the goal to convey the insights to the reader and leave the other EQs fully in the appendix/technical report.

Questions for authors

None.

2 comments
AuthorsOpenReview ↗

Follow-up on Replay and the Benchmark Contribution

Dear Reviewer QaBP,

Thank you for raising questions about replay and what the benchmark measures. Section 3.1 now defines how replay tracks approval for each item and parameter version. Appendix C proves its equivalence to the approval rule under the stated assumptions.

The new experiments distinguish two questions. Table 7 shows that adding execution time improves judgments when message timestamps are already available. Table 6 shows that, given the same event histories, explicit rules and structured events raise the number of approval-order pairs solved on both sides from 3/70 for GLM and 2/70 for Kimi to 70/70 for both. This lets the benchmark test what evidence is needed and how reliably models use it.

Which concern, if any, remains open for you after these revisions?

Best regards,

The authors

Dear Reviewer QaBP,

With only one day left in the author discussion, we would appreciate a brief indication of which concern, if any, remains central to your assessment. Section 3.1 and Appendix C now give the replay definition and proof. Table 6 also shows what the benchmark can reveal: given the same event histories, explicit rules and a structured event table improve performance from 3/70 pairs for GLM and 2/70 for Kimi to 70/70 for both.

Does either replay correctness or the benchmark's empirical contribution remain a concern for you?

Thank you for your time.

Reviewer_SFGCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper asks whether an auditor can check that an AI agent’s action was approved by a human before it ran. A chat transcript cannot show this, because it does not record the order of events: an agent that acts first and gets the approval afterwards looks the same as one that waited. The authors build 1,800 simulated sessions, in which small LLMs play both the human operator and the agent. A fixed rule labels every action as approved or not, based on the system’s event log. LLM judges then decide for each action whether it was approved, given either the transcript alone or the transcript plus the event log. With the event log, the stronger judges are almost always right. But even with the event log, the LLM judges fail on harder cases, such as approvals that arrive after the action or approvals that were revoked. A simple program that applies the rule directly to the event log gets all cases right.

Critical strengths
  1. The question is clear and practical. Agents increasingly act after a quick human “ok”, and the example of an agent acting first and collecting approval afterwards shows well why a transcript is not enough.

  2. The paper shows that a transcript-only audit is easy to fool. When the agent’s messages falsely claim approval, some judges reading only the transcript approve almost all unapproved actions, while judges with the event log are rarely fooled.

Critical weaknesses
  1. The main result follows largely from the setup. The paper’s main result is that judges who get the event log label actions much more accurately than judges who only get the transcript, and that a program applying the approval rule to the event log is always right. But the correct labels are themselves computed from the event log, with that same rule. So the event log is guaranteed to contain the answer, and the program is guaranteed to be right. The authors say this themselves: the result is “partly definitional” and the program agrees with the labels “by construction”.

  2. The judge failures are not tested with frontier models. With the event log, the LLM judges still fail on harder cases, such as late, outdated, or withdrawn approvals, while a simple program gets them right. But these harder cases were only tested with three open models of 9B or smaller and two small hosted models. With a different prompt, one 9B judge already got the order cases right, so frontier models may not make these mistakes. Without that test, it is unclear whether LLM judges are unsuitable for this task or whether the tested models are too small.

Questions for authors
  1. How do frontier models perform on the harder cases? Since these cases are a few hundred test pairs, this test seems affordable.

  2. The main datasets contain none of the harder cases. How often do late, outdated, or withdrawn approvals occur in real agent runs?

4 comments
AuthorsOpenReview ↗

Follow-up on Stronger Reasoning Models

Dear Reviewer SFGC,

Thank you for asking how the benchmark performs with stronger reasoning models. In Table 6, GLM-5.3 and Kimi-K2.6 each solve both sides of all 70 approval-order pairs when given explicit rules and a structured view of the same event histories. On the full hard-case set, they answer 1,025/1,168 and 1,150/1,168 correctly, respectively.

Do these results address your concern about whether the benchmark remains informative for stronger models?

Best regards,

The authors

Dear Reviewer SFGC,

With only one day left in the author discussion, we would appreciate your view on one point. Table 6 holds the event histories fixed and measures how the instructions and event presentation affect the models' judgments. Does this comparison address your concern that the main finding follows from how the labels are defined?

Thank you for your time.

The experiments on stronger models address my concern about model coverage, but my main concern about the empirical contribution remains. I’m therefore keeping my score.

Dear Reviewer SFGC,

Thank you for confirming that the new experiments address your concern about model coverage.

In Table 6, both configurations use the same models and the same underlying event histories. The replay rule fixes the correct answers in both conditions. What the experiment reveals is that the ordering errors are recoverable from the records already available. When given explicit rules and a structured event table, each model goes from getting both sides right on at most 3 of 70 order pairs to getting both sides right on all 70. That change in model behavior is the empirical result.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Any Questions About Our Responses?

Dear Reviewers,

Thank you again for taking the time to read our paper and write your reviews. Your feedback helped us see where we needed to explain our work more clearly, and the paper is better for it.

If you’ve had a chance to read our replies, is there anything you’re still unsure about or would like us to address?

AuthorsOpenReview ↗

Final Rebuttal Summary for the Area Chair

Dear Area Chair,

Thank you for guiding our work through the review process. The reviewers' questions helped us clarify how we determine whether an action was approved and test how reliably models make that judgment from the same event records. We appreciate the time it takes to consider the original reviews alongside the revised manuscript and new results.

Strengths noted in the reviews

  • A practical question about verifying approval before agents act (SFGC, QaBP).
  • Matched pairs that distinguish finding the relevant events from applying the approval rule (xUNr).
  • Extensive evaluation across open-weight models (QaBP).

Key concerns and responses

  • Is replay precisely defined and correct? (QaBP) Section 3.1 now specifies the replay procedure and its assumptions. Table 3 gives worked examples, and Appendix C proves that replay implements the stated approval rule.
  • What does the benchmark reveal beyond computing reference labels? (QaBP, SFGC, xUNr) Table 6 compares two configurations using the same event histories. With explicit rules and a structured event table, GLM-5.3 and Kimi-K2.6 go from solving both sides of 3/70 and 2/70 approval-order pairs, respectively, to 70/70 each. This tests how reliably models apply the rule when the evidence is already available.
  • Do executable checks and timestamps help? (xUNr) A program generated by Kimi and frozen before evaluation correctly classifies 1,165/1,168 hard cases (Appendix K). Adding execution time to message timestamps raises correct judgments from 68/140 to 136/140 for GLM and from 70/140 to 140/140 for Kimi (Table 7).
  • Judge selection and error rates (xUNr). Table 4 includes all seven judges, including Yi. Section 4.2 gives the revocation counts and denominators, and Table 8 calculates prefix false approval rates over negative examples.
  • Deployment scope (SFGC, xUNr). Section 3.2 distinguishes coverage in the controlled tests from frequency in deployment. Section 4.4 reports procedural confirmation separately from broader authorization in external traces.

We would especially appreciate your assessment of whether the replay proof and the comparison in Table 6 address the concerns about correctness and empirical contribution.

Best regards,
The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.