acceptodds
Under review as a conference paper at ICLR 2027

Review the Window, Not the Step: Oversight of Agents That Choose Their Own Workflow

Abstract

An AI agent that decomposes its own task also decides what a per-step monitor sees. We model this as a zero-sum inspection game: the agent spreads a harmful objective across a workflow of its chosen length, and a principal spends a budget of reviewer reads. If recognition is convex in the share of a plan that a reviewer reads, the agent's best response is to spread harm evenly, and at equilibrium no per-step review policy, however it selects steps, beats uniformly random review; at a fixed audit rate, per-step detection decays as in workflow length. Reading steps jointly at an unpredictable location escapes this decay and dominates per-step review by a factor . We measure the recognition exponent and test the theory on 255 harmful DecomposedHarm tasks with matched benign controls. On its agent tasks at a 1% false-positive budget, per-step review detects 0.15–0.22 of harmful tasks however many steps it reads, while a random window of four steps detects 0.40–0.50. The window's advantage is significant in eleven of twelve headline comparisons, holds when reviewers read only 18–35% of each workflow, and degrades but survives an agent that rewrites its plan to evade per-step review, which drives per-step detection to near zero. Targeting, the standard prescription in the literature, pays off mainly once evidence is aggregated: at tight budgets the strongest budget-matched policy we test reads the window the monitor finds most suspicious. Monitors should aggregate evidence across steps before deciding, and safety measured on fixed decompositions can overstate safety against agents that choose their own.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI control, trusted monitoring, LLM agents, oversight, decomposition attacks, audit allocation, inspection games, game theory, AI safety

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_benwOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

This paper introduces a theoretical model for how monitoring quality decays as a harmful trajectory gets split into steps in an adversarial inspection game. It claims that, if monitoring is performed per step, agents themselves can essentially decide what is being monitored by splitting up a harmful trajectory into separate smaller steps that may be harder to monitor. The paper finds that per-step monitoring is relatively weak and does not improve for larger step budgets in specific settings with a tight budget (although not in general). Moreover, targeting ‘suspicious’ steps does not help much. Instead, aggregating small random windows generally performs better than individual per-step monitoring. The paper also introduces a scattering of theoretical predictions that it attempts to verify experimentally. Experiments are performed on a fixed task-decomposition dataset, DecomposedHarm.

Critical strengths
  1. Monitoring is an important and current topic in agentic oversight, and a deeper understanding of how monitoring can be performed at low budgets is valuable to the community.
  2. The idea to model the agent as choosing the decomposition to make monitoring difficult is interesting, and it is clean enough to provide some interesting and testable hypotheses.
Critical weaknesses
  1. The experimental verification of the hypotheses of the theoretical model are only verified on a single static benchmark, DecomposedHarm. In particular, the hypotheses are not tested in an actual agentic setting, where an agent chooses how to decompose tasks. Instead, the experiments show that an LLM reviewer improves when it sees small windows of steps in a plan, compared to seeing individual steps only. In this benchmark, the plans are all pre-written. Given that the agentic decomposition is one of the main motivations of the theory, this seems like a significant gap.
  2. The paper is extremely dense, especially in sections 5 and 6, and difficult to parse as a result. This makes it hard to tell to what degree the experiments actually support the rest of the theoretical predictions. For instance, as far as I can tell, the experiment that fits the model parameter p treats every step as containing a uniform fraction of the total harm, and the “monolithic statement” that specifies the full plan often directly states the harmful objective. Then, even if recognition is not convex in the number of steps, we would could still expect this procedure to give p > 1. Overall, I am not confident that the experiments support the theoretical claims, nor am I very confident in my ability to tell this from the paper.
Questions for authors

Can you comment on the confounding variables in the estimates of p and a in the DecomposedHarm benchmark. In particular, can you comment on 1) how you control for the fact that different representations (e.g., MONO, FULL) of amount of harmful evidence affect monitor performance beyond just the theoretical harm-mass, 2) map harm-mass of individual steps to the decomposed plans in the dataset, 3) control for the amount of harmful evidence as context increases when estimating the parameter a?

POST-REBUTTAL: I have increased my soundness rating to a 2, as well as changed my rating from a clear reject (1) to a weak reject (2) in response to the authors' comments. My remaining concerns are in a comment below, and appear difficult to fully address in the current reviewing round.

6 comments
AuthorsOpenReview ↗

Official Comment 1/2

Dear Reviewer benw,

Thank you for your helpful comments; they have significantly improved the paper! You identified a real measurement problem: a statement of the whole objective can be easier to recognize simply because it names the objective, so our original estimate of the recognition exponent combined the amount of evidence with how explicitly it was expressed. We replaced that comparison in Section 5 with two controlled contrasts (Q2 and Q3 below). At the 2% target, replacing benign filler with harmful-plan steps at a fixed length of four raises the joint read's lead over the best separate verdict from 4 to 37 points, and adding seven benign steps around one fixed harmful step lowers joint detection from 32% to 9%. The comparison between review policies never used that estimate, so we do not think the flaw carries over to the paper's main results (see W2 in the second comment). This first comment answers your three questions.

Q1: Clarity of evidence

The new experiment compares excerpts drawn from decomposed plans, all presented in the same format. It uses no monolithic statement as an endpoint. We vary the number of steps from the harmful plan and the number of benign filler steps, calibrating each condition on benign excerpts constructed the same way (Section 5, P2 and Appendix B.4). Replacing filler with a step from the harmful plan changes the wording and can change the token count even when the number of steps stays fixed, and Appendix C.2 says so.

Q2: Measuring the evidence in a step

The benchmark does not give us ground-truth shares, so the primary comparison uses a count of steps from the harmful plan. On a four-step excerpt we compare Sonnet's joint verdict with the highest separate verdict on those same steps.

Table R4: Gain of a joint verdict over the highest separate verdict on the same four steps, in percentage points (Figure 6).

Steps from the harmful plan5% target2% target1% target
1 of 4+3+4−2
4 of 4+22+37+29
Difference+19+33+31

Each difference has Holm-adjusted p=0.0015p=0.0015. Joint review gains more when several pieces of evidence are present, and this does not require assigning each piece an equal share. In plain terms, at a fixed excerpt length, doubling the number of harmful-plan steps more than doubles detection at the 2% and 1% targets in all eight estimates (five of the eight intervals exclude proportionality), and is close to proportional at 5% (slopes 0.97 to 1.05, and 0.73 for Sonnet at length four; Table 4).

No conclusion rests on the fitted pp. All six intervals for pp in Table 5 include one, and the estimate moves with the specification (1.00 to 1.26 with model-annotated shares, 0.66 to 0.94 under a power law; Appendix C.2), so the fits do not establish convexity and we report pp and aa only as descriptions of these excerpts. The support for compounding is the direct contrasts, Table R4 and the slopes of Table 4, over the one-to-four-step range we test. The policy comparisons use neither exponent.

Q3: Holding the text fixed as context increases

Within each draw, we keep the core text identical and add benign filler. We audited the text across 13,610 excerpts, including benign controls. The task is the bootstrap unit.

Table R5: Sonnet's detection as benign steps are added around fixed harmful steps.

Fixed harmful stepsBenign steps added5% target2% target
10 → 742% → 20%32% → 9%
20 → 664% → 40%49% → 23%

All four prespecified dilution contrasts have Holm-adjusted p=0.002p=0.002. The position of the evidence can change, and each condition has its own calibrated threshold. The comparison measures how calibrated detection changes with added context at fixed core text. In Table 5, the fitted dilution exponent a^\hat{a} ranges from 0.29 to 0.59 and the product (1−a^)p^(1-\hat{a})\hat{p} from 0.46 to 0.81 (with intervals crossing one in three of six cells). Proposition 3 isolates the fixed-threshold asymptotic regime where (1−a)p<1(1-a)p<1 would eventually favor isolated review, whereas Proposition 4 shows that at finite BB a single joint verdict is still favoured for any a<1a<1 because separate verdicts must each clear a higher threshold to hold the task-level false-positive rate (see W2 and Table 3).

Dilution of this size is why, on long workflows when reviewer attention is scarce, we recommend a joint read of a limited set of monitor-selected steps rather than spending the budget on the whole trajectory: on LinuxArena runs of 60 to 70 actions, the reviewer reads the monitor's top eight (second comment).

(continued in 2/2)

AuthorsOpenReview ↗

Official Comment 2/2

W1: Agent-written plans and executed workflows

We agree the gap you describe was real, and the revision addresses it. We do think the static benchmark was the right place to start. The theory says the agent's best response is to split its objective into steps that each look innocuous. DecomposedHarm is a published instance of that, with benign plans in the same format, so it tests what the paper claims about the reviewer on the kind of workflow the theory says the reviewer will face. The original benchmark held the decomposition fixed, apart from a separate rewriting check. The revision tests the agent's choice in three ways.

1. An agent that knows the policy. We give an agent a benign main task and a side objective and tell it how the plan will be reviewed. In the eight-step condition, gpt-4.1 returned plans for 70 of 88 objectives.

Table R6: Detection with eight steps or actions read, by false-positive target. Rows 3 and 4 are the executed LinuxArena runs of item 3.

5%2%1%
Agent-written plans (Table 7): random steps judged separately0.490.260.19
Agent-written plans: random steps judged together0.710.580.46
LinuxArena (Table 1): monitor's top actions judged separately0.530.390.36
LinuxArena: the same actions judged together0.850.800.79

At the 2% target on the agent-written plans, a random contiguous window gives 46% and the monitor-chosen window gives 62% (versus 30% for monitor-chosen steps judged separately). On the 34 plans whose side objective a strict model judge confirms, the gain at the 2% target was 31 points. Across conditions, on the 94 plans that cover the main task as fully as a typical benign plan, it was 14 to 18 points. A second generator, gpt-4o, gives 24% and 49%. When the side objective was given as a single statement alongside the main task, agents chose the component actions themselves and still distributed them over several steps; in all 336 plans, no step carried half the objective (Table 8). On those plans the joint read gains 13 (gpt-4.1) and 14 (gpt-4o) points at the 2% target, and 16 and 19 at 1%.

2. A forced allocation. Confining the objective to one, two, four or eight of twelve steps shows what each allocation pays: four-step targeted review falls from 94% to 55%, and the lead of a joint read of eight over separate review of eight goes from −2 to +20 points (5% target, Table 9).

3. Executed runs. The LinuxArena comparison uses 47 executed, successful attack runs and 200 honest runs. Judging the same eight monitor-selected actions together raises detection from 39% to 80% at the 2% target (paired gain 41 points, 95% interval [22, 55]).

On benchmark decompositions, on plans from an agent that knows the policy and is given the actions, and on executed LinuxArena attacks, reading the same budget together roughly doubles detection at tight false-positive targets; the gain is smaller when the agent also chooses the actions.

Two limits and one check. The agent-written plans are written, not executed, and LinuxArena is executed but its agent did not know our policy; the two settings are meant to be read together. With only four reads in the gpt-4.1 conditions announcing isolated or four-step review, the gain was 4 to 5 points, (Table 6); with gpt-4o it was 16 to 17, and with eight reads it is 15 to 32. The check is that the gain is not bought with false alarms: on held-out benign plans the joint read flags 3.0%, 0.8% and 0.3% at the three targets, against 4.8%, 1.0% and 0.2% for separate review.

W2: Policy comparisons and fitted exponents

The comparison between review policies uses their verdicts directly. On the same four sampled steps, a single joint verdict beats the best of five ways of pooling separate scores by 21 to 24 points across the three matched false-positive targets (Table 3), and a pooler trained on harmful labels by 22 to 24 points (16 to 24 at matched rates, Appendix C.1). With every policy calibrated on the same 583 benign tasks, the random window is ahead of isolated review in all twelve comparisons, by 11 to 29 points. Neither fitted exponent enters these comparisons, and Proposition 4 explains why a gain is expected even under linear recognition: every separate verdict must clear a higher threshold to hold the task-level false-positive rate.

Section 5 now presents the review-policy comparison first and the evidence about its mechanism separately, as four predictions each stated before their test.

Thank you for identifying where the original presentation blurred those two claims. We hope the controlled experiment and the separation of the policy comparison from the exponent fits resolve your concerns about soundness, and that you will consider revisiting your scores.

I appreciate the work the authors did in addressing my comments and questions. The new experiments go a long way towards addressing my concerns. In particular, the authors demonstrate that it is feasible in real agent plans for harm to be distributed such that detection is improved by joint monitoring of individual steps. As a result, I am willing to raise my score to 2. However, I still think there are some issues with the paper that make me uncomfortable recommending acceptance. Some number of issues:

  1. The paper's experiments do not distinguish between agents writing plans to avoid monitors, or agents simply writing plans over multiple steps naturally (and thus decomposing harm naturally), and hence the connection the authors make between their experiments and theory is overstated. E.g.:

On p.9:

“This empirically confirms our theoretical finding that agents that choose their own allocation spread harm across workflow steps…”

“…joint review at an unpredictable location leaves the malicious agent with no good response (Theorem 2).”

On p.10:

“The agent’s equilibrium/unconstrained allocation…”

There are some controls that could help further distinguish the two hypotheses: a) Have the agent generate plans without access to monitor instructions, and see if the plans decompose differently. b) Do an iterative experiment where agents can adapt their plans after monitor feedback, and compare to agents that can adapt their plans with only task feedback.

  1. I believe there is an error in their interpretation of Prop 4. in this rebuttal:

Proposition 4 shows that at finite B a single joint verdict is still favoured for any a < 1

I believe they only show this asymptotically as B grows (p.18, final line of the proposition statement).

  1. I agree with reviewer EjKk that the paper still suffers from clarity issues, and that presentation needs to be improved. I appreciate the authors' efforts to rewrite sections of the paper, but difficulties remain and I do not believe it is upon the reviewers to point to every offending passage. In particular, the paper's prose is 'mannered' to a degree that genuinely hinders readability.
Reviewer_EjKkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents some theoretical (and empirically supported) experiments associated with monitoring agents. They assume that there is an agentic system that is given a task. This agent can be either honest or malicious. In the latter case, the agent attempts to also complete a secret harmful subtask. It will conceal this in its completion of the larger task. Simultaneously, they model an adversarial scenario in which there is a reviewer who is monitoring the agent as it works. The goal of a reviewer is to identify the harmful task and, downstream, stop it from completion.

In their work, they present a theoretical model for understanding the dynamics between the agent and reviewer. In particular, they are interested in understanding whether there exist any particular monitoring strategies that could be maximally effective in a worst case scenario. They show that, under a series of relatively reasonable assumptions, a strategy in which the monitor is able to check a window of substeps during task completion, can achieve best possible chances for identifying harm.

Personally I found the idea, theoretical findings, and experimental evaluation to be quite interesting. However, the paper could do with some improvements to clarity for readers who are not as familiar with certain aspects of the theoretical framing. For more details, please see weaknesses section below.

Critical strengths

I think the topic is really interesting. The problem is well motivated. The theoretical model feels rather reasonable and the empirical experiments validate it's applicability to real datasets.

Critical weaknesses

Clarity.

Notational/Theoretical.

First, it would be beneficial for an un-initiated reader to go over the intuition behind the notations with a simple example. This would help immensely with clarity. Second, it would be nice to have a table in the appendix where all the notation is concretely tagged and described in natural language. Third, I think it would be good to have a bit more concreteness about the assumptions. For example, I think the monitor score equation S_t=h_t+\epsilon_t be an assumption. Fourth, a little bit more "sign posting" would be nice. Not everyone is aware of what a zero-sum game actually entails and what a saddle point ultimately means. I think it would really help with bringing the main message of the paper home.

Motivation

It seems like you have multiple models/participants: an agent that is engaging in a task (either malicious or harmless), a monitor (also played by an agent), and then a human reviewer as well (that sometimes appears in the narrative but is simulated by a more expensive model in the experiments). Would be nice if this could be a bit more clear since I definitely found it rather confusing as I was reading and had to go back several times.

Experiments

This is the section that definitely needs the most work. It was really challenging for me to understand the findings of your results. For example, "per-step review is capped, whatever it targets, and one window lifts the cap" is just too condensed to make a lot of sense. In the paragraph, there are loads of quantitative values but not so much intuition for: what this means and how it can generalize beyond your specific dataset. Just in general... this section felt much too condensed and very hard to decipher. Even Table 1 had no guidance in the caption for how to interpret the results.

Questions for authors

I have no questions.

4 comments

Dear Reviewer EjKk,

Thank you for the encouraging review, and for a list of requests concrete enough to act on one by one. Your review raised no concerns about soundness; each weakness you listed is primarily about presentation, and we hope each is addressed in the revised PDF, which we reviewed throughout for clarity. We group our response under the two areas of your review: the clarity of the setup and theory, and the experiments.

1. Clarity of the setup and theory

  • Who the participants are. Figure 1 now shows who does what. The principal announces a review policy, the agent chooses its workflow, a cheap monitor scores every step, and the reviewer reads only the steps the policy selects and decides whether to flag the workflow. The principal and the agent are the two strategic players. Monitor and reviewer are language models in our experiments; a human reviewer motivates the attention budget.
  • A worked example. Section 3 now works through four steps, two reads, total harm normalized to one, and q(H)=H2q(H)=H^2. If each step carries a quarter of the objective, a single step is recognized with probability 1/161/16. Two separate reads detect with probability 1−(15/16)21-(15/16)^2, about 12%. One joint read sees half the objective and detects with probability (1/2)2(1/2)^2, or 25%.
  • Explicit assumptions. As you suggested, Section 3 now labels St=ht+ϵtS_t=h_t+\epsilon_t as Assumption 3, where hth_t is the amount of the side objective carried by step tt and ϵt\epsilon_t is monitor noise.
  • A notation table. Appendix D, Table 10 collects the symbols and their meanings.
  • Signposting. Each theorem and lemma is now followed by intuition for each of its parts, and terms that may be new to some readers are defined in words. For example, "zero-sum" means the principal seeks to maximize the same detection probability that the agent seeks to minimize, and at a saddle point each is doing as well as it can given the other's choice.

2. Experiments

  • How Section 5 is organised. It now states each theoretical prediction before its test, takes the experiments in order with robustness checks, and explains each result in plain English. Submitted Table 1 is now Table 2 (Appendix C.1), and its caption spells out how to read each cell (for example, that +0.32+0.32 means 32 percentage points more harmful tasks are caught).
  • What the main result means (the sentence about per-step review being "capped"). That wording left out what the budget buys and what the detection rate measures. On DecomposedHarm we give the Sonnet reviewer a budget of four steps and calibrate each policy to flag 1% of benign plans. Judging the four highest-scoring steps separately detects about 18% of harmful plans. Reading one randomly located window of four steps detects 50%, a difference of 32 percentage points. One reason is that two actions can each look routine on their own, yet together provide evidence of a side objective. Even on the same four sampled steps, one joint judgment detects 21 to 24 points more harmful plans than the best of five rules for combining the four separate scores (Table 3).
  • How robust it is. Across both reviewers and all three false-positive targets the window's gain is 13 to 32 points, and eleven of the twelve comparisons are significant after correction for multiple comparisons (Table 2). It does not require a strong reviewer: Haiku sees similar improvements when it reads steps jointly.
  • How the results generalize. Your comment on generalization pushed us to test whether the comparison carries beyond this benchmark, and we added two settings.
    • Plans an agent writes itself. An agent hides a side objective inside a main task after being told eight steps will be read together. Judging eight random steps separately detects 26% of these plans; judging them together detects 58% (2% target, Table 7). Left free to choose, the agents also spread the side objective across several steps (Table 8).
    • Executed runs. On LinuxArena, where agents carry out software-engineering tasks, reading the same eight monitor-selected actions together detected 80% of successful attacks, versus 39% when judged separately (2% target, Table 1).
  • Where reading together should not help. When the whole objective sits in one step there is nothing to add up, and we observe no gain there, as the theory says (Section 5, P4).

With the presentation revised and the comparison repeated in two further settings, we hope you will consider revisiting your score, and we are happy to clarify anything further.

Thank you for all your hard work and your additional experiments. I have looked over the updated version of the paper. While Figure 1 is great for explaining the flow of the paper. I think that there still needs to be some significant effort in revising the clarity of the text.

Let me give some concrete examples (these are, by no means exhaustive):

A limit on per-step review. We formulate an inspection game in which the inspectee allocates a fixed objective across the very units the inspector samples, a mechanism that prior inspection games do not model (§2), and we solve it exactly. The solution separates two questions that fixed-trajectory evaluations treat together. Reading the monitor’s highest-scoring steps weakly beats random selection on every fixed workflow, for any continuous noise law. Once the agent chooses the workflow, as is now standard in AI applications, it does not: the agent’s best response is to spread the objective evenly, and against that no per-step rule that reads B steps in expectation guarantees more than random selection, whose worst-case detection we call V(B,n) (Theorem 1). The even split is derived, not assumed, and it depends on the policy: against a reviewer that reads the lowest-scoring steps, concentrating beats spreading (Proposition 1). Under q(H) ≍Hp with p > 1, V decays as Θ(n1−p) at a fixed audit rate, so that reading a constant fraction of a longer workflow does not preserve the guarantee (Corollary 1)

This is taken directly from the revised version of the draft and it is overwhelmingly dense to read and interpret even though I have already read the entire paper and feel I have a relatively good grasp of its contents. For example "The solution separates two questions that fixed-trajectory evaluations treat together" -- you must now state what those questions are. Similarly, "Once the agent chooses the workflow, as is now standard in AI applications, it does not: the agent’s best response is to spread the objective evenly, and against that no per-step rule that reads B steps in expectation guarantees more than random selection, whose worst-case detection we call V(B,n) (Theorem 1)" is an extremely long sentence that is quite hard to follow.

The reason a joint read helps determines when it stops helping. Harmful steps read together should reinforce one another, and benign steps read alongside them should blur the picture. To separate the two, we cut excerpts containing one, two or four steps of a harmful task, pad them with benign filler steps, and have the reviewer read each excerpt jointly. Adding filler while the harmful steps stay fixed isolates dilution; swapping filler for harmful steps at a fixed length isolates evidence.

"The reason a joint read helps determines when it stops helping" --> Feels like very confusing language.

I believe that this work is interesting but I also think that presentation is a crucial component of a successful contribution. As such, I am not ready to raise my score.

Dear Reviewer EjKk,

Thank you for re-reading the manuscript and for pointing out specific sentences that remained overly dense. We totally agree that clear exposition is essential, and that the main paper needed further editing.

We have uploaded an updated PDF with a comprehensive pass through the entire paper, especially focusing on the model, experiments, analysis, and intuition.

  1. Section 1: We broke up long multi-clause sentences into shorter, single-idea sentences and explicitly stated the two questions separated by our game-theoretic formulation: (1) What can per-step review guarantee on a fixed workflow? and (2) But what if the agent chooses the workflow knowing the policy?
  2. Section 5: We tried to clearly explain the motivation behind each experiment and connect it to our goals, in a much clearer manner than before.
  3. Broader readability and clarity across the paper: We also audited the rest of the main text to break up long confusing sentences, standardize policy terminology across text, figures, and tables (adding a full reference table in Table 11), clarifying transitions, modifications, extensions, and results.

We hope the updated manuscript reads much more smoothly and addresses your clarity and presentation concerns. Please let us know if any specific passage still needs clarification, because we would be glad to refine it further. We sincerely thank you for working with us to improve the quality of the paper!

Dear Reviewer EjKk,

As the discussion window nears its close, we wanted to send a brief follow-up: following both your feedback and Reviewer benw's note on presentation, we completed an additional pass across the entire paper in our latest uploaded PDF. We would be grateful to know whether the updated manuscript resolves your presentation concerns.

As an example, here are the two passages you quoted, as they read in the current PDF:

Contributions (§1):

We model oversight as an inspection game (Figure 1): a principal commits to a review policy, and an agent chooses its workflow knowing that policy. In classical inspection games the inspectee chooses whether or when to violate, and the steps the inspector samples are taken as given (§2). In ours the agent also chooses how much of its side objective each reviewed step carries, shaping the very evidence the reviewer sees. Solving this game separates two questions that empirical fixed-workflow evaluations treat together. What can per-step review guarantee on a fixed workflow? Reviewing the monitor's top-scoring steps detects harm at least as often as reviewing random steps. But what if the agent chooses the workflow knowing the policy? In that case, targeting loses its edge: the agent's best response is to spread its side objective evenly across steps. Every step then carries the same amount of evidence, making the monitor's ranking uninformative, so no per-step policy, targeted or otherwise, can guarantee a higher detection rate than random review (Theorem 1). When recognition is convex (small fragments are recognised less than proportionally), this guarantee also shrinks as workflows lengthen, even when the reviewer reads a fixed fraction of every workflow (Corollary 1).

P2 opening (§5):

From our model, we expect two effects when a reviewer reads several steps together. As in P1, evidence compounds: several pieces of a side objective are easier to recognise than one. However, the model also cautions that benign steps can dilute the evidence: unrelated text around the evidence makes harm harder to recognise. To measure each effect separately, we build excerpts that contain one, two or four steps of a harmful plan plus benign filler steps, and the reviewer reads the whole excerpt. For dilution, we keep the harmful steps fixed and add filler; for compounding, we keep the length fixed and swap filler for harmful steps.

We revised the rest of the paper for clarity and presentation the same way, and had outside colleagues check the main text for clarity. We think these rounds of revisions have tremendously improved the clarity and presentation of the paper from the original submission, and thank you for your helpful comments. Thank you again for helping us improve the paper!

Reviewer_sLjZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper considers a variant of inspection games between a malicious AI agent and a monitor. Whereas usual inspection games would have a fixed sequence of steps from nature, the AI agent's choices inherently determine the workflow trajectory that is to be inspected. If the monitor's ability to detect misalignment is convex in the amount of evidence it acquires across different steps, then the paper shows that the agent's optimal strategy is to "equally spread" the harmful steps while the monitor's optimal strategy is to inspect uniformly at random proportional to a fixed review budget. The authors empirically test their theory's predictions on DecomposedHarm, which is a benchmark consisting of harmful objectives for an AI agent.

Critical strengths

The model and theory are an attempt at addressing an important problem in the safety of agents, i.e. treating the decomposition of the task as a strategic decision by the agent. The equilibrium result is clean and clear. The experiment section is decently comprehensive.

Critical weaknesses

The mathematical machinery is standard and not a novelty of its own. Given the paper is largely theoretical this is a big weakness. The experiments are also not really an instantiation of the theory's setting to a meaningful enough degree, the benchmark is meant for harm tasks not really trying to do a task while possibly hiding harm. The actual equilibrium results are not really tested.

Questions for authors

Can you clarify why DecomposedHarm is suitable enough for the experiments compared to an agent working on a main task while secretly inserting a vulnerability?

If harm is spread uniformly doesn't this just make scores carry no real meaning at a given step in a trajectory so random inspection just makes sense by construction, no? Doesn't this abstract away too much about the importance of actual sequence of actions and is there that much deep to Theorem 2?

4 comments
AuthorsOpenReview ↗

Official Comment 1/2

Dear Reviewer sLjZ,

Thank you for the review, and for asking which part of the equilibrium result follows from an even split and which part we need to prove. Your question helped us clarify the argument in Section 4.1. This first comment covers the theory (Q2, W1); the second covers the experiments (Q1, W2, W3).

Q2: "Random inspection just makes sense by construction … is there that much deep to Theorem 2?"

As you point out, once the agent splits the objective evenly, every step has the same recognition probability and any rule reading exactly BB steps does equally well. This is part (iii) of the submitted version's Theorem 2 (now Theorem 1), and the revision says it is immediate. The theorem as a whole does not hold by construction, however: the even split is something it has to derive.

The remaining question is whether an agent facing a reviewer who targets the highest scores could lower detection still further by choosing an uneven split. Parts (i) and (ii) settle that question in one line: for every allocation hh,

Dτ(h)≥Dρ(h)≥Dρ(un)=Dτ(un)=V(B,n),D_\tau(h)\ge D_\rho(h)\ge D_\rho(u_n)=D_\tau(u_n)=V(B,n),

where DD is the detection probability, τ\tau and ρ\rho are targeted and random review, and unu_n is the even split. The first inequality is part (i): targeting detects at least as often as random selection. The second is part (ii): random selection detects least at the even split. So no uneven split does better for the agent against either policy. Neither inequality is immediate:

  • Part (ii). The proof needs to control the probability of at least one recognition, since two recognitions on one workflow still count as one detection. Jensen's inequality bounds only the expected number of recognitions; closing the gap takes Maclaurin's inequality for symmetric means.
  • Part (i). Policies must be compared as distributions over read sets. With recognition probabilities (0.5,0.5,0,0)(0.5, 0.5, 0, 0) and two reads, reading steps 1 and 2 or steps 3 and 4 detects with probability 0.375, while reading steps 1 and 3 or steps 2 and 4 detects with probability 0.5, although every step is read half the time under both. The proof therefore couples targeted and random review through one draw of the monitor noise.
  • The even split depends on the policy. Against a reviewer that reads the lowest-scoring steps, concentrating the objective in one step does better for the agent (Proposition 1).

The lesson is not that the reviewer should pick steps at random. It is that once the agent spreads its objective evenly, no rule for picking which BB steps to judge one at a time detects more often than a random pick, which detects with probability V(B,n)V(B,n). To do better, the reviewer has to judge the selected steps together (Theorem 2).

W1: "The mathematical machinery is standard"

The inequalities we use are classical, and Appendix A now says so in its first paragraph. We respectfully disagree that this is a weakness, because the primary theoretical contribution is the model. It lets the agent choose how much of its objective each inspected unit contains; in prior inspection games (Avenhaus et al.; von Stengel) the inspectee chooses whether or when to violate, and the inspected units are given. The solution then says three things that we see as novel theoretical contributions.

  1. Targeting helps on a fixed workflow but not in the worst case. A reviewer can benefit substantially from targeting on a fixed workflow and still gain nothing in its worst-case guarantee from better targeting. The data show the same pattern: with one step read at the 5% target, targeted review leads random review by 45 points in the query domain, where one question carries the harm, by 24 to 31 points on agent-written plans, where about half the steps carry the objective, and by 15 points on the benchmark's agent plans, where every step does (Appendix C.3).
  2. The per-step guarantee decays with length. Under q(x)≤κxpq(x)\le\kappa x^p with p>1p>1, it decays as n1−pn^{1-p} even when a fixed fraction of the workflow is read (Corollary 1).
  3. Changing what the reviewer judges raises the guarantee. Under perfect aggregation, a joint read of BB uniformly sampled steps guarantees q(Bσ/n)≥V(B,n)q(B\sigma/n)\ge V(B,n) (Theorem 2 and Appendix A.5).

Sequence of actions

Your question about sequence identifies a real limit of the model, which we left out of scope for tractability. Two workflows with the same allocation can have different causal dependencies or different indispensable actions, and our game abstracts from those differences. Corollary 2 varies how many steps carry the objective, and LinuxArena tests review on ordered executions. How detection depends on the order of the actions itself remains open, and Section 6 now says so.

(continued in 2/2)

AuthorsOpenReview ↗

Official Comment 2/2

Q1, W2: Is DecomposedHarm a suitable test?

DecomposedHarm tests one half of the theory directly, and we added your setting for the other half. The theory makes one claim about the agent (it will spread its objective) and one about the reviewer (against a spread objective, reading steps together detects more than judging them separately). DecomposedHarm tests the second: it fixes the decomposition so only the review policy changes. A main task adds steps that carry none of the objective, which is the case k<nk<n of Corollary 2.

Following your suggestion, the revision adds two settings with both a main task and a hidden side objective.

  • Experiment 1: an agent hides a side objective inside a main task, knowing how it will be reviewed (Table 7). We give gpt-4.1 a benign main task and a harmful side objective to complete in one plan, and tell it in advance that a reviewer will read eight of its steps together. We review the 70 plans with a budget of eight steps at a 2% false-positive target. Judging eight steps one at a time detects 26% (random steps) or 30% (monitor-targeted steps). Reading eight steps together detects 58% (random eight-step set) or 62% (monitor-chosen window). A second generator, gpt-4o, gives 24% vs. 49%. Being told the policy did not change what the agent did: it split the side objective the same way under every instruction (Table 8). This experiment therefore cannot show what a different split would have cost it; the forced-allocation test under W3 below does.
  • Experiment 2: an agent carries out the attack (Table 1). LinuxArena records agents completing a software-engineering task while also completing a hidden side task, such as introducing a remote-code-execution vulnerability. This is the setting in your question. In each of 47 successful attack runs (calibrated on 200 honest runs) the monitor picks the eight most suspicious actions, about 13% of the run, and the reviewer sees these exact eight actions in both conditions. Judged one at a time, they reveal 39% of the attacks at the 2% target. Read together, the same eight actions reveal 80%.

In the setting you proposed, the theory's prediction holds. Theorem 2 and Proposition 4 say that a fixed budget detects more when read together than when spent on separate verdicts. Both experiments show this at all three false-positive targets (5%, 2%, 1%), so the benchmark result was not an artefact of tasks that lack a main task.

W3: "The actual equilibrium results are not really tested"

The original draft tested the reviewer's side; the revision adds three checks on the agent's side.

1. Left free, the agent spreads its objective. In all 336 generated plans no single step carries half the objective; the agent places it in about half of the plan's steps, in near-equal shares (Table 8). This includes 110 plans written without a list of required actions, so the spread is not inherited from the benchmark. On those plans the joint read's gain is smaller: 13 to 14 points at the 2% target, and 16 to 19 at 1%. Theorem 1 derives an even split over all steps under costless divisibility; the plans support the qualitative prediction of spreading and leave optimality open.

2. Spreading is what defeats targeting (Theorem 1). What an agent chooses does not show that the choice pays, so we forced the objective into k=1k=1, 2, 4 or 8 of twelve steps (56, 54, 44 and 28 plans).

Table R2: Detection under a forced allocation, 5% target (Table 9).

Steps carrying the objective (kk)1248
Targeted, 4 steps judged separately0.940.610.550.55
Random, 4 steps judged separately0.340.380.430.42
Random, 8 steps judged separately0.640.510.520.47
Random, 8 steps judged together0.620.600.680.68

Random isolated review falls as predicted with 8 reads (0.64 to 0.47) and at the 1% target with 4 reads (0.31 to 0.22); it is flat with 4 reads at 5%, where P2 finds recognition near-proportional. Because refusal rates differed across kk, we repeated the comparison on the 20 objectives that have both a one-step and an eight-step plan: spreading lowers targeted detection by 33 points [13, 54], and targeting's lead over random review shrinks by 44 points [22, 63] (95% intervals).

3. Reading together gains as the objective spreads, and not before (Corollary 2). In Table R2 the joint read trails separate review by 2 points at k=1k=1 and leads by about 20 points at k=8k=8 (on the 20 paired objectives, the lead grows by 30 points [6, 53] at 5%). Corollary 2 also predicts levels at k=1k=1: using targeted detection (94%) for whole-objective recognition, it gives 31%, 63% and 63% for four and eight random isolated reads and one joint read of eight. We observe 34%, 64% and 62%.

We hope this addresses your concerns and that you will consider revising your score.

I thank the reviewers for the response. While I appreciate that the authors consider the model to be the contribution of this paper, I think it does not stand enough on its own without being able to derive more nontrivial theory or show more compelling experiments.

Dear Reviewer sLjZ,

Thank you for following up. To clarify, we do not view the model in isolation as the sole contribution; rather, the contribution is the exact minimax characterization of divisible oversight (and other theorems and propositions) and the empirical tests, some of which directly build on your original questions (Q1, W2, W3). To use the remaining discussion time well, we would like to clarify which aspects of the revised results remain insufficient in your view.

1. Experiments (Q1, W2, W3).

In your review, you asked how our findings carry over to "an agent working on a main task while secretly inserting a vulnerability" and noted that the equilibrium predictions were untested. In the revision, we added executed LinuxArena trajectories where an agent completes a software-engineering task while secretly inserting a vulnerability (such as remote code execution), where joint review reading the same monitor-selected actions together rather than separately raises detection from 39% to 80% at a 2% false-positive target (Table 1).

We also tested the agent's equilibrium incentives directly via 336 agent-written plans and a forced-allocation sweep (k=1,2,4,8k = 1, 2, 4, 8, Table 9), where spreading cuts targeted detection from 94% to 55% and empirical rates match Corollary 2 within 3 percentage points. These experiments test both the predicted benefit of spreading and the benefit of joint review.

2. Theory (W1, Q2).

We agree the proofs use classical mathematical tools. Alongside the minimax game values (Theorems 1–2), the results we believe are non-obvious and important theoretical contributions are: (i) that better targeting adds nothing to the reviewer's worst-case guarantee once the agent adapts (Theorem 1), (ii) the exact partial-spread trade-offs between targeting and aggregation (Corollary 2), and (iii) that one joint verdict beats separate verdicts at the same budget even when recognition is linear (p=1p = 1) via the task-level false-positive threshold tax (Proposition 4).

Knowing what concrete issues remain in your view would be very helpful for further improving the paper.

Thank you!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Overall Response & Summary of First Round Revisions

Dear Area Chair and Reviewers,

Thank you all for the thoughtful reviews! We are glad that all three reviewers found the problem important and the model clean enough to test ("the equilibrium result is clean and clear", sLjZ; "interesting and testable hypotheses", benw; "the theoretical model feels rather reasonable", EjKk). We respond to each reviewer separately; this comment covers the questions shared across reviews and what changed.

Why we started with DecomposedHarm, and what we added

Several reviews asked how well results on pre-written plans carry over to agents that choose their own workflows. Our game has two sides. Theorem 1 says the agent's best response is to split its objective into steps that each look innocuous. Theorem 2 says what the reviewer should then do. DecomposedHarm supplies the first: published decompositions of harmful objectives into individually innocuous steps. It lets us apply different review policies to the same plans, so it tests the reviewer's side of the game. What it holds fixed is the agent's move.

We added two experiments in Section 5 that do not hold it fixed: plans that agents wrote for a main task and a side objective after being told how they would be reviewed (Table 7), and executed LinuxArena runs in which an agent completes a software-engineering task and a hidden side task (Table 1).

Table R1: Joint against separate review at the same reviewer budget. Brackets are paired 95% intervals.

SettingWhat the comparison establishesResult at the 2% false-positive target
DecomposedHarm, same four sampled steps (Table 3)Can a joint verdict recover information lost when each step is scored separately?Sonnet's joint verdict beats the best of five pooling rules by 24 points [13, 33].
Agent-written plans, required actions listed (Table 7)Does the advantage survive when the agent writes the plan knowing that eight steps will be read together?Random separate review detects 26%, a random joint read 58%. Paired gain 32 points [20, 42].
Agent-written plans, objective only (Appendix C.3)Does it survive when the agent also chooses the component actions?Gain of 13 points (gpt-4.1) and 14 (gpt-4o); 16 and 19 at the 1% target.
Executed LinuxArena runs (Table 1)Does joint review help on actions an agent actually took?On 47 successful attacks, separate review of the monitor's top eight actions detects 39%, joint review of the same actions 80%. Paired gain 41 points [22, 55].

Reading together about doubles detection in rows 2 and 4. When the agent also chooses the actions (row 3), the gain is smaller and clearest at the 1% target.

How agents allocate, and what it costs them (sLjZ, benw)

We measured how agents distributed their side objective and how detection changed when we varied that distribution (Tables 8 and 9).

  • Agents spread. In 336 plans, no single step carried half the objective; agents placed it in about half of a plan's steps, in near-equal shares. This supports the qualitative prediction of spreading. It does not show that the agents found the optimum.
  • Spreading pays. With the objective forced into one step, targeted review detects 94% and a joint read gains nothing. Spread over eight steps, targeted review falls to 55% and the joint read leads separate review by about 20 points (5% target).
  • Levels match. With all the harm in one step, the three detection rates Corollary 2 predicts are each within three points of what we observe.

Evidence and context (benw)

Reviewer benw was right that our original estimate of the recognition exponent combined the amount of evidence with how explicitly it was stated. We replaced it with controlled contrasts on 13,610 excerpts, and we report the fitted recognition exponent as descriptive only: its intervals include one, so the fits do not establish convexity. The comparison between review policies does not use those fits: on the same four sampled steps, one joint verdict beats the best of five rules for pooling separate verdicts by 21 to 24 points (Table 3).

Presentation (sLjZ, EjKk, benw)

We revised the whole paper for clarity. Figure 1 and a worked example in Section 3 introduce the participants and walk through the review process. Each part of each theorem and lemma has accompanying intuition. Section 5 states each theoretical prediction, presents the experiments in order with robustness checks, and explains each result in plain English. Appendix D is a table of notation.

Numbering

Submitted Theorem 2 is now Theorem 1 (value), submitted Theorem 4 is Theorem 2 (joint read), submitted Corollary 3 is Corollary 1 (length), and submitted Propositions 5 to 9 are Propositions 1 to 5. Corollary 2 (concentration) is new.

We are grateful for reviews that made this a clearer and stronger paper, and are happy to answer further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.