- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2(+1)
Read review
Paper summary
This paper introduces a theoretical model for how monitoring quality decays as a harmful trajectory gets split into steps in an adversarial inspection game. It claims that, if monitoring is performed per step, agents themselves can essentially decide what is being monitored by splitting up a harmful trajectory into separate smaller steps that may be harder to monitor. The paper finds that per-step monitoring is relatively weak and does not improve for larger step budgets in specific settings with a tight budget (although not in general). Moreover, targeting ‘suspicious’ steps does not help much. Instead, aggregating small random windows generally performs better than individual per-step monitoring. The paper also introduces a scattering of theoretical predictions that it attempts to verify experimentally. Experiments are performed on a fixed task-decomposition dataset, DecomposedHarm.
Critical strengths
- Monitoring is an important and current topic in agentic oversight, and a deeper understanding of how monitoring can be performed at low budgets is valuable to the community.
- The idea to model the agent as choosing the decomposition to make monitoring difficult is interesting, and it is clean enough to provide some interesting and testable hypotheses.
Critical weaknesses
- The experimental verification of the hypotheses of the theoretical model are only verified on a single static benchmark, DecomposedHarm. In particular, the hypotheses are not tested in an actual agentic setting, where an agent chooses how to decompose tasks. Instead, the experiments show that an LLM reviewer improves when it sees small windows of steps in a plan, compared to seeing individual steps only. In this benchmark, the plans are all pre-written. Given that the agentic decomposition is one of the main motivations of the theory, this seems like a significant gap.
- The paper is extremely dense, especially in sections 5 and 6, and difficult to parse as a result. This makes it hard to tell to what degree the experiments actually support the rest of the theoretical predictions. For instance, as far as I can tell, the experiment that fits the model parameter p treats every step as containing a uniform fraction of the total harm, and the “monolithic statement” that specifies the full plan often directly states the harmful objective. Then, even if recognition is not convex in the number of steps, we would could still expect this procedure to give p > 1. Overall, I am not confident that the experiments support the theoretical claims, nor am I very confident in my ability to tell this from the paper.
Questions for authors
Can you comment on the confounding variables in the estimates of p and a in the DecomposedHarm benchmark. In particular, can you comment on 1) how you control for the fact that different representations (e.g., MONO, FULL) of amount of harmful evidence affect monitor performance beyond just the theoretical harm-mass, 2) map harm-mass of individual steps to the decomposed plans in the dataset, 3) control for the amount of harmful evidence as context increases when estimating the parameter a?
POST-REBUTTAL: I have increased my soundness rating to a 2, as well as changed my rating from a clear reject (1) to a weak reject (2) in response to the authors' comments. My remaining concerns are in a comment below, and appear difficult to fully address in the current reviewing round.
6 comments
Official Comment 2/2
W1: Agent-written plans and executed workflows
We agree the gap you describe was real, and the revision addresses it. We do think the static benchmark was the right place to start. The theory says the agent's best response is to split its objective into steps that each look innocuous. DecomposedHarm is a published instance of that, with benign plans in the same format, so it tests what the paper claims about the reviewer on the kind of workflow the theory says the reviewer will face. The original benchmark held the decomposition fixed, apart from a separate rewriting check. The revision tests the agent's choice in three ways.
1. An agent that knows the policy. We give an agent a benign main task and a side objective and tell it how the plan will be reviewed. In the eight-step condition, gpt-4.1 returned plans for 70 of 88 objectives.
Table R6: Detection with eight steps or actions read, by false-positive target. Rows 3 and 4 are the executed LinuxArena runs of item 3.
| 5% | 2% | 1% | |
|---|---|---|---|
| Agent-written plans (Table 7): random steps judged separately | 0.49 | 0.26 | 0.19 |
| Agent-written plans: random steps judged together | 0.71 | 0.58 | 0.46 |
| LinuxArena (Table 1): monitor's top actions judged separately | 0.53 | 0.39 | 0.36 |
| LinuxArena: the same actions judged together | 0.85 | 0.80 | 0.79 |
At the 2% target on the agent-written plans, a random contiguous window gives 46% and the monitor-chosen window gives 62% (versus 30% for monitor-chosen steps judged separately). On the 34 plans whose side objective a strict model judge confirms, the gain at the 2% target was 31 points. Across conditions, on the 94 plans that cover the main task as fully as a typical benign plan, it was 14 to 18 points. A second generator, gpt-4o, gives 24% and 49%. When the side objective was given as a single statement alongside the main task, agents chose the component actions themselves and still distributed them over several steps; in all 336 plans, no step carried half the objective (Table 8). On those plans the joint read gains 13 (gpt-4.1) and 14 (gpt-4o) points at the 2% target, and 16 and 19 at 1%.
2. A forced allocation. Confining the objective to one, two, four or eight of twelve steps shows what each allocation pays: four-step targeted review falls from 94% to 55%, and the lead of a joint read of eight over separate review of eight goes from −2 to +20 points (5% target, Table 9).
3. Executed runs. The LinuxArena comparison uses 47 executed, successful attack runs and 200 honest runs. Judging the same eight monitor-selected actions together raises detection from 39% to 80% at the 2% target (paired gain 41 points, 95% interval [22, 55]).
On benchmark decompositions, on plans from an agent that knows the policy and is given the actions, and on executed LinuxArena attacks, reading the same budget together roughly doubles detection at tight false-positive targets; the gain is smaller when the agent also chooses the actions.
Two limits and one check. The agent-written plans are written, not executed, and LinuxArena is executed but its agent did not know our policy; the two settings are meant to be read together. With only four reads in the gpt-4.1 conditions announcing isolated or four-step review, the gain was 4 to 5 points, (Table 6); with gpt-4o it was 16 to 17, and with eight reads it is 15 to 32. The check is that the gain is not bought with false alarms: on held-out benign plans the joint read flags 3.0%, 0.8% and 0.3% at the three targets, against 4.8%, 1.0% and 0.2% for separate review.
W2: Policy comparisons and fitted exponents
The comparison between review policies uses their verdicts directly. On the same four sampled steps, a single joint verdict beats the best of five ways of pooling separate scores by 21 to 24 points across the three matched false-positive targets (Table 3), and a pooler trained on harmful labels by 22 to 24 points (16 to 24 at matched rates, Appendix C.1). With every policy calibrated on the same 583 benign tasks, the random window is ahead of isolated review in all twelve comparisons, by 11 to 29 points. Neither fitted exponent enters these comparisons, and Proposition 4 explains why a gain is expected even under linear recognition: every separate verdict must clear a higher threshold to hold the task-level false-positive rate.
Section 5 now presents the review-policy comparison first and the evidence about its mechanism separately, as four predictions each stated before their test.
Thank you for identifying where the original presentation blurred those two claims. We hope the controlled experiment and the separation of the policy comparison from the exponent fits resolve your concerns about soundness, and that you will consider revisiting your scores.
I appreciate the work the authors did in addressing my comments and questions. The new experiments go a long way towards addressing my concerns. In particular, the authors demonstrate that it is feasible in real agent plans for harm to be distributed such that detection is improved by joint monitoring of individual steps. As a result, I am willing to raise my score to 2. However, I still think there are some issues with the paper that make me uncomfortable recommending acceptance. Some number of issues:
- The paper's experiments do not distinguish between agents writing plans to avoid monitors, or agents simply writing plans over multiple steps naturally (and thus decomposing harm naturally), and hence the connection the authors make between their experiments and theory is overstated. E.g.:
On p.9:
“This empirically confirms our theoretical finding that agents that choose their own allocation spread harm across workflow steps…”
“…joint review at an unpredictable location leaves the malicious agent with no good response (Theorem 2).”
On p.10:
“The agent’s equilibrium/unconstrained allocation…”
There are some controls that could help further distinguish the two hypotheses: a) Have the agent generate plans without access to monitor instructions, and see if the plans decompose differently. b) Do an iterative experiment where agents can adapt their plans after monitor feedback, and compare to agents that can adapt their plans with only task feedback.
- I believe there is an error in their interpretation of Prop 4. in this rebuttal:
Proposition 4 shows that at finite B a single joint verdict is still favoured for any a < 1
I believe they only show this asymptotically as B grows (p.18, final line of the proposition statement).
- I agree with reviewer EjKk that the paper still suffers from clarity issues, and that presentation needs to be improved. I appreciate the authors' efforts to rewrite sections of the paper, but difficulties remain and I do not believe it is upon the reviewers to point to every offending passage. In particular, the paper's prose is 'mannered' to a degree that genuinely hinders readability.
Official Comment 1/2
Dear Reviewer benw,
Thank you for your helpful comments; they have significantly improved the paper! You identified a real measurement problem: a statement of the whole objective can be easier to recognize simply because it names the objective, so our original estimate of the recognition exponent combined the amount of evidence with how explicitly it was expressed. We replaced that comparison in Section 5 with two controlled contrasts (Q2 and Q3 below). At the 2% target, replacing benign filler with harmful-plan steps at a fixed length of four raises the joint read's lead over the best separate verdict from 4 to 37 points, and adding seven benign steps around one fixed harmful step lowers joint detection from 32% to 9%. The comparison between review policies never used that estimate, so we do not think the flaw carries over to the paper's main results (see W2 in the second comment). This first comment answers your three questions.
Q1: Clarity of evidence
The new experiment compares excerpts drawn from decomposed plans, all presented in the same format. It uses no monolithic statement as an endpoint. We vary the number of steps from the harmful plan and the number of benign filler steps, calibrating each condition on benign excerpts constructed the same way (Section 5, P2 and Appendix B.4). Replacing filler with a step from the harmful plan changes the wording and can change the token count even when the number of steps stays fixed, and Appendix C.2 says so.
Q2: Measuring the evidence in a step
The benchmark does not give us ground-truth shares, so the primary comparison uses a count of steps from the harmful plan. On a four-step excerpt we compare Sonnet's joint verdict with the highest separate verdict on those same steps.
Table R4: Gain of a joint verdict over the highest separate verdict on the same four steps, in percentage points (Figure 6).
Each difference has Holm-adjusted p=0.0015. Joint review gains more when several pieces of evidence are present, and this does not require assigning each piece an equal share. In plain terms, at a fixed excerpt length, doubling the number of harmful-plan steps more than doubles detection at the 2% and 1% targets in all eight estimates (five of the eight intervals exclude proportionality), and is close to proportional at 5% (slopes 0.97 to 1.05, and 0.73 for Sonnet at length four; Table 4).
No conclusion rests on the fitted p. All six intervals for p in Table 5 include one, and the estimate moves with the specification (1.00 to 1.26 with model-annotated shares, 0.66 to 0.94 under a power law; Appendix C.2), so the fits do not establish convexity and we report p and a only as descriptions of these excerpts. The support for compounding is the direct contrasts, Table R4 and the slopes of Table 4, over the one-to-four-step range we test. The policy comparisons use neither exponent.
Q3: Holding the text fixed as context increases
Within each draw, we keep the core text identical and add benign filler. We audited the text across 13,610 excerpts, including benign controls. The task is the bootstrap unit.
Table R5: Sonnet's detection as benign steps are added around fixed harmful steps.
All four prespecified dilution contrasts have Holm-adjusted p=0.002. The position of the evidence can change, and each condition has its own calibrated threshold. The comparison measures how calibrated detection changes with added context at fixed core text. In Table 5, the fitted dilution exponent a^ ranges from 0.29 to 0.59 and the product (1−a^)p^ from 0.46 to 0.81 (with intervals crossing one in three of six cells). Proposition 3 isolates the fixed-threshold asymptotic regime where (1−a)p<1 would eventually favor isolated review, whereas Proposition 4 shows that at finite B a single joint verdict is still favoured for any a<1 because separate verdicts must each clear a higher threshold to hold the task-level false-positive rate (see W2 and Table 3).
Dilution of this size is why, on long workflows when reviewer attention is scarce, we recommend a joint read of a limited set of monitor-selected steps rather than spending the budget on the whole trajectory: on LinuxArena runs of 60 to 70 actions, the reviewer reads the monitor's top eight (second comment).
(continued in 2/2)