acceptodds
Under review as a conference paper at ICLR 2027

Negative-Space Security: Omission Attacks and the Observability Bound on AI Monitoring

27 views

Abstract

Monitoring changes to a codebase or system typically means inspecting what was done: the diff that was produced or the commands that were run. We study omission attacks, where harm comes from a required action that was silently not performed. Because the omitted action leaves no trace in the diff or transcript, omissions expose an observability limit: if honest and attack trajectories induce the same visible-input distribution, any monitor of that input is at chance. We introduce OmissionBench, which combines 94 incident-anchored pull-request triples (with honest, commission, and omission arms) and 285 live-agent LinuxArena sysadmin trajectories. Diff-only LLM monitors score omissions 0.05–0.13 AUROC below matched commission attacks; a Sourcegraph Cody-style rubric proxy evaluated through Claude Sonnet reaches 0.65 AUROC on omissions. In live trajectories, the strongest transcript monitor falls from 0.96 to 0.76 AUROC as attacker capability increases, indicating that it detects execution residue rather than omitted obligations. Detection recovers when monitors read an independent obligation list or post-execution state: obligation-aware prompting improves omission AUROC by up to +0.16, and a simulated-output state-audit surrogate reaches 0.952 AUROC at 100% recall and 16% FPR. Robust AI-control monitoring should audit what agents leave behind, not only what they do.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Omission, AI Control

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_oHdDOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies omission attacks where an agent causes harm by silently skipping a required action. It argues that monitors reading only diffs or transcripts cannot detect what those inputs do not contain. It introduces OmissionBench, with two parts: 94 synthetic, incident-anchored PR triples and 285 LinuxArena sysadmin trajectories. Diff-only LLM monitors score roughly 0.06–0.13 AUROC lower on omissions than on commissions. Transcript-monitor AUROC falls as attacker capability rises. A state-audit monitor, which receives simulated shell-check responses from a hidden oracle, reaches 0.952 AUROC. The paper concludes that monitors should audit post-execution state.

Critical strengths

The most interesting finding is in §3: transcript-monitor AUROC on the live-agent track drops from 0.955 to 0.758 as the attacker moves from Haiku to Opus. The authors read this as monitors flagging execution clumsiness. If it holds up, this matters for AI control, because it suggests action monitors may degrade exactly as agents become more capable. It is the one result that goes beyond a restatement of the paper’s premise, though it rests on few tiers and one task distribution.

Critical weaknesses
  1. Limited novelty and trivial core claim. The central thesis is that a monitor cannot detect information absent from its input. That is a direct consequence of the data-processing inequality and needs no benchmark to establish. The proposed method of checking post-conditions against the post-execution state, is standard practice in runtime verification, compliance auditing, and missing-check detection, all of which the paper itself cites (CRIX, RoleCast). Relabeling this as an agentic monitor contributes no new mechanism. Without a technically new method or a surprising empirical result on the headline claim, the contribution is mostly reframing.
  2. A large share of the paper disclaims its own results. The state-audit numbers come from simulated responses from a hidden oracle. State-oracle rows score AUROC 1.00 even for irrelevant predicates, which indicates label leakage. The human panel was simulated, the Cody result is a proxy, and the real-PR split does not reconcile. Table 1 and the scope paragraph exist mainly to narrow claims after the fact. The paper is also internally inconsistent: Limitations calls cross-family attackers untested, which section 3 contradicts.
Questions for authors

State monitors score AUROC 1.00 even with irrelevant predicates, which suggests the oracle’s output leaks the label. How is the 0.952 state-audit result not affected by the same leakage?

Reviewer_YYZ5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

placeholder. waiting for AC descision on desk rejection.

Critical strengths

placeholder. waiting for AC descision on desk rejection.

Critical weaknesses

placeholder. waiting for AC descision on desk rejection.

Questions for authors

placeholder. waiting for AC descision on desk rejection.

Reviewer_UDQdOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies omission attacks, where an AI agent causes harm by failing to perform a required action rather than taking an explicitly malicious one. The authors introduce OmissionBench, consisting of 94 PR triples and 285 LinuxArena trajectories. They find that diff-only monitors detect omission attacks less reliably than matched commission attacks, while transcript-based monitors show declining omission detection performance across the evaluated attacker models. They also propose agentic state auditing, where a monitor derives obligations from the task and checks the resulting system state. The approach achieves promising detection results, although the evaluation relies on simulated shell-check responses rather than actual execution.

Critical strengths
  1. Important problem formulation.

The paper highlights a meaningful limitation of action-based monitoring: harmful omissions may leave little observable evidence. This is particularly relevant for autonomous agents operating in security-sensitive environments.

  1. Useful empirical investigation.

The evaluation covers multiple monitoring approaches, models, and settings. The paired scenarios and diagnostic experiments, particularly the investigation of oracle leakage, provide useful insights into the challenges of omission detection.

Critical weaknesses
  1. The added value of agentic monitoring is not established.

The main concern is that the evaluation does not distinguish between the benefits of additional state observability and those of agentic capabilities such as autonomous obligation discovery, test selection, and adaptive investigation. The tool-access ablation shows that allowing tool calls improves AUROC from 0.873 to 0.971. While this supports the value of additional state information, it does not establish the incremental contribution of agentic reasoning. A non-agentic monitor receiving equivalent state observations or predefined checks might achieve similar results. The paper lacks a controlled comparison against such baselines. Furthermore, the state-audit experiments rely on simulated shell-check responses rather than real execution. Although the authors acknowledge this limitation, it remains unclear how well the results transfer to an actual environment. The acknowledged label leakage in the separate state-oracle experiments also raises concerns about possible shortcuts in simulated evaluations, although equivalent leakage has not been established for the agentic pipeline.

  1. Some conclusions about existing monitors and attacker capability are insufficiently supported.

The claim that transcript monitors detect execution clumsiness rather than omissions is not causally established. The authors observe decreasing detection performance against stronger attacker models, but do not directly measure whether redundant commands, execution errors, or other signs of clumsiness explain this trend. Task-level controls do not isolate this mechanism. More fundamentally, the cross-family capability analysis ranks attacker tiers using observed monitor AUROC, which is also the outcome being analyzed. This makes the monotonic ordering partly a consequence of the ranking procedure. The reported slope and statistical significance therefore do not independently establish a relationship between attacker capability and omission detectability. Although the within-Claude comparison uses recognizable model tiers, the broader cross-family conclusion requires an independently defined capability measure. Important baseline details also remain unclear. In particular, the information available to LinuxArena transcript monitors is not sufficiently specified, despite the demonstrated importance of task specifications. The Cody-style baseline reproduces a published review rubric through Claude Sonnet but is not validated against the original product, limiting conclusions about industrial code-review performance.

  1. The evidence for generalization to real-world pull requests is limited.

The PR-review experiments provide useful evidence within the constructed benchmark, but the real-PR evaluation has acknowledged methodological confounds. In particular, the authors note that the real-PR counts do not reconcile and that the omission split is confounded by empty patches and explicit obligation labels. These issues limit the usefulness of the real-PR results as evidence of external validity. Consequently, it remains uncertain how well the reported detection patterns generalize to realistic development workflows in which obligations may be implicit and omissions may be less easily identifiable.

Questions for authors
  1. Monitor inputs: Did the LinuxArena transcript monitors receive the complete task specification and security obligations, or only the execution traces? Does the reported performance degradation persist when task specifications are provided?

  2. Simulated tools: How are responses to arbitrary shell checks generated? Could the authors provide representative tool-call trajectories and explain how label leakage is prevented in the agentic evaluation?

  3. Agentic contribution: Have the authors compared the proposed monitor against a non-agentic baseline receiving equivalent state information or predefined checks? How often does the agentic monitor independently identify the correct obligation and select an appropriate check?

  4. Clumsiness hypothesis: Did the authors directly measure execution clumsiness and test its relationship to monitor predictions? Can attacker capability be defined using an independent metric rather than observed detection AUROC?

  5. Cody baseline: Which components of Sourcegraph Cody were reproduced, and was the implementation validated against the original system on a shared benchmark?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.