acceptodds
Under review as a conference paper at ICLR 2027

Closing the Text Channel: Typed Adaptive Evidence Acquisition for AI Control

Abstract

AI control uses trusted monitors to detect unsafe behavior from powerful, potentially adversarial AI systems. In code-generation settings, however, the material being inspected, such as source code, comments, and execution output, may itself contain attacker-written text that misleads the monitor. For adaptive monitors, this creates an additional vulnerability: such text can influence not only the final judgment, but also what the monitor chooses to inspect next, thereby shaping the evidence used for subsequent decisions. We introduce *Typed Adaptive Evidence Acquisition* (TAEA), which addresses this problem by placing both evidence acquisition and final scoring behind the same trusted, typed evidence boundary. Trusted probes convert program structure and behavior into predefined structured observations, preventing unrestricted attacker-controlled text from directly influencing either process. We prove that, for a frozen monitor under shared trusted randomness, edits that preserve all registered measurements leave the entire inspection path, score, and verdict unchanged. Within this protected interface, TAEA interprets evidence in context and selectively acquires additional measurements when they are expected to improve the decision. Experiments on two code-sabotage benchmarks show that TAEA achieves the highest among evaluated monitors and is substantially more robust under the evaluated adaptive attacks.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Safety, AI Control, AI Monitoring

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_KKjAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies sabotage detection when attacker-controlled code, comments or execution text can influence both a monitor's judgment and its subsequent evidence acquisition. TAEA places scoring and acquisition behind a shared interface of preregistered static facts and finite categorical probe receipts. It combines route-specific logistic models with interactions and a learned policy that obtains one mandatory source measurement and at most one further source or behavioral measurement. For a frozen deterministic monitor, the paper proves transcript invariance for submissions that agree on all registered measurements under shared trusted randomness, and gives a total-variation stability bound. Experiments on BigCodeBench-Sabotage and APPS report the highest pAUC@10% among the evaluated monitors. Measurement-preserving text edits leave TAEA unchanged; a separate APPS evaluation permits feedback-guided program rewrites and reports 20% evasion among TAEA's initial detections after up to 20 attempts. Acquisition ablations and source-score fusion across five model sizes provide additional evidence.

Critical strengths
  1. A precise and relevant security boundary. Applying the same registered evidence interface to both scoring and action selection addresses an important failure mode of adaptive oversight. The treatment of categorical failures and fresh-state execution makes the assumptions inspectable. Although the proof is straightforward, it usefully specifies what is protected and when the entire inspection transcript must remain unchanged.
  2. Substantial empirical evidence of practical utility. The evaluation includes two benchmarks, multiple monitor families, paired task-bootstrap comparisons, text manipulation, feedback-guided rewrites, and acquisition ablations. The APPS adaptive evaluation is particularly valuable because the attacker may change the measurements: TAEA retains 144/218 detections after attack, compared with 75/218 for Full-Evidence and 73/218 for the injection-augmented classifier (Table 6). The shared-initial-detection comparison in Appendix G.4 further strengthens this evidence beyond comparing conditional ASRs alone.
Critical weaknesses
  1. Data construction and operating-point details leave important validity questions unresolved. Appendix D.1 says BCB sabotage submissions are admitted through the frozen typed-measurement pipeline, without specifying the admission criteria or candidate attrition. In addition, the calibration split and its separation from model/policy selection are unclear. Table 6 reports a 5% calibration-FPR target without the realized benign test FPR; TAEA's clean TPR there is 82.57%, whereas Table 1 gives 80.28% as the maximum attainable TPR at test FPR <= 5%. This needs explanation before fixed-threshold robustness comparisons can be fully assessed.
  2. The experiments do not sufficiently isolate the proposed mechanisms. Table 7 compares useful policy variants, but does not separately ablate interaction features, the continuation gate and the source/behavior switch, or compare scoring-only isolation with isolation of both scoring and acquisition in the same adaptive architecture. On BCB, the pAUC gain over random continuation is only 1.04 points and over the uncertainty band 0.76 points, without uncertainty estimates for these differences. Measurement-preserving text attacks validate the interface invariant rather than general sabotage robustness; the broader result relies on one APPS attack generator with 20 attempts and validity checks on three public tests plus the original backdoor input. The evidence supports a promising system under the evaluated conditions, but leaves the distinct contribution and broader robustness uncertain. 3.The method is introduced abstractly; readers must piece together from the appendices . A compact end-to-end example, along with a table linking each registered field and probe to its possible receipt and downstream decision, would make the method easier to audit and reproduce. 4.Some terminology and examples also need sharper alignment. Table 3 defines capability change in terms of newly introduced capability categories, while Appendix J.1 describes an additional file-write example; the relationship between those descriptions should be clarified. The paper should also make more prominent that the theorem’s interface-level invariance covers a broader formal condition than the narrower text-comment intervention evaluated in Section 4.3. Finally, small labels in multi-panel Figures 4, 5, 7, and 9 make some results harder to inspect at paper scale.
Questions for authors
  1. Data and reference clarification: What exactly admits or rejects a generated BCB submission, how many candidates were excluded, and do the criteria depend on registered-probe differences or detectability?How does TAEA behave on correct independent solutions or substantial benign refactorings?
  2. Calibration clarification: Please specify calibration tasks, sample counts and separation from route fitting, alignment, policy selection and final evaluation; report thresholds and realized benign test FPRs for Tables 2 and 6. How should the APPS clean TPRs of 80.28% in Table 1 and 82.57% in Table 6 be reconciled?
  3. Mechanism validation: Under the same registered features and route-model family, how much remains after removing interaction terms, fixing the second action, or learning only the continuation gate? Can the authors separate scoring-only protection from the shared scoring/acquisition boundary?
  4. Attack-protocol clarification: What precise validity criteria explain the low APPS text-edit coverage (43/218 benign and 144/218 sabotage)?
Reviewer_iM7WOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

This paper proposes an adaptive monitoring framework that checks sabotages in AI-generated code while preventing the monitor itself from being affected by attacker-written text. The approach converts program code to predefined structured observations, progressively choose what to examine next, and infer safety from these structured observations. Experiments on two code-sabotage benchmarks show that the approach achieves highest pAUC compared to baselines.

Critical strengths
  • Preventing the monitoring models from getting affected by attacker-authored text is an important and challenging problem in AI safety research.
  • It is theoretically and empirically validated that as long as registered measurement don't change, the inspection path and the final safety verdict of the proposed approach won't change.
  • Empirically, the sabotage detection performance improves over selected baselines.
Critical weaknesses

Trusted reference solution assumption

In Sec. 3.2, the paper writes "A file-write event, for example, can represent expected task behavior when it also occurs in the trusted reference." L744 in Appendix A also mentions "source comparisons use a trusted reference implementation". Therefore, I consider a trusted reference is necessary to the proposed framework. However, I am confused by such assumption because in real-world coding tasks, we almost never have a reference solution for a coding task - a "solution" is what an coding agent is supposed to produce.

This question can stem from misunderstanding, so please make it more explicit whether the proposed approach indeed requires a reference solution, why the assumption is justified, and whether the compared baselines also employ this assumption.

The reason of improvement over baselines is unclear

The proposed approach aims to enhance robustness to adversarial manipulation, which is justified by the results in Table 2. However, it is not quite clear why the detection FPR also improved significantly in Table 1. I consider it as some "side effect" of the proposed approach, but the improvements need more justification.

Questions for authors

Please confirm whether the approach requires trusted reference solution at both training and inference time, and which of selected baselines use the same assumption.

Reviewer_uycvOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper presents TAEA, a method for detecting sabotage in generated code that does not use an LLM as a monitor. Analysis tools first convert the change in code into code properties and predefined types, Then classifiers are trained to use these observations to assign a suspicion score. Up to two different code properties are used singularly or in combination to assign this score. TAEA achieves better detection at lower false-positive rates than the evaluated baselines.

Critical strengths
  • Using a monitor without an LLM is an interesting approach. It avoids directly exposing an LLM judge to attacker-written code comments and execution output that could manipulate its assessment.
  • Automatically representing code properties and changes as structured, typed observations is a promising idea. It gives the monitor a controlled set of inputs, although these capture only selected properties rather than every change.
  • The adaptive attack evaluation is a strength.
Critical weaknesses
  • Generalization is insufficiently tested. The classifiers are trained and evaluated on tasks from the same benchmark. There is no reported experiment training on BCB and testing on APPS, or testing on a different source of sabotage. This matters because a classifier could learn patterns specific to how the benchmark’s attacks were generated, rather than features that reliably identify sabotage in practice.

  • It is difficult to tell what drives the improvement. The paper should report which first probe, second source probe and behavioral probe were selected for each benchmark. Without this information, it is hard to understand why the detector succeeds or predict where it will fail.

  • The mathematical presentation obscures a relatively simple method. I think the whole paper would have been clearer without the formulas, using plain-language descriptions and concrete examples instead. Table 3 and Algorithm 1 would be much more useful near the beginning. Additionally, the paper repeatedly explains new concepts using terms it has not yet defined. Section 2, for example, introduces an “acquisition policy” that selects “measurements based on the evidence observed so far” without explaining what those measurements or evidence actually are.

  • The practical threat scenario needs a clearer explanation. Does the attacker submit a small modification to previously approved code, or write the entire program? Who supplies the starting code and reference implementation? It is difficult to understand the intended real-world scenario: who the attacker and victim are, what the attacker wants to achieve, and why the victim is being targeted.

  • The trusted reference is a central but insufficiently explained input. Several probes measure changes relative to a trusted implementation, yet the paper does not clearly explain where that implementation comes from or how it is validated. It should also clarify whether benign submissions are compared against themselves or against a separate reference.

  • The automatic measurements need more concrete definitions. For behavioral probes, Table 3 lists outcomes such as stable and divergent, but it is unclear how the system determines whether an output represents a meaningful change in behavior. This can require understanding the task and the output’s meaning. The paper should systematically explain how it turns that knowledge into automatic checks, which kinds of programs these checks support, and where they fail. The paper should systematically explain how it turns that knowledge into automatic checks, which kinds of programs these checks support, and where they fail.

  • Obvious edge cases are not discussed. A new file-writing capability may be suspicious, but a malicious write could be added to a program that already writes files. Conversely, a legitimate implementation could differ substantially from the reference. How well do the probes handle these cases?

  • The theorem provides a narrow guarantee. It establishes that submissions with identical registered measurements produce identical investigations and verdicts. It does not establish that those measurements capture sabotage or that the verdict is correct. It is unclear why the theorem is really meaningful and needed to be added to the main of the paper.

  • The acquisition budget needs a stronger practical justification. Equation 1 constrains measurement cost; it does not require the smallest possible budget. The implementation nevertheless permits only one mandatory probe and one optional probe. Why is this the appropriate restriction, particularly when source-analysis probes may be inexpensive?

  • An important baseline is missing: combine all static probes in one classifier. A natural design would compute all static measurements first, then selectively request an expensive behavioral measurement if needed. The fixed-two baseline does not answer whether this simpler approach would perform as well as TAEA. Actual runtime costs would also be more informative than counting a static comparison and a sandbox execution as one acquisition each.

Questions for authors

See Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.