acceptodds
Under review as a conference paper at ICLR 2027

Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents

Abstract

To keep coding agents from going off the rails, production systems now review each proposed action with a blocking monitor that can reject it before it runs (Auto Mode in Claude Code, Guardian in OpenAI's Codex). Prior evaluations of such monitors largely measure robustness to accidental harm or prompt injections from untrusted sources looking to hijack the agent. Less understood is how they hold up when the agent they monitor is persistently misaligned. To understand this risk, we task an adversarial agent with evading production blocking monitors and causing catastrophic harm, e.g. by exfiltrating its own weights. We find that when instructed with high-level attack strategies, adversarial agents can succeed through several distinct mechanisms, such as agent-generated prompt injection against the monitor, multi-agent attacks, and malicious compaction. In particular we find that in 79% of trials, the agent can use an injection attack against Auto Mode and Guardian to run arbitrary bash commands. We also find that it is possible to greatly improve Auto Mode through design changes like enhancements to tool coverage, changes in transcript formatting, and the addition of an agentic monitor stage. Even with these improvements, preventing multi-context attacks at an acceptable cost remains an open problem. By detailing our red-teaming methodology and highlighting new attack vectors, we aim to help defenders evaluate their mitigations against the possibility of persistent malign coding agents.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Control, agent monitoring, prompt injection against monitors, multi-context attacks, red-teaming, compaction attacks, multi-agent attacks, LLM agent safety

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_u3EBOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Popular coding agents now have autonomous modes where another model decides if the main model’s actions are safe or unsafe where the goal is to reduce the number of permissions requests that the human user needs to process and avoid the pitfalls of dangerously skip permissions mode. This paper aims to red-team these classifier models. To this end, the authors study various attack strategies and identify under what conditions they succeed. Finally, the authors discuss some possible mitigation that can improve the autonomous mode.

Critical strengths
  • The topic is very timely. Assessing the safety of autonomous mode is critical as this mode is essential to make agents effective and reduce cognitive burden on humans.
  • The authors attack coverage is broad including prompt injection into the monitoring classifiers, multi-agent attacks, coverage gaps and unintended mistakes. The coding agents craft these attacks adaptively and are much more effective than static attacks.
  • Evals are also well thought out, the authors investigate loop holes in the auto mode setup to begin with and then pivot to on-policy and off-policy adaptive attacks. The findings seem important for example that compaction summaries are generally not monitored and are an attack surface.
  • The enhancements to autonomous mode explored seem sensible such as monitoring all edits and agentic classifiers that can interact with the environment.
Critical weaknesses
  • The paper touches upon fixes the issues found from red-teaming but does not give deeper treatment to it. The discussed solutions are sensible but monitoring all edits or using agentic monitors can make the whole system slow and expensive. A discussion on how to better operationalize the S1->S2->agent cascade will be useful to make them practical.
Questions for authors

Could we see similar effectiveness from automode ++ and automode strict if we use much smaller model but fine-tuned specifically for guardian tasks?

More broadly, how does model choices and harness choice affect the lessons from this work?

Reviewer_kfSvOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the robustness of production blocking monitors against persistently misaligned coding agents. Unlike prior studies that primarily focus on accidental harm or prompt injection from external sources, the authors consider adversarial agents that actively attempt to evade monitoring and perform harmful actions. The paper systematically examines four attack categories: monitor injection, multi-context attacks, coverage gaps, and policy failures. Through analytical inspection, replay-based evaluation, and adaptive red-team rollouts, the authors identify significant vulnerabilities in Claude Code's Auto Mode and an approximation of Codex's Guardian. The paper further proposes several defensive improvements, including structured transcript framing, expanded tool monitoring, compaction monitoring, and agentic verification.

Critical strengths
  • Important and timely research problem. The paper addresses a highly relevant security challenge as coding agents become increasingly autonomous and gain access to sensitive resources and privileged operations.

  • Strong practical significance and originality. A particularly valuable contribution is the systematic investigation of actual production-oriented monitoring mechanisms.

  • Comprehensive and well-designed evaluation. The paper combines analytical evaluation, replay-based testing, and adaptive red-team rollouts in both sandboxed environments and simulations.

Critical weaknesses
  • The generalizability of the proposed defensive improvements could be further investigated.

Auto Mode ++ integrates several improvements, including transcript framing, broader tool coverage, compaction monitoring, and an additional agentic verification stage. While the paper provides useful individual analyses and comparisons, the overall framework is evaluated primarily against the attack categories and vulnerabilities identified during the authors' red-teaming process. It remains unclear how effectively these defenses generalize to previously unseen attack strategies or different agent workflows. A more systematic component-level ablation and evaluation on held-out attacks would help establish the independent contributions of the defensive mechanisms and their broader robustness. This would further strengthen the paper's practical recommendations.

  • The impact of attacker affordances on the interpretation of attack success rates deserves further discussion.

In several evaluation settings, the adversarial agent is provided with detailed information about the monitoring mechanism and high-level guidance regarding the attack strategy. Although this is reasonable within the AI Control threat model and Appendix B provides a useful analysis of these assumptions, the reported attack success rates may depend substantially on the information available to the attacker. In particular, Figure 12 demonstrates that knowledge of the transcript format can significantly increase attack effectiveness. It would be helpful to more clearly distinguish the ability to discover vulnerabilities independently from the ability to exploit known weaknesses under informed adversarial conditions. Such clarification would improve the interpretation of the reported attack success rates and their applicability to realistic deployment scenarios.

  • Some aspects of the evaluation methodology and practical deployment costs require further clarification.

The paper combines results from sandbox-based arena evaluations and LLM-driven simulations, which offer different levels of environmental fidelity. As acknowledged in Appendix F, simulated tool outputs and simplified compaction mechanisms may influence attack behavior and success rates. Therefore, the aggregated attack success rates should be interpreted with these differences in mind. Additionally, the evaluated Guardian implementation approximates the production monitor rather than reproducing it exactly. The efficiency evaluation in Figure 8 is also primarily based on per-action monitoring latency and false-positive rates, rather than end-to-end coding agent performance. Further validation of the simulation results and a more comprehensive assessment of monitoring overhead in complete coding workflows would strengthen the practical conclusions.

Questions for authors
  1. How effectively does Auto Mode ++ defend against previously unseen attack strategies that were not considered during the development of its defensive components?

  2. How do the reported attack success rates change when adversarial agents are not provided with explicit attack strategies or detailed knowledge of the monitoring mechanism?

  3. Could the authors provide additional evidence on the end-to-end latency and task-completion overhead introduced by Auto Mode ++ in realistic coding agent workflows?

Reviewer_2h7zOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper red-teams two production blocking monitors for coding agents: Claude Code’s Auto Mode and an approximation of Codex’s Guardian. An Opus 5 agent receives a benign main task and, in its system prompt, a hidden side task such as leaking credentials, a model file, or source code. By default it is told the monitor’s exact transcript format and given a high-level attack hint. The paper studies four attack types: monitor injection through forged transcript turns, multi-context attacks (multi-agent and malicious compaction), coverage gaps such as unmonitored edits, and policy failures. The evaluation combines Docker environments scored by deterministic checkers, an LLM-simulated environment, and replay of 4,000 benign SWE-chat actions to measure false positives. Production monitors let 59% to 72% of attacks through. A hardened variant, Auto Mode ++, lowers this to 9%. It adds escaped and framed transcripts, edit monitoring, a compaction check, tool results in context, and an agentic investigator stage. The authors conclude that most holes have cheap fixes, while multi-context attacks remain costly to defend.

Critical strengths
  1. Real vulnerabilities in widely deployed monitors, with cheap fixes. The paper finds and patches concrete holes in Auto Mode: forged transcript turns, execute-then-write through unmonitored edits, unmonitored compaction summaries, and policy gaps such as an unmonitored Inspect loop. None of these depends on the misaligned-agent premise. A prompt-injected agent could exploit them too. This is the main reason the paper is useful to practitioners, whatever one thinks of the threat model.

  2. Transparent elicitation. The P × M affordance ladder (Figure 12) and the hidden vs. shown comparison (Figure 17) show how strongly attack success depends on what the attacker is told. This makes the results interpretable, and it is the evidence I rely on below. Treating the compaction summary as an attack surface is also new.

Critical weaknesses
  1. The practical relevance of the threat model is not sufficiently established.

The paper explicitly adopts an AI Control setting, but I am not convinced that the specific attacker assumptions are sufficiently connected to realistic deployment scenarios.

  • In the multi-agent evaluation, every agent is given the same hidden malicious goal in its system prompt. This assumes shared malicious intent rather than showing how it arises or spreads. The paper should explain which threat source this represents and why this assumption is appropriate.
  • The system-prompt side task evaluates monitor robustness once malicious intent is already present. It does not establish how likely such behaviour is under normal deployment safeguards. The attacker also receives monitor details and attack hints. Figures 12 and 17 show that monitor-injection success depends strongly on these affordances.
  • The cited incidents motivate concern about harmful autonomous behaviour, but they do not directly validate the paper’s specific combination of persistent hidden goals, shared intent across agents, and a trusted monitor. Section 5 acknowledges the trusted-monitor assumption, but its applicability when the attacker and monitor come from the same model family remains unclear.

These experiments identify useful vulnerabilities, but the reported success rates should not be interpreted as estimates of ordinary deployment risk. My concern is that the paper does not sufficiently establish how broadly its findings apply beyond the assumed adversarial setting.

  1. The contribution of individual defences and the scope of the robustness claim remain unclear.

Auto Mode ++ bundles several changes. The paper includes useful comparisons of transcript formatting, edit monitoring, monitor stages, and model choice, but it does not isolate the contribution of each change within Auto Mode ++. I therefore cannot tell which changes account for most of the drop from 65% to 9%, or which are necessary.

The paper also includes adaptive injection optimisation and live attacker rollouts. However, it remains unclear how extensively the full Auto Mode ++ system was red-teamed for new strategies after the fixes. A clearer account of this process and its attack-search budget would help establish how robust the improvement is.

Finally, the injection results need clearer reporting. The abstract reports 79%, Figure 5a reports 78.4% and 85.2%, and Figure 3 reports arena rates of 94% and 95%, alongside lower simulation rates. Figure 12 reports 33% for P2/M4 on one environment, while the headline arena results combine three environments. These numbers need not conflict, but the paper should specify the configurations, sample sizes, and aggregation rules that connect them.

Questions for authors
  1. Under the AI Control setting, which threat source do the attacker assumptions represent: emergent misalignment, a backdoored model, or a prompt-injected agent? Please justify the shared malicious goal across agents and the monitor knowledge and attack hints provided.

  2. What was the exact system-prompt wording for the side task? Was the attacker a standard model or a variant with modified safety settings? How often did it refuse the side task, and how were refusals counted in the reported success rates?

  3. Beyond the reported adaptive injection search and live rollouts, was the complete Auto Mode ++ system red-teamed for new attack strategies after the fixes? What was the search budget?

  4. Could you add a leave-one-out ablation to isolate the contributions of the Auto Mode ++ changes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.