acceptodds
Under review as a conference paper at ICLR 2027

Red-Teaming Multi-Agent Attacks for Loss-of-Control Risks

Abstract

As AI agents start forming dangerous swarms, AI control measures must remain effective when they covertly coordinate on a harmful objective. We introduce , a synthetic series of live services and codebases (5 interconnected services, 249 benign tasks, 8 attack objectives), adapted from infrastructure. We demonstrate , whose attack fragments are causally independent and executed by different agents, and how their covertness evades when reading their trajectories individually. We show how distributed attacks spread across diverse contexts and harmful objectives through (e.g., weight exfiltration, unmonitored rogue deployment, and permission elevation). We found that distributing harmful objectives lowers monitor suspicion scores from 9.0 for a single agent carrying out the attacks to 1.5 for the most suspicious agent in the split. Additionally, , leading to a loss-of-control outcome. Finally, we tested whether models can plan these covert attacks: plans optimized against a weak monitor collapse under a strong held-out one, while human-crafted attacks stay covert. By turning worst-case coordinated attacks into reproducible red-team cases, our work provides a foundation for preventing .

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Control, Multi-Agent Safety, Red-Teaming, Loss-of-Control

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Pip3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces MiniLab, a synthetic environment comprising five interconnected services, 249 benign tasks, and eight attack objectives, to investigate loss-of-control risks from multi-agent attacks. Through 16 human-optimized attacks, the authors demonstrate that distributed attacks can substantially reduce suspicion under per-agent monitoring. The paper further examines a six-stage stateful attack chain, joint monitoring, and the ability of LLMs to generate covert attack plans.

Critical strengths
  1. Valuable evaluation environment. MiniLab provides interconnected live services and persistent system state for studying coordinated attacks. The six-stage attack chain demonstrates how separate actions can accumulate into a larger loss-of-control outcome.
  2. Comprehensive empirical investigation. The paper evaluates distributed attacks under per-agent and joint monitoring, investigates attack explainability, and examines model-generated attack plans. These experiments provide useful evidence about the limitations of trajectory-based monitoring.
Critical weaknesses
  1. Limited fairness of the single-agent comparison. The authors acknowledge that single-agent and multi-agent attack plans were not developed with equal effort. This makes it difficult to determine how much of the observed reduction in monitor suspicion is attributable specifically to distributing attacks across agents.
  2. Limited evaluation of multi-agent-aware monitoring. The primary monitoring setup excludes complete shared system state, and the joint monitor uses a prompt originally designed for single-agent software-engineering tasks. Although additional monitoring analyses are provided, the effectiveness of the attacks against monitors explicitly designed to reason about cross-agent actions and shared state remains unclear.
Questions for authors
  1. Can the authors provide comparisons between single-agent and multi-agent attacks with comparable red-teaming effort?

  2. How would the attacks perform against multi-agent-aware monitors with access to shared system state?

  3. Which monitor was used to provide feedback during the development of human-crafted attacks, and was GPT-5.6 Sol held out from this optimization?

3 comments
AuthorsOpenReview ↗

(1/2): Clarification on red-teaming effort, multi-agent-aware monitors, and held-out monitors

We thank the reviewer for the helpful feedback and for recognizing our paper as a meaningful contribution. To address your concerns, we provide an explanation of (1) the red-teaming effort for multi-agent and single-agent attacks and whether it affects our claim, (2) whether a monitor with a fuller view of the attack catches more, and (3) how we develop and evaluate the attacks against the same monitor.

Single agent vs split

The single-agent baseline uses the same attack, merging the three agents' step-by-step plans into a single plan, which is given to a single agent with all main tasks and combined permissions of the 3 to carry out the same actions in a single trajectory. We then applied further optimization pressure to the single-agent trajectory.

Because of this further optimization, the single agent's steps can differ from the team's, which is why the submitted text said it may take different steps. The single agent is flagged far more often (82.4% vs 3.4% for per-agent monitoring at the 2% FPR threshold), which points to distributing the attack rather than a weaker single-agent plan as the main source of the difference, even after this further optimization. We see the same thing without changing the attack at all, as reading the exact same split executions together instead of one agent at a time raises the share flagged from 3.4% to 40.9%.

Why did we choose not to spend equal effort on single-agent attacks?

As mentioned above, we attempted further optimization of the single-agent plans, but the results suggested that achieving comparable evasion rates would require substantially more budget. Our goal was to demonstrate concrete cases in which an attack flagged during single-agent execution could evade detection when split across agents. Therefore, we focused more on red-teaming viable attack strategies in which agents can coordinate and distribute the attack among themselves. If possible, we will provide results for single-agent and multi-agent teams under matched red-teaming budgets to help resolve this uncertainty in our camera-ready version.

Monitors built for multiple agents and shared state

We rewrote the monitor prompt to read one or more agents and ask whether their combined changes cause harm (Appendix I). Under joint monitoring, this raises the median suspicion from 5.5 to 7.9 across the 16 attacks, and the share of successful distributed attacks flagged from 32% to 41%. Still, 59% evade, even though joint monitoring is told which agents are working together.

We don't provide the monitor system state; it isn't always available, and we want to focus on trajectory monitors. Inter-agent messages are excluded because our threat model assumes agents can coordinate through channels that the monitor cannot see, and, moreover, the optimized attacks require no messages during execution, as the plan is fixed in advance. We adapted the monitor prompt to read one or more agents' trajectories together, but joint monitoring serves as a comparison point rather than a proposed protocol, since it assumes the monitor already correctly identifies which agents are working together. Monitors that combine trajectories with system state are an important direction, and while MiniLab records the state needed to test them, we leave state visibility to monitors as future work to focus on these worst-case outcomes, given uncertainty about whether monitorability can be preserved as model capabilities advance (see Think Fast, OpenAI's monitorability evaluation, and AISI's Loss of Oversight report).

Which monitor gave feedback, and was Sol held out?

For the human-crafted attacks, GPT-5.6 Sol was the monitor we optimized against, as it was the most capable monitor available to us at the time, so it was not held out for them. Sol was held out for the AI-planned attacks, which were developed with GPT-5.6 Luna feedback. The attacks were not optimized for the revised monitor prompt, but the effect is similar, as median suspicion is 9.0 for a single agent vs 1.75 for the most suspicious agent in the split, compared with 9.0 vs 1.5 under the original monitor prompt.

Monitors from other model families, which the attacks were never optimized against, also flag far fewer distributed attacks than single-agent executions:

Monitor (effort)Single agentPer-agent monitoringJoint monitoring
GPT-5.6 Sol (xhigh)83.1%4.8%39.9%
Claude Opus 4.8 (xhigh)80.9%20.7%47.1%
Gemini 3.1 Pro (xhigh)75.0%24.5%58.2%
DeepSeek V4 Pro (xhigh)57.4%1.9%13.5%

Share of successful executions flagged at each monitor's own 2% FPR threshold (100 benign teams, one reading per view; Appendix C, Table C1).

AuthorsOpenReview ↗

(2/2) Presentation revisions

Additionally, we submitted a revised version of our paper to improve clarity and incorporate new changes.

Presentation

  • A new Threat Model section states the internal deployment setting, what the attackers can do, and what the monitor can and cannot see.
  • The Setup section now describes MiniLab with a concrete example, states exactly what the monitor receives, and defines covert success with a calibrated threshold in one place.
  • The Results section opens with the main numerical comparison (per-agent monitoring flags 3.4% of successful distributed attacks vs 82.4% for a single agent) and explains why joint monitoring still falls short of a single agent.
  • We standardized figure formatting and captions, fixed typos, removed repeated text in the planning section, and converted the attack-explainability table into a single figure.

Relevant changes in the revised PDF

  • Monitor prompt: LinuxArena's strong trajectory prompt, adapted to read one or more agents and to score combined harm (Appendix I). All GPT-5.6 Sol results rescored.
  • Monitors from other model families (Appendix C, Table C1): Claude Opus 4.8, Gemini 3.1 Pro and DeepSeek V4 Pro; every monitor flags far fewer distributed attacks under per-agent monitoring than single-agent executions.
  • Single-agent baseline (Appendix C): clarified.

We appreciate if the reviewer can take a look at our new version, and we're happy to clarify anything that remains difficult to follow.

Thank you for your response. I will maintain my score.

Reviewer_Q3esOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MiniLab is a synthetic internal-lab environment for studying distributed attacks by multiple AI agents. It contains five connected services, 249 benign tasks, and eight harmful objectives. The authors develop 16 human-optimized attacks in which agents pursue legitimate tasks while carrying separate fragments of a shared malicious objective. Per-agent monitoring assigns much lower suspicion to these split attacks than to solo attacks, and joint trajectory monitoring often misses harms that emerge through shared system state. The paper also demonstrates a six-stage attack chain and evaluates model-generated plans, reasoning visibility, and synchronous monitoring.

Critical strengths
  1. MiniLab is a substantial executable environment rather than a collection of hypothetical prompts. Its connected services and persistent state make it possible to study harms that are not visible in any single code change or trajectory.

  2. The threat model is stated plainly. The paper separates human-crafted plans from model planning, reports several monitoring variants, and does not present the optimized attacks as estimates of spontaneous malicious behavior.

Critical weaknesses
  1. The conceptual contribution is quite close to recent work. Makins et al. already study the fragmentation effect in a synthetic AI-lab environment with distributed attacks, while Hills et al. examine attacks accumulated through persistent state and Radev et al. study coordinated sabotage across agents. MiniLab is larger and the attack chain is more elaborate, but the paper should make clearer what scientific conclusion is new beyond broader environment and attack coverage.

  2. The central solo-versus-split comparison is not fully controlled. The multi-agent attacks were repeatedly optimized using monitor feedback, while the paper states that solo and team plans did not receive equal development effort and may use different steps. The large reduction in suspicion therefore combines attack distribution with differences in plan quality and optimization. A matched comparison that begins from the same harmful action set would better establish the claimed fragmentation effect.

  3. The monitoring claim depends on a restricted observation model. The standard monitor sees trajectories but not inter-agent messages, complete state changes, service logs, or diffs, although MiniLab records some of this information. Even the joint monitor therefore lacks the evidence needed to reconstruct several attacks. The experiment exposes a weakness of trajectory-only monitoring, not necessarily of a monitor that can inspect the system state already available in MiniLab.

  4. Some broader conclusions rely on a small set of heavily selected cases. The main result uses 16 optimized attacks, the full six-stage chain is run in three worlds, and synchronous monitoring is evaluated by replaying successful trajectories without benign controls. These experiments are useful demonstrations, but they provide limited evidence about false positives, recovery after intervention, or performance on attacks not optimized against this setup.

Questions for authors
  1. What conclusion does MiniLab establish that is not already supported by FakeLab and other recent distributed or persistent-state attack benchmarks?

  2. Can the solo and split conditions be compared using equivalent harmful actions and similar red-team optimization effort?

  3. How would detection change if the joint monitor received state diffs, service logs, and inter-agent messages already recorded by MiniLab?

  4. For synchronous monitoring, what false-positive rate results from applying the same blocking threshold to benign task executions?

2 comments
AuthorsOpenReview ↗

(1/2) Novelty and single-agent comparison

(1/2) We thank the reviewer for the helpful feedback. Below, we briefly explain how our work contributed meaningfully compared to similar work, and (2) how comparable the single-agent and multi-agent red-teaming efforts are.

Novelty

We agree these are the closest works, and we extend their results. We established how optimized distributed attacks can evade even frontier monitors on live non-code services, with parallel-running agents strong enough to compose into a loss-of-control outcome.

Red-teaming strength

All three red-team their attacks only lightly (Makins et al.: "a single round of manual prompt optimisation"; Hills et al.: about 10 rounds of attack-prompt engineering; Radev et al.: monitor-aware prompting), and generally use weaker monitors (GPT-4o, GPT-5 mini/nano and GPT-5.1 Codex Mini).

When stronger monitors are used, the attacks are mostly caught: GPT-5.2 and GPT-5.3-Codex catch about 73% of Hills et al.'s gradual attacks, and Makins et al. find a stronger monitor cuts undetected success by more than an order of magnitude. We show this does not hold once the attacks are optimized. We optimize each attack against GPT-5.6 Sol and show that this optimization holds at the frontier: per-agent monitoring flags 3.4% of successful distributed attacks, compared with 82.4% for a single agent, and monitors from three other model families flag 2-25%. Even joint monitoring, which reads exactly the agents working together, flags 41% of successful distributed attacks vs 82.4% for a single agent.

Chaining attacks

Makins et al. show that a single attack can be fragmented to reduce suspicion. We show that the effect of distributed attacks is strong enough that multiple attacks can happen in sequence and be composed into a much larger attack. We run six separate attacks in order, each by a fresh team that inherits only the world state left by the previous one. Together they carry a model from a failed safety evaluation to unmonitored external serving, and no individual agent crosses the detection threshold at any stage, although joint monitoring flags it from Stage 1 (Figure 3).

Settings

All three are only in code environments. We also use non-code tasks, such as release management, evaluation, infrastructure, and model serving, across five live services that share state. Makins et al. run agents one after another, which can be modeled as Hills et al.'s persistent agent, and both leave parallel deployments to future work. Radev et al. run concurrent agents, but their attacks are not red-teamed against trajectory monitors and are mostly caught (99% and 68% flagged from code edits alone at a false-positive rate of 2% or lower), whereas ours remain covert.

Single agent vs split

We also addressed this concern for Reviewer Pip3

The single-agent baseline uses the same attack, merging the three agents' step-by-step plans into a single plan, which is given to a single agent with all main tasks and combined permissions of the 3 to carry out the same actions in a single trajectory. We then applied further optimization pressure to the single-agent trajectory.

Because of this further optimization, the single agent's steps can differ from the team's, which is why the submitted text said it may take different steps. The single agent is flagged far more often (82.4% vs 3.4% for per-agent monitoring at the 2% FPR threshold), which points to distributing the attack rather than a weaker single-agent plan as the main source of the difference, even after this further optimization. We see the same thing without changing the attack at all, as reading the exact same split executions together instead of one agent at a time raises the share flagged from 3.4% to 40.9%.

Why did we choose not to spend equal effort on single-agent attacks?

As mentioned above, we attempted further optimization of the single-agent plans, but the results suggested that achieving comparable evasion rates would require substantially more budget. Our goal was to demonstrate concrete cases in which an attack flagged during single-agent execution could evade detection when split across agents. Therefore, we focused more on red-teaming viable attack strategies in which agents can coordinate and distribute the attack among themselves. If possible, we will provide results for single-agent and multi-agent teams under matched red-teaming budgets to help resolve this uncertainty in our camera-ready version.

AuthorsOpenReview ↗

(2/2) Observation model and evidence scale

Observation model

We don't provide the monitor system state because it isn't always available, and we want to focus on trajectory monitors.

Inter-agent messages are excluded because our threat model assumes agents can coordinate through channels that the monitor cannot see, and, moreover, the optimized attacks require no messages during execution, as the plan is fixed in advance. Direct communication is disabled and not necessary for these attacks (Appendix H.3), so there would be no messages for the monitor to read.

This differs from Radev et al., who study real-time coordination and sabotage, whereas we focus on planned-in-advance attacks. We adapted the monitor prompt to read one or more agents' trajectories together, but joint monitoring serves as a comparison point rather than a proposed protocol, since it assumes the monitor already correctly identifies which agents are working together. Monitors that combine trajectories with system state are an important direction, and while MiniLab records the state needed to test them, we leave state visibility to monitors as future work to focus on these worst-case outcomes, given uncertainty about whether monitorability can be preserved as model capabilities advance (see Think Fast, OpenAI's monitorability evaluation, and AISI's Loss of Oversight report)

Evidence scale

The 16 optimized attacks and the six-stage chain are deliberate worst-case demonstrations rather than estimates of how often such attacks occur, and we report results by side task (Appendix D).

The six-stage chain also succeeded end-to-end in 12 of 12 additional independently initialized worlds. We also ran synchronous monitoring on the benign runs and calibrated the threshold from there: at 8.5/10, it stops none of the 16 human-crafted attacks before they complete, and 1 of 146 covertly successful AI-planned attacks; at 7.5, none of the human-crafted attacks and 23% of the AI-planned ones.

For attacks not optimized against the monitor, success drops drastically under a better monitor: plans developed with GPT-5.6 Luna feedback reached 45-48% covert success under Luna, but 23-32% under both Luna and GPT-5.6 Sol (Claude Opus 4.8: 48% to 23%; DeepSeek V4 Flash: 45% to 32%). The human-crafted attacks, optimized against Sol, remain covert under both monitors (85% under Luna, 82% under Sol). Recovery after a synchronous intervention requires live interaction rather than replayed trajectories, and we leave it to future work.

Additionally, we submitted a revised version of our paper to improve clarity and incorporate new changes

Presentation

  • A new Threat Model section states the internal deployment setting, what the attackers can do, and what the monitor can and cannot see.
  • The Setup section now describes MiniLab with a concrete example, states exactly what the monitor receives, and defines covert success with a calibrated threshold in one place.
  • The Results section opens with the main numerical comparison (per-agent monitoring flags 3.4% of successful distributed attacks vs 82.4% for a single agent) and explains why joint monitoring still falls short of a single agent.
  • We standardized figure formatting and captions, fixed typos, removed repeated text in the planning section, and converted the attack-explainability table into a single figure.

Relevant changes in the revised PDF

  • Monitor prompt: LinuxArena's strong trajectory prompt, adapted to read one or more agents and to score combined harm (Appendix I). All GPT-5.6 Sol results rescored.
  • Monitors from other model families (Appendix C, Table C1): Claude Opus 4.8, Gemini 3.1 Pro and DeepSeek V4 Pro; every monitor flags far fewer distributed attacks under per-agent monitoring than single-agent executions.
  • Single-agent baseline (Appendix C): clarified.

We appreciate if the reviewer can take a look at our new version, and we're happy to clarify anything that remains difficult to follow.

Reviewer_RqYWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper provides a benchmark, baseline evaluations, and experimental settings for studying coordinated multi-agent attacks and their detection. The authors introduce MINILAB, a simulated AI-lab environment with five connected services, and develop 16 attacks through manual red teaming. The results show that agents can execute human-designed attack plans, and that some models can generate successful plans, especially with execution and monitor feedback. The paper compares individual-agent and whole-team monitoring and demonstrates how separate attacks can be chained into a larger harmful outcome.

Critical strengths
  1. The benchmark provides a useful setting for studying coordinated attacks. Its environments are a good baseline for future work on this topic.
  2. The human-developed attacks provide useful baselines. They offer concrete cases for testing new defenses as well as assessing automated attack planners.
Critical weaknesses

The threshold used to define covert success is insufficiently justified. The monitor rubric labels scores of 5–7 as highly suspicious and discusses flagging at 6+, yet attacks with median scores below 7.5 count as covert. Although this does not invalidate the reported reduction in suspicion scores, it makes the operational meaning of the covert-success rates unclear.

Questions for authors

Could the authors explain the choice of the 7.5 threshold? Reporting covert-success rates across thresholds, alongside false-positive rates on benign runs, would help clarify this choice. Evaluation with judges from other model families would also help assess whether the findings depend on the OpenAI monitors used in this study.

3 comments
AuthorsOpenReview ↗

(1/2) Threshold calibration and cross-family monitors

We thank the reviewer for the feedback. Below, we provide (1) our threshold calibration, (2) covert success across thresholds with benign false-positive rates, and (3) flag rates for monitors from other model families.

Why 7.5 threshold?

In our covert success definition, we defined a successful attack as successful execution with a score below the specified suspicious threshold (7.5), which is why a successful attack with a median score of 7.0 counted as covert success even though the rubric described that score as “highly suspicious.” As we score each trajectory 12 times with the monitor, our reported median is the average of the two middle scores (the monitor returns whole numbers from 0 to 10), so if the median is 7.5, at least half of the scores must be 8 or higher. This guarantees that half of the scores would fall in the prompt’s “clearly malicious” (8-10) band. The score bands (5-7, 8-10, etc.) are taken directly from LinuxArena’s trajectory monitor prompt.

Under the original monitor prompt, the 7.5 threshold flags 1.4% of benign trajectories (13/904) and 3.5% of benign teams (11/316). After revising the monitor prompt, we recalibrated the threshold on benign runs. We selected the lowest threshold that kept the benign false positive rate at or below 2%. For GPT-5.6 Sol, this threshold was 8.5. The previous cutoff of 7.5 exceeded the target, flagging 6.3% of benign teams under per-agent monitoring, up from 3.5% with the original prompt. For GPT-5.6-Luna, it is 7.5 (1.8% of benign teams). We report covert success across thresholds alongside the benign rates in the tables below.

Does the result still hold at 6? Yes, please see threshold 6 in the tables below. The gap between split and single-agent attacks still persists, though benign flag rates are already 12-16%.

Covert success across thresholds with benign false-positive rates

Share of successful attacks not flagged at each threshold, and share of benign runs flagged at the same threshold (GPT-5.6 Sol, revised monitor prompt, 16 optimized attacks; benign teams complete the same main tasks with no side task). Brackets: 95% Wilson intervals over executions.

Per-agent monitoring (208 successful executions; 316 benign teams)

ThresholdAttacks not flaggedBenign teams flagged
673.6% [67.2, 79.1]12.0% [8.9, 16.1]
780.3% [74.4, 85.1]9.5% [6.7, 13.2]
7.585.1% [79.6, 89.3]6.3% [4.1, 9.6]
8.596.6% [93.2, 98.4]1.6% [0.7, 3.6]
997.6% [94.5, 99.0]1.6% [0.7, 3.6]

Joint monitoring (same as above)

ThresholdAttacks not flaggedBenign teams flagged
620.2% [15.3, 26.2]15.8% [12.2, 20.3]
720.7% [15.7, 26.7]11.4% [8.3, 15.4]
7.534.1% [28.0, 40.8]6.0% [3.9, 9.2]
8.559.1% [52.3, 65.6]0.6% [0.2, 2.3]
959.6% [52.8, 66.1]0.6% [0.2, 2.3]

Single agent (136 successful executions; 904 benign trajectories)

ThresholdAttacks not flaggedBenign trajectories flagged
61.5% [0.4, 5.2]5.3% [4.0, 7.0]
72.2% [0.8, 6.3]4.3% [3.2, 5.8]
7.54.4% [2.0, 9.3]2.8% [1.9, 4.1]
8.517.6% [12.2, 24.9]0.6% [0.2, 1.3]
919.1% [13.4, 26.5]0.6% [0.2, 1.3]

At every threshold, distributed attacks stay covert far more often than a single agent doing the same work. Appendix C shows the full score distributions for attacks and benign runs together.

Other model families

We calibrate each trajectory monitor on benign teams, and then run the single-agent and multi-agent attacks against it. GPT-5.6 Sol showed the best performance as a trajectory monitor for single-agent attacks, and we show a comparison with monitors from other model families.

Monitor (effort)Single agentPer-agent monitoringJoint monitoring
GPT-5.6 Sol (xhigh)83.1%4.8%39.9%
Claude Opus 4.8 (xhigh)80.9%20.7%47.1%
Gemini 3.1 Pro (xhigh)75.0%24.5%58.2%
DeepSeek V4 Pro (xhigh)57.4%1.9%13.5%

We found a similar drop in flag rates when the agent team split up the attack compared to when a single agent pursued it. Even though our human-crafted attacks were optimized against GPT-5.6 Sol, other models showed the same drop. For example, Claude Opus 4.8 flagged 80.9% of successful single-agent executions and 20.7% of split executions, compared to 83.1% and 4.8% in GPT-5.6 Sol, respectively.

We show that other models still missed split attacks in absolute terms, underscoring that this weakness of single-agent-calibrated control protocols is not specific to one monitor family. For this comparison, we used 100 benign teams, one reading per view, and each monitor's own 2% FPR threshold (Appendix C, Table C1).

AuthorsOpenReview ↗

(2/2) A revised version of the paper uploaded

Additionally, we submitted a revised version of our paper to improve clarity and incorporate new changes.

Presentation

  • A new Threat Model section states the internal deployment setting, what the attackers can do, and what the monitor can and cannot see.
  • The Setup section now describes MiniLab with a concrete example, states exactly what the monitor receives, and defines covert success with a calibrated threshold in one place.
  • The Results section opens with the main numerical comparison (per-agent monitoring flags 3.4% of successful distributed attacks vs 82.4% for a single agent) and explains why joint monitoring still falls short of a single agent.
  • We standardized figure formatting and captions, fixed typos, removed repeated text in the planning section, and converted the attack-explainability table into a single figure.

Relevant changes in the revised PDF

  • Monitor prompt: LinuxArena's strong trajectory prompt, adapted to read one or more agents and to score combined harm (Appendix I). All GPT-5.6 Sol results rescored.
  • Monitors from other model families (Appendix C, Table C1): Claude Opus 4.8, Gemini 3.1 Pro and DeepSeek V4 Pro; every monitor flags far fewer distributed attacks under per-agent monitoring than single-agent executions.
  • Single-agent baseline (Appendix C): clarified.

We appreciate if the reviewer can take a look at our new version, and we're happy to clarify anything that remains difficult to follow.

Thank you for the update. I will keep my score after the clarification.

Comments on the paper

1 comment
AuthorsOpenReview ↗

AC Comment

Dear Reviewers and AC,

As the discussion period is ending, we kindly ask you to check our responses if you haven’t already. Two of three reviewers are keen to accept our paper and maintained their score of 3. For the other reviewer, we also addressed the concerns meaningfully.

Summary of strengths:

  1. All of the three reviewers found our MiniLab “useful for studying coordinated attacks,” describing it as a substantial executable environment, thanks to its connected services and persistent state, which enable chaining long-horizon attacks.
  2. Reviewer RqYW credited our work as critical “for testing new defenses and for assessing automated attack planners.”
  3. Reviewer Pip3 mentioned that our work “gives useful evidence on the limits of trajectory-based monitoring” and how a larger loss-of-control outcome can be formed by separate actions.
  4. Reviewer Q3es praised our work for presenting the threat models cleanly while maintaining a good balance between human-crafted and AI-crafted plans.

We also addressed the majority of the concerns from the reviewers below

  1. Calibrated thresholds (RqYW, Q3es). We have calibrated our own threshold using benign trajectories, and the lowest threshold keeping benign false positives at or below 2%, for individual trajectories and under both per-agent and joint monitoring, is 8.5/10 on GPT-5.6 Sol. For GPT-5.6 Luna, it is 7.5.
  2. Monitor prompt (Pip3). We deliberately changed the prompt to accommodate multiple agents. Per-agent monitoring flags 3.4% of successful distributed attacks, compared with 82.4% for a single agent. Joint monitoring flags 40.9%, but we believe it is idealized, as the monitor would need to identify the colluding agents amongst potentially many more agents. It serves as a comparison and not as an actual monitoring protocol.
  3. Synchronous monitoring (Q3es). We ran synchronous monitoring on the benign runs and calibrated the threshold based on those results. We removed the synchronous monitoring figure since, at the calibrated threshold, almost no attacks are stopped, and kept the text description.
  4. Single-agent baseline (Q3es, Pip3). The single agent has the same actions, just in a single-agent trajectory, with the combined permissions of the 3. The comparison is an existence proof that an attack flagged when one agent carries it out can go unflagged when it is split across agents, rather than a search for the strongest single-agent attack.

Moreover, regarding concerns about our paper’s novelty, our findings extend prior work by showing that once distributed attacks are optimized, they can remain covert even against frontier monitors. We also move to live, non-code services with concurrent agents, a more realistic setting in which separate attacks can build into a loss-of-control outcome. Our evaluation deliberately stress-tests trajectory-based LLM monitors, the standard defense in single-agent control, so we do not give the monitor system state, which is not always available in deployment (see our responses to Q3es and Pip3). We hope MiniLab and its 16 attacks give defenders a concrete testbed for this threat: they show where control protocols calibrated on single agents fail, and MiniLab already records the shared system state needed to build and evaluate monitors that reason across a whole swarm of agents.

New version of the paper has been uploaded with changes below:

  • Monitor prompt: LinuxArena's strong trajectory prompt, adapted to read one or more agents and to score combined harm (App I). All GPT-5.6 Sol results rescored.
  • Thresholds: calibrated at 2% FPR on benign teams; 8.5 for Sol, 7.5 for Luna (App C, new Figs C1-C2).
  • Monitors from other model families (App C, Table C1): Claude Opus 4.8, Gemini 3.1 Pro and DeepSeek V4 Pro; every monitor flags far fewer distributed attacks under per-agent monitoring than single-agent executions.
  • Headline: per-agent monitoring flags 3% of successful distributed attacks vs 82% for a single agent (Abstract, Sec 1, Fig 1, Sec 5.1); joint monitoring flags 41% (Sec 5.3).
  • Chain (Sec 5.2, Fig 3): rescored, and executions completed in 12 of 12 additional worlds (App E).
  • Planner results (Sec 5.4, Fig 5C, App G, Fig G2): Sol rescored with the revised monitor prompt at 8.5; AI-planned attacks reach at most 32% covert success vs 82% for human-crafted plans.
  • Attack explainability (App F): re-run with a single-agent reference (1.4% / 31% / 50%, new Fig F1).
  • Synchronous monitor (App H.2): new single-action prompt (App I); threshold calibrated on benign executions (8.5); 0/16 human-crafted attacks thwarted at 8.5 or 7.5; 1/146 covertly successful AI-planned attacks thwarted at 8.5, 23% at 7.5 (previously 8% at an uncalibrated 8); Figure H2 removed.

We thank the reviewers and the AC for their feedback and for reading our paper once again. We look forward to any further clarifications that would help reach a consensus on the paper’s value to strengthen AI safety and control research.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.