acceptodds
Under review as a conference paper at ICLR 2027

More Agents, Same Exposure: Measuring the Safety of Tool-Partitioned Agent Teams

Abstract

An LLM agent acts: it moves money, edits records, or restarts services. An external adversarial input against it therefore ends in a wrong and often irreversible action, rather than a wrong answer. Separation of privilege suggests a remedy: partition the tools across a team of agents so that an attack must either reach an agent that holds the write tool directly or travel there through another agent's output. We test both paths of this principle with Safemas, a measurement framework instantiated in four simulated domains with 240 tools and 990 scenarios that compare a single agent against four multi-agent architectures on three models. We find that the principle does not hold for orchestrated teams: partitioning does not protect the agent that can act and the channels it creates add new exposure. Attacks that also apply to a single agent succeed as often on orchestrated teams (39-40% against 41%), and with attacks on the inter-agent channels included, 53% of attacks on the orchestrated teams succeed. The attack vector matters more than the structure: a tampered inter-agent message succeeds in 76% of runs, more than any injected instruction or poisoned tool return, on all three models. The partition also impacts task completion. Orchestrated designs complete 42-49% of the task, against 90% for the single agent, while making 16-60x as many model calls; sharing reads across all agents recovers most of that gap. We release the code, the environments and the run traces at https://anonymous.4open.science/r/safemas-6946/.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
multi-agent LLM security, agentic framework, tool partitioning, separation of privilege

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_drdeOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper challenges the assumption that applying classical "least privilege" (partitioning tools across specialized sub-agents) improves security in multi-agent LLM systems. Using SAFEMAS—an empirical benchmark spanning 4 domains (Banking, DevOps, Healthcare, Workspace), 240 tools, 24 multi-step tasks, 5 team topologies, and 3 LLM backbones—the authors evaluate 990 scenarios to show that tool partitioning merely relocates the attack surface. Under identical prompt injection attacks, orchestrated teams fail at rates similar to a single agent (39%–40% vs. 41%), but total attack success climbs to 53% once inter-agent channels are targeted. Inter-agent message tampering proves most lethal (76% success rate overall, up to 85% on GPT-5.6 Luna) because orchestrator delegation strips away original prompt provenance, causing sub-agents to trust tampered messages implicitly. Furthermore, partitioning severely degrades utility, dropping task completion from 90% (single agent) to 42%–49% while increasing LLM calls by 16× to 60×.

Critical strengths

Challenging Core Architectural Dogma: Demonstrates that rigid tool partitioning creates vulnerable communication channels rather than reducing exposure.

Deterministic Evaluation Grounding: Evaluates outcomes via post-run database state diffs rather than subjective LLM-as-a-judge scoring.

Comprehensive Vector Taxonomy: Disentangles instruction injection, indirect tool poisoning, and in-transit message tampering across multiple agent roles.

Shared-Read Counterfactual Control: Proves that the performance tax stems from data isolation rather than team coordination.

Critical weaknesses

Single-Run Sampling (N=1): Point estimates without variance bounds due to high multi-agent API costs. Suggestion: Run Monte Carlo repetitions on a 10% sub-sample to establish confidence intervals. (BUT, I understand that the entire array of experiments is vast, so that would be prohibitive, unless the authors choose a critical subset of the overall array in order to formally establish significance).

Unweighted Output Severity: Evaluates all unauthorized writes equally regardless of real-world impact. Suggestion: Categorize state changes by risk level (e.g., log edit vs. financial transfer).

Static Adversary Strategy: Uses fixed single-payload injections. Suggestion: Test adaptive, multi-turn prompt optimizations.

Questions for authors

How would message authentication protocols (e.g., cryptographic signatures) affect the success rate of the message tampering vector?

Did any orchestrator prompt-engineering patterns successfully preserve origin provenance during instruction delegation?

Did the expanded read scope in the shared-read counterfactual introduce new indirect tool-poisoning pathways?

Would you consider using the results of your large experimental array as an initial; exploration, and from there design a smaller, but more formal experiment with enough repetitions to establish 10% confidence intervals?

2 comments

How would message authentication protocols affect the success rate of the message tampering vector?

Authenticating messages and protecting their integrity, for example with a MAC, removes this vector. We include it because prior work treats the channel as an attack surface (arXiv 2502.14847, 2511.05269, 2606.30602), which lets us compare it with the other two vectors. Two caveats: agent-to-agent authentication needs a key infrastructure that is not yet standardized (draft-klrc-aiagent-auth-02), and it does not stop an instruction injected at the orchestrator, which the orchestrator then sends itself in an authentic message. This is now in Section 6.

Did any orchestrator prompt-engineering patterns successfully preserve origin provenance during instruction delegation?

We did not vary the orchestrator's prompt. Rerunning the scenarios with other prompts would be informative, but reaching the same statistical confidence would need a comparable number of runs for each prompt, which we could not afford for this revision.

Did the expanded read scope in the shared-read counterfactual introduce new indirect tool-poisoning pathways?

No. With shared reads a poisoned tool return succeeds in 17% of runs against 24% with partitioned reads, and no position becomes easier to attack across the three models (Section 5.3, Table 7).

Would you consider a smaller, more formal experiment with enough repetitions to establish 10% confidence intervals?

We now run every scenario three times and report 95% intervals throughout (Section 5, Table 4). The headline comparisons are within ±7 percentage points. Single cells of Table 4 are wider (±7 to ±15 percentage points for attack success), because their width is set by the number of distinct attacks (42), which further repetitions do not change.

Unweighted output severity.

Every successful attack counts the same in the paper, and we now state this as a limitation (Section 6, limitation 4). If the reviewer finds it necessary, we can compute a severity-weighted analysis from the released traces. We did not plan severity levels before running the experiments, so the result would be post hoc and its conclusion weaker, which is why it is not in the current draft.

Static adversary strategy.

Our attacker places one fixed payload at one position, so every rate is a floor for an adaptive attacker (Section 6, limitation 1).

Dear Reviewer drde,

Thank you again for your review. As the discussion period closes today, we wanted to check whether our response and the revised manuscript address your concerns, in particular the repeated runs with confidence intervals and the discussion of message authentication. If any point remains unclear, we would be glad to clarify it before the discussion closes.

Thank you for your time and consideration.

Reviewer_RykyOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SAFEMAS to compare a single tool-using agent with four tool-partitioned team architectures under external prompt injection. It instantiates four simulated domains with 240 tools, 24 tasks, and 42 attacks, expanded into 990 scenarios evaluated on three models. The study distinguishes payload delivery, attacker-directed state changes, required-write completion, and coordination cost. It reports similar pooled attack rates at positions shared by the single agent and orchestrated teams, higher rates when orchestrator positions are included, and substantial benign utility losses that largely recover when reads are shared. The work highlights shared-state propagation and trust in orchestrator messages as relevant attack paths.

Critical strengths

S1. A useful framework for examining concrete state changes across architectures.

The benchmark uses deterministic checks of tool execution and environment outcomes rather than an LLM judge, and reuses underlying tasks and payloads across configurations. Separating delivery, attack outcomes, legitimate writes, and model-call costs makes the evaluation more informative than refusal rates alone. The examples of a relay modifying a record that another agent subsequently consumes expose an important interaction between local tool permissions and shared state.

S2. The evaluation includes meaningful capability controls and detailed results.

The shared-read intervention preserves write ownership and the coordination protocol while substantially improving benign utility; centralized teams recover 33–55 percentage points across models. This supports a concrete explanation for much of the measured capability loss. The per-position results and separate model results are also useful, and the high susceptibility of the tested orchestrator-message positions warrants attention even if the broader architectural conclusions require qualification.

Critical weaknesses

W1. The reported ASR does not distinguish state corruption from completed downstream harm.

Section 4 defines attack success as the attacker's value landing in environment state. The released grader follows this broad criterion, allowing unified success from a matching sink call, a state-level plant, or a final-state harm predicate. Record corruption is a meaningful compromise. However, Section 3.4 describes indirect cascades as a relay planting a value that the target later reads and acts upon, which is a stronger outcome than a successful intermediate plant. This distinction matters to conclusions about protecting the agent capable of the final harmful action.

The paper provides an OSF trace archive, and the code records state-level delivery and harm separately. Q1 — Clarification and analysis of existing outcomes: Please identify the scoring field behind Tables 4 and 6 and separately summarize plant, downstream-harm, and sink-execution rates for indirect attacks by architecture and model. How do the cascade and architectural conclusions change under a downstream-harm criterion? I have not recomputed the archive, so the impact on rankings remains unresolved.

W2. The shared-read intervention establishes utility recovery without the corresponding safety comparison.

Section 5.3, Figure 3, and Table 5 show substantial recovery of benign utility when reads are shared, while preserving write ownership and the coordination protocol. This is a strong capability control. Its attack-success results are not reported. Because independent teams already share reads while the communicating architectures initially partition them, the safety–utility comparison after this capability recovery is important to the broader architectural conclusion. The manuscript also acknowledges that role prompts and ownership information vary across architectures.

Q2 — Additional validation relevant to the architectural claim: Please provide a focused matched-position attack comparison for the existing shared-read configurations, reporting the separated outcomes in W1 together with legitimate-write utility, or delimit the safety conclusion to the configurations as originally evaluated.

The conclusion should also remain explicit about the tested opportunities. The final message-tampering positions are orchestrator-to-relay and orchestrator-to-target. Source comments report an earlier peer-channel campaign with approximately 45% non-traversal and 0/29 harm among tampered runs; a brief explanation of its exclusion would clarify final coverage. The paper already acknowledges single-run point estimates, different position mixtures, and unweighted damage. These are scope qualifications: the reported pooled rates should remain descriptive rather than establishing statistical equivalence or a fixed-distribution increase in risk.

Overall assessment. I lean toward weak rejection because the conclusions about downstream cascades and architectural safety–utility trade-offs need the outcome separation in W1 and the control comparison in W2. The framework and observed vulnerabilities are useful contributions. These are borderline evidence-weighting concerns; channel-coverage and statistical qualifications are secondary, and the available code does not determine whether the rankings will change.

Questions for authors

See weaknesses

5 comments

Please identify the scoring field behind Tables 4 and 6 and separately summarize plant, downstream-harm, and sink-execution rates for indirect attacks. How do the conclusions change under a downstream-harm criterion?

Tables 4 and 6 use the attack_succeeded field of each trace. We renamed the fields of the two steps so that the code, the traces and the paper use the same words: relay_write is the corrupted record (the reviewer's plant) and target_write is the attacker's value in the target's records (downstream harm). Section 4 now states that an indirect attack succeeds on the corrupted record, and the second step is reported separately as cascade success C = Pr(value at target | attack succeeds), per architecture and model in Table 4 and per position in Table 6.

We chose the corrupted record because the second step depends on task progress and not on the attack: in 62% of runs with a corrupted record, the target agent never reads it afterwards. The corruption is also persistent: it is checked on the post-run state, so in all 2,392 such runs the attacker's value is still in the record for any later read.

Indirect attacks, three models (single / centralized / hybrid / decentralized / independent):

OutcomeRate
Record corrupted56 / 64 / 60 / 47 / 21%
Attacker's value in the target's records13 / 10 / 11 / 6 / 1%
Target's write tool executed with it13 / 7 / 8 / 6 / 1%

Under the downstream criterion, the order on the attacks every architecture faces is unchanged (27 / 30 / 32 / 29 / 11%). Over all attacks, the difference between orchestrated teams and the single agent narrows from 13 percentage points (+7 to +20) to 6 percentage points (−1 to +14), which Section 5.1 now reports. There is therefore no evidence that orchestrated teams are safer than the single agent under either criterion.

Please provide a matched-position attack comparison for the shared-read configurations, with the separated outcomes and legitimate-write utility.

This is the new Table 7 (Appendix A), discussed in Section 5.3: the same 602 scenarios with partitioned and with shared reads, reporting utility under attack, direct and indirect attack success, and cascade success. With shared reads, utility under attack rises from 42–57% to 82–84%, direct attacks fall from 60% to 45%, indirect attacks from 61% to 55%, and cascade success from 17% to 9%.

Scope points raised in the review (earlier peer-channel campaign, descriptive rates, confounds, unweighted damage).

  • Peer channel. The earlier campaign mentioned in the source comments was a pilot in which the sender was not fixed, so the tampered message was often never sent. The subagent-message positions are now part of the benchmark with a fixed sender (Sections 3.4 and 5.2).
  • Descriptive rates. We agree. The text says "about as often" and not "as often", every rate in Table 4 carries its interval, and we do not claim statistical equivalence.
  • Role prompts and ownership information differ across architectures. This is not controlled and is stated as a limitation (Section 6, limitation 2).
  • Unweighted damage. Stated as a limitation (Section 6, limitation 4).
Reviewer_RykyOpenReview ↗

Reply to the author’s response

Thank you to the authors for their detailed response and revised manuscript, which have largely addressed my original concerns.

I have one further question: during shared read operations, the success rates of direct and indirect attacks against decentralised teams are 31.9 per cent and 36.9 per cent respectively (Table 7), which are lower than the average values for single agents of approximately 40 per cent and 56 per cent (Table 4). Please report the paired differences and confidence intervals for the attack positions common to both scenarios, or clarify in Sections 1 and 7 that the conclusion that only independent teams are attacked less frequently applies solely to the partitioned read configuration.

Once this point has been clarified, I am inclined to raise the rating to ‘weak acceptance’.

AuthorsOpenReview ↗

Reply to the reviewer's response

Thank you for the follow-up. You are right. We report the comparison below and have restated the claim in Sections 1 and 7.

Paired difference. Decentralized teams with shared reads and the single agent face the same 42 attacks, delivered through an injected instruction or a poisoned tool return. The difference below is computed on those attacks. Its 95% interval comes from the same cluster bootstrap as the rest of the paper.

AttacksDecentralized, shared readsSingle agentDifference (percentage points)
All34.3%48.1%−13.8 (−20.6 to −6.8)
Direct31.9%40.2%−8.3 (−17.6 to +1.1)
Indirect36.9%56.2%−19.4 (−28.5 to −10.5)
DeepSeek-V4-Flash33.8%46.5%−12.7 (−21.4 to −3.8)
GPT-5.6 Luna28.6%51.5%−22.9 (−32.6 to −13.4)
MiniMax-M2.740.5%46.3%−5.7 (−13.3 to +1.8)

So decentralized teams with shared reads are attacked less often than the single agent. The decrease is clear, with the whole interval negative, for all attacks together, for indirect attacks, and on DeepSeek-V4-Flash and GPT-5.6 Luna. For direct attacks and on MiniMax-M2.7 the trend is the same, although the upper end of the interval is slightly positive (+1.1 and +1.8). With partitioned reads the same teams are at parity with the single agent, +0.8 (−5.1 to +6.8). Under attack they complete 81.7% of the task with shared reads, against 88.1% for the single agent.

For completeness, the centralized and hybrid teams with shared reads are also lower than the single agent on these attacks, by 6 and 7 percentage points (both intervals −12 to −1). They still face the attacks on the orchestrator, which the single agent does not, so over all their attacks they remain above it (55% against 48%).

Changes in the manuscript.

  • Sections 1 and 7 no longer single out the independent teams. They now say that the only teams clearly attacked less often than the single agent are teams whose agents share their reads.
  • Section 5.3 reports that, with shared reads, injected instructions and poisoned tool returns succeed in 34% of runs on the decentralized teams, against 48% on the single agent, which faces the same attacks.

This is consistent with our conclusion: the teams that are attacked less are the ones that partition less.

Thank you again. While you and the other reviewers look at these changes, we will do a last pass on the manuscript for polishing.

Reviewer_XURpOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Distributing tools across multiple agents is widely assumed to contain prompt-injection damage, yet malicious instructions can still reach an authorized agent through inter-agent messages or shared records. This paper presents SAFEMAS, a benchmark spanning five agent architectures and four simulated domains. Attack success, task completion, and coordination cost are measured against the final environment state. Results indicate that orchestrated teams provide little protection against shared-channel attacks, while additionally introducing exposed communication channels and substantial task-completion overhead.

Critical strengths

1.Well-motivated security question. The work asks whether distributing tool access actually contains external attacks, rather than testing whether a single agent refuses a malicious user request. 2.Outcome-based measurement. Attack success and task completion are verified against the final environment state, removing dependence on an LLM judge and capturing harmful actions that co-occur with successful task execution. 3.Informative failure analysis. The experiments separate direct attacks from cascades through shared records, and the shared-read control clarifies why partitioned teams underperform on task completion.

Critical weaknesses

1.Asymmetric attack coverage. The paper claims to attack every agent and channel, yet Table 6 reports message tampering only for orchestrator-to-agent messages. Peer-to-peer channels exist in decentralized and hybrid teams, but their tampering results are omitted. This gap undermines the claim that non-orchestrated teams are safer under matched exposure. 2.Insufficient isolation of the partitioning effect. Architectures differ in role prompts, tool-ownership knowledge, and read access; independent teams are the only configuration sharing all reads in the main comparison. The shared-read control is evaluated on benign tasks only, so it remains unclear whether its utility gains preserve security or merely trade reduced coordination for wider exposure. 3.Overbroad conclusion on privilege separation. Each sensitive write still requires a single agent to act, and shared records lack integrity checks. Showing that an attacker can reach that agent does not establish whether partitioning bounds the damage achievable from a fixed initial foothold; that containment benefit is never directly measured. 4.Weak statistical support. Each scenario is run once, and the grid reuses only 24 tasks and 42 attacks. Without repeated runs and uncertainty estimates accounting for these shared tasks and payloads, rates such as 39–40% versus 41% do not support claims of equivalence across architectures.

Questions for authors

1.Could the authors add peer-message tampering and re-rank architectures under matched attack positions and budgets? 2.How does attack success change under the shared-read control, and do the conclusions hold when tool-ownership information and role prompts are controlled? 3.Restricting the attacker to the same initial foothold, does partitioning reduce the range of unauthorized actions or records that can be affected? 4.Could the authors repeat representative scenarios and report paired confidence intervals accounting for shared tasks and attack payloads?

2 comments

Could the authors add peer-message tampering and re-rank architectures under matched attack positions and budgets?

We added it. On the hybrid and decentralized teams, a message from another subagent is now tampered at two positions, to the target agent (direct) and to the relay agent (indirect): 120 new scenarios, 1,080 runs. The sender is fixed in advance as a third subagent whose own work stream needs a value only the attacked agent can read, so the partition forces it to write to that agent (Section 3.4).

  • The request is sent in 86% of runs on decentralized teams and 71% on hybrid teams, and the attack succeeds in 47% of runs at the target agent and 42% at the relay agent (Section 5.2, Table 6).
  • A tampered subagent message succeeds in 45% of runs, against 82% for a tampered orchestrator message, 70% for an injected instruction and 21% for a poisoned tool return (Figure 2b).
  • With and without these positions, attack success is 59% and 64% on hybrid teams and 49% in both cases on decentralized teams. The average falls on hybrid teams because the new positions are weaker than the orchestrator's, but they are two more places where an attack succeeds in 42–47% of runs. This supports our conclusion: each channel a team adds is additional attack surface.
  • Matched positions. Not every position exists in every architecture: the orchestrator's positions exist only on centralized and hybrid teams, and the subagent message only on hybrid and decentralized teams. We therefore rank the architectures on the positions they all share: single agent 48%, centralized 53%, hybrid 53%, decentralized 49%, independent 21% (Section 5.1).
  • Matched budgets. The attacker's budget is the same everywhere, one payload at one position. We did not match the model-call budget: a team spends many more calls than a single agent by design, and capping them would lower attack success on teams because they complete less of the task, not because they are safer.

How does attack success change under the shared-read control, and do the conclusions hold when tool-ownership information and role prompts are controlled?

We reran the 602 attacked scenarios of the centralized, hybrid and decentralized teams with shared reads, three times on each model (Section 5.3, Table 7). Attacks succeed less often in every architecture: direct attacks fall from 60% to 45% and indirect attacks from 61% to 55%, while utility under attack rises from 42–57% to 82–84%. Role prompts and ownership information are not controlled: each architecture keeps its own, and we now state this as a limitation (Section 6, limitation 2).

Restricting the attacker to the same initial foothold, does partitioning reduce the range of unauthorized actions or records that can be affected?

We do not measure this directly, and we have narrowed the claim to match. Each attack has a single goal, so we measure whether a payload reaches the agent able to carry it out, not how far an attacker can reach from one foothold. Structurally, the partition does bound direct reach: writes are split into disjoint groups, so a compromised subagent can directly invoke only its own group. Anything beyond that must travel through a shared record or a message, which is the path our indirect attacks measure. Measuring reach from a fixed foothold needs attacks with open-ended goals, which we list as a limitation (Section 6, limitation 4).

Could the authors repeat representative scenarios and report paired confidence intervals accounting for shared tasks and attack payloads?

Every scenario is now run three times on each model (16,056 runs). All intervals are 95% cluster-bootstrap intervals that resample attacks (or tasks when there is no attack) and keep the three runs of a scenario together (Section 5, Table 4). Orchestrated teams are attacked more often than the single agent by 13 percentage points (+7 to +20) over all attacks. On the attacks both face, the difference is 5 percentage points (−3 to +12), which is why we say "about as often".

Dear Reviewer XURp,

Thank you again for your review. As the discussion period closes today, we wanted to check whether our response and the revised manuscript address your concerns, in particular the tampered messages between subagents and the attack results with shared reads, which we added at your request. If any point remains unclear, we would be glad to clarify it before the discussion closes.

Thank you for your time and consideration.

Comments on the paper

3 comments

We thank the reviewers for their comments, which helped us clarify and improve the manuscript. We addressed as many of them as we could, and the changes are in the revised manuscript.

The code and the run traces are updated as well. Code, environments and scenarios: https://anonymous.4open.science/r/safemas-6946/. Traces of all 16,056 runs: https://osf.io/xfrzh/overview?view_only=d3ef65c997204c1b97340591be471019.

The main changes are:

  • More runs. The first version had 2,970 runs; the revision has 16,056. Every scenario is now run three times on each model, and we provide confidence intervals as requested by the reviewers. DeepSeek-V4-Flash and MiniMax-M2.7 are served locally from their public checkpoints.
  • A shared-read control. We reran the centralized, hybrid and decentralized teams with every subagent able to read all the data, with and without attacks (6,066 runs; Section 5.3, Table 7).
  • Tampered messages between subagents, on the hybrid and decentralized teams (1,080 runs; Sections 3.4 and 5.2).
  • A cascade metric C, reported next to attack success (Section 4, Tables 4 and 6). We also aligned the field names in the code and traces with the terminology used in the paper.

The main conclusion is unchanged, but two results moved. Decentralized teams, attacked less often than the single agent in the first version, are now at parity (49% against 48%). On DeepSeek-V4-Flash, orchestrated teams are no longer attacked less often than the single agent.

AuthorsOpenReview ↗

Follow-up on our response and revised paper

Dear Reviewers,

Thank you again for the reviews. We have posted a response to each review and uploaded a revised version of the paper.

As the discussion period closes soon, we would be grateful to hear whether our response addresses your concerns, and we are happy to clarify any remaining point in the time left.

Best regards, The authors

Dear Reviewers,

Thank you for your efforts so far. As we are at the end of the discussion period, I would like to kindly encourage you to review the authors' rebuttals and engage in further discussions, particularly if the authors have addressed your questions or concerns.

Thank you for your time and contributions!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.