- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Collaborative Shadows, a backdoor attack on LLM-based multi-agent systems. The attack embeds code fragments in agent tools and releases encrypted fragments through tool outputs into agents’ memories. A hidden assembly routine implanted in the MAS runtime then retrieves, decrypts, assembles, and executes the fragments to perform the attack. The paper also introduces Multi-Role Collaboration Bench and a sandboxed evaluation framework. Experiments report high attack success rates on two controlled benchmarks while broadly preserving benign task performance, with lower attack success rates on more complex tasks.
Critical strengths
The paper brings attention to security risks beyond model parameters. The paper examines how compromised tools and runtime components can introduce backdoor behavior into multi-agent workflows without modifying the underlying LLM. This broadens the security assessment beyond model-level vulnerabilities and motivates scrutiny of the surrounding execution infrastructure.
The paper presents benchmark and sandbox for systematic evaluation. Multi-Role Collaboration Bench combines specialized agent roles with private and shared tools. Together with the sandboxed evaluation framework, it provides a controlled setting for studying attack activation, target selection, and benign task performance.
Critical weaknesses
-
The paper has an overly strong threat model and reliance on a hand‑crafted trigger subtask. The attacker is assumed to have full control over every tool at integration time, which limits the applicability of the attack to scenarios where the entire MAS pipeline can be compromised. Moreover, the global instruction that forces the exact tool‑call sequence is manually added to each benchmark instance. In realistic user interactions such a precise trigger is unlikely to appear. Consequently, the reported ASR may overstate the real‑world risk.
-
The method’s advantage over single-agent backdoors is insufficiently established. DemonAgent already incorporates tool poisoning, encrypted fragments, memory-based collection, and payload reconstruction. Although this paper identifies target selection and collaboration dependence as MAS-specific challenges, it introduces them only briefly, without explaining why they are consequential or why existing methods cannot adequately address them. The paper also does not sufficiently demonstrate how its method resolves these challenges beyond adapting the existing mechanism to multiple agents. This weakens the claimed novelty.
-
The paper lack comparison with existing defenses and insufficient analysis of stealth mechanisms. The only detection experiments are manual/LLM audits and an entropy check, both of which are trivially bypassed by the authors’ own encryption step. State‑of‑the‑art defenses, including tool‑output schema validation, cross‑agent behavior anomaly detectors, or memory integrity signatures, are not evaluated. Without showing that the attack remains effective against these more realistic safeguards, the claim of “high stealth” is unconvincing.
-
The success of the proposed approach heavily depends on flawless task execution. The reported attack success rate collapses from >95 % on the simple benchmarks to 51–77 % on the more complex MultiAgentBench‑DB subset, revealing that the backdoor only activates when every step of the multi‑agent collaboration proceeds without error. Because any failure in tool invocation, memory logging, or inter‑agent communication aborts the primitive collection, the attack’s effectiveness is tightly bound to “smooth” task execution, a condition rarely guaranteed in real‑world deployments. This undermines the authors’ claim that the threat is broadly applicable across LLM‑based MAS.
Questions for authors
- Could you report ASR when the auxiliary subtask is omitted and the agents must decide on tool usage autonomously? This would clarify how dependent the attack is on the handcrafted trigger.
- Could you provide complete auxiliary-subtask prompts and explain how they induce all required tool calls? Does the attacker submit their own request or modify another user’s request?
- Have you tried integrating a simple schema validator that rejects any tool output containing unexpected fields or a cross‑agent sequence monitor that flags unusually long chains of public‑tool calls? If so, how does ASR change?
- Your experiments involve agents that share the same underlying LLM. How does the attack behave when agents use different models or when some agents lack access to the poisoned public tool?
- Does “memory” refer to the agent’s memory module, message history, or a separate execution log? How are encrypted tool outputs preserved, and which record does the assembly routine read?
3 comments
W3 and Q3:
A backdoor is designed to preserve expected behavior during ordinary operation while enabling concealed malicious behavior under attacker-selected activation conditions. In our setting, the malicious deployer knows the implanted mechanism and trigger, while ordinary users do not (§3.1, lines 143-157; §3.5, lines 250-254). The attacker’s knowledge of the implanted mechanism should be distinguished from the information available to ordinary users and defenders. Defenses must examine apparently normal tool outputs and collaborative execution without assuming prior knowledge of the specific implanted backdoor or its activation conditions.
The normal tool-output contract already permits the Cookies string field; the poisoned response retains the expected fields and types while carrying encrypted primitives inside that permitted field (§3.4 lines 229-243; Appendix C, Fig. 14). A validator rejecting unexpected fields therefore has no structural violation to reject. Likewise, the ciphertext enters Agent Memory through the tool’s normal returned observation (Eqs. (6)–(7)). The attack does not depend on modifying previously recorded agent memory or breaking its integrity.
The auxiliary task uses ordinary tool calls, such as querying a table and generating a report, without requiring unusually long public-tool chains. The malicious effect arises when the implanted runtime reconstructs and executes the distributed primitives, a relationship that call-chain length alone does not reveal (§3.5 lines 264-269 and 286-290).
We have evaluated potential defenses for backdoor detection from both semantic and statistical perspectives. Appendix D, Table 5 reports that Llama Guard 4 classifies 100% of the activation requests as safe, while Llama Prompt Guard 2 classifies 100% as benign across all three benchmarks. §4.4, Table 2 evaluates human and GPT-4.1 audits of complete serialized agent memories, including intermediate tool outputs: both detect 100% of plaintext traces and 0% of AES-encrypted traces. Encryption may introduce detectable statistical differences between poisoned and normal tool outputs, which is why §4.4 additionally evaluates entropy. §4.4, lines 423-431 applies Bintropy to 256-byte blocks of complete serialized agent memories. Both average and maximum block entropies remain below the published thresholds, yielding a 0% flag rate.
W4:
As discussed in §5.2, lines 472-475, activation requires the model to follow task instructions and perform the specified tool calls—foundational capabilities of a functioning tool-using agent. A model unable to execute these calls would also struggle with the legitimate tasks for which the MAS is deployed. Instruction following and tool use are also central agentic capabilities in LLMs. The current model’s capabilities are already sufficient to meet the needs of attacks. Our attack exploits these capabilities: agents follow the requested workflow, while the implanted runtime reconstructs and executes the payload once the required primitives are collected** (§3.5, lines 264–269 and 286–290; Appendix A, Algorithm 1)**.
More demanding tasks can hinder the required interactions and leave the primitive collection incomplete. However, the construction does not require every step of multi-agent collaboration to proceed without error and the paper imposes no such requirement. §3.4 (lines 229-238) defines a uniqueness regulator that releases each tool’s primitive at most once during execution, preventing repeated invocations from introducing duplicate copies of that primitive. Tool retries, therefore, do not automatically invalidate activation. Moreover, attacks are essentially related to the invocation of tools, not the smoothness of tasks. The paper does not impose “smooth” or error-free completion of the entire task as an activation condition.
To examine how attack execution and benign task completion coexist, we additionally report their joint success rates. Joint success requires both objectives to succeed in the same execution. Joint success is evaluated at the task level.
| Model | AgentBench-MAS (DB) | Multi-Role Collaboration Bench | MultiAgentBench | |||
|---|---|---|---|---|---|---|
| ASR | Joint | ASR | Joint | ASR | Joint | |
| Qwen3-30B-A3B | 99 | 94 | 95 | 92 | 77.14 | 48.57 |
| GLM-4.5-Air | 99 | 93 | 98 | 93 | 71.41 | 54.29 |
| Kimi-K2-Instruct | 98 | 69 | 99 | 87 | 55.71 | 47.14 |
| Gemini-2.5-Pro | 99 | 93 | 100 | 88 | 51.43 | 48.57 |
| GPT-4.1 | 100 | 95 | 97 | 95 | 55.71 | 52.86 |
These results demonstrate that the backdoor can activate while the original task is successfully completed. ASR also exceeds joint success for every backbone, showing that attack activation can occur even when the original task is not fully completed.
Q4:
Using the same backbone across role-specialized agents is an established MAS evaluation configuration. CAMEL evaluates cooperation between two GPT-3.5-turbo agents, while ChatDev uses ChatGPT-3.5 across its software-development roles. MultiAgentBench likewise reports its database collaboration results separately for each backbone. The official smolagents multi-agent or Codex likewise assigns the same model to the manager and its managed agent. Using a common backbone makes model comparisons easier to interpret, while mixing models can confound the effects of different model capabilities. However, we still provide a simple mixed-model evaluation as a supplementary result on AgentBench-MAS (DB), using GPT-4.1 as the manager, with Qwen3-30B-A3B-Instruct-2507, GLM-4.5-Air, and GPT-4.1 assigned to Agents 1–3, respectively. Under the attack condition, ACC is 95% and ASR is 100%; Under the non-attack condition, ACC is 95% and ASR is 0%. These results demonstrate successful activation with heterogeneous backbones and the same task accuracy across the two evaluated conditions.
Regarding tool access, §3.2 (lines 169-175) explicitly defines the public tool pool as shared by all sub-agents. Both benign and poisoned public tools are exposed through this same pool. Because the backdoor is covertly implanted, ordinary users do not know which tools are poisoned. Our setting does not introduce a separate access policy based on this hidden poisoning status: poisoned public tools remain accessible through the same interface and permissions as benign public tools. Selectively withholding them from particular sub-agents would change the shared-access configuration explicitly defined in the paper.
The construction does not require every participating agent to invoke the payload-bearing tool: the designated target must invoke it, while other agents may supply conditional primitives through their available private or shared tools (§3.4, lines 206-211). The shared tool pool, primitive placement, and distributed collection procedure are already explicitly specified in §3.1–§3.5.
Q5:
“Memory” refers to each agent’s own execution memory (M_i) (§3.2, lines 171-175). Our MAS uses smolagents (§4.1), whose normal execution loop records tool calls and observations in Agent Memory, thereby preserving the result returned by poisoned tools (§3.4, lines 229-243, Eqs. (6)–(7)). The encrypted fragments are therefore stored directly in Agent Memory as part of the tool observations. After task execution and result aggregation, the implanted runtime routine scans these agent memories, extracts and decrypts the marked fragments, and validates and assembles the primitive collection before execution (§3.5, lines 264–269 and 286–290; Appendix A, Algorithm 1).
Thank the reviewer's feedback. We address the weaknesses and questions below.
W1, Q1 and Q2:
§3.1, lines 139-150 explicitly defines our application-supply threat model: a malicious deployer supplies a deployed MAS application to enterprises or operates a public service. The construction requires poisoning selected tools, not compromising every tool, and embedding an assembly routine during integration. After deployment, these components remain fixed. The attacker knows the implanted mechanism and submits their own activation request through the normal user interface, without modifying another user’s request or acquiring additional runtime privileges.
The objection that a precise trigger is unlikely to arise in ordinary user interactions conflates accidental activation with deliberate backdoor activation. Activation is elicited by following the tool-use requirements of the attacker-submitted task. The attacker can explicitly assign ordinary tasks to designated agents and request the relevant tool calls. In the DB example below, the selected agent queries its managed table and passes the result to generate_report. This supplies the target-specific data, while the required collaborative tool interactions supply the remaining primitives (§3.3, lines 189–194; §3.4, lines 206-211; §3.5, lines 250-263). And our high ASR result demonstrates its feasibility.
Appendix D.1–D.3 describes the auxiliary-task. The AgentBench-MAS (DB) prompt template is provided below. Complete examples will be added in the final version.
The no-trigger comparison requested in Q1 is already reported in the “w/o modified” ablation (§4.4, Fig. 3). With poisoned tools retained and the auxiliary activation task removed, ASR is 0% for all five models on AgentBench-MAS, and 4% for Qwen3-30B-A3B and 0% for the other four models on Multi-Role Collaboration Bench. §5.1, Fig. 4, and Appendix F also provide complementary trigger probability analysis
W2:
The §1, lines 42-55 identify two MAS-specific challenges: selecting among heterogeneous target agents and conditioning activation on their collaborative execution. Target selection matters because different roles have access to different tools and data. DemonAgent’s original single-agent formulation does not represent this request-directed choice among multiple data-holding agents. Our construction supports it within the same poisoned MAS: the shared payload-bearing tool incorporates the designated role’s output (§3.3, lines 189-194 and §3.4, lines 206-211), allowing the attacker to change whose data enter the payload without replacing the implanted components. Collaborative activation conditions the attack on primitives produced through different agents’ tool interactions; the runtime reconstructs and executes the payload only after collecting and validating the required set (§3.5 lines 264-269 and 286-290). §4.2 (lines 338-341) evaluates fixed-seed selection among target agents, and §4.3 (lines 357-362) verifies the target-specific outcomes.
§4.4 (lines 414-418) relates encryption-based concealment to DemonAgent. §5.1, lines 446-450, Fig. 4, and Appendix F provide an analytical comparison with a single-tool activation baseline: under the default configuration and stated neutral prior, its 30.0% accidental activation probability is reduced by approximately 35× through fragmentation