acceptodds
Under review as a conference paper at ICLR 2027

TrojanAgent: A Malicious Agent Bypassing Guardrails to Execute Dangerous Actions in Multi-Agent Systems

Abstract

Dynamic orchestration enables LLM-based agents to select and compose other agents at runtime for complex tasks, making agent specifications a security-critical attack surface. A malicious agent that enters the workflow can receive subtasks and potentially invoke tools to execute dangerous actions. However, existing poisoning attacks primarily target passive capabilities such as tools or skills. In this work, we introduce TrojanAgent, a novel attack that targets the external agent itself, which becomes an active participant in the workflow once selected. TrojanAgent first manipulates the agent selection process, where an embedding model retrieves candidate agents and an LLM-based orchestrator further selects agents from these candidates. Once the malicious agent is selected into the generated workflow, it executes malicious operations while bypassing the target system’s agent guardrails at runtime. Our experiments demonstrate the effectiveness of TrojanAgent in both agent selection and runtime guardrail bypass using a corpus of 1,547 real-world agent specifications collected from public platforms. At the agent selection stage, TrojanAgent achieves an average 23% higher Agent Hit Rate (AHR) than existing manual-based and automated poisoning baselines. At runtime, TrojanAgent achieves an average 33% higher Bypass Success Rate (BSR) than the baselines, highlighting the need for defense strategies tailored to dynamic agent orchestration.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Generative Orchestration, Multi-Agent Systems, Agent Security, LLM Agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_RnYbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TrojanAgent combines task-aware agent-specification augmentation with AMR-based instruction fusion to influence agent selection and bypass runtime guardrails. Evaluation covers 50 tasks, 1,547 specifications, six retrievers, three target models, and two guardrails, with strong selection results and substantial TaskShield bypass gains.

Critical strengths

S1: The attack considers both workflow selection and post-delegation behavior, addressing two relevant security boundaries in dynamic orchestration. S2: Evaluation spans several retriever families and measures tool execution under two guardrails. The TaskShield results show a substantial component-level improvement over SCH.

Critical weaknesses

W1: Separate selection and runtime metrics do not establish the joint attack outcome. BU requires only one delegated subtask to succeed, which is weaker than full benign-task preservation (Section 3.1; Equation 4; Section 5.1). W2: The AMR-to-text prompt also performs explicit text obfuscation. The graph-builder ablation does not clearly specify which transformations remain, limiting attribution of gains to AMR fusion (Section 5.3; Appendix A.1). W3: The deviation, intent-preservation, modification-cost, and task-preservation scores are insufficiently operationalized. Selection of the final Pareto candidate is also unclear, limiting reproducibility of the optimization (Sections 4.2–4.3).

Questions for authors

Q1: What is the joint rate of selection, guardrail passage, malicious execution, and full benign-task completion on the same trials? Q2: Does the no-graph ablation retain the same text-obfuscation instructions and generation budget? Q3: How are the optimization scores computed, and what rule selects the final candidate from the Pareto frontier?

3 comments
AuthorsOpenReview ↗

Thank you, reviewer RnYb! (1/3)

Response for Weaknesses and Questions

Weakness 1: “Separate selection and runtime metrics do not establish the joint attack outcome. BU requires only one delegated subtask to succeed, which is weaker than full completion of the original task (Section 3.1; Equation 4; Section 5.1).”

Question 1: What is the joint rate of selection, guardrail passage, malicious execution, and full benign-task completion on the same trials?

We thank the reviewer for raising this important point. Following the reviewer’s suggestion, we add two complementary metrics: the end-to-end attack success rate (ASRe2eASR_{e2e}) and full legitimate-task utility (BUfullBU_{full}). Specifically, ASRe2eASR_{e2e} measures the joint occurrence of malicious agent selection, guardrail bypass, and malicious execution, while BUfullBU_{full} measures whether the original benign task is fully completed. We report these two outcomes independently, following prior evaluations such as AgentDojo [1], Rethinking Indirect Prompt Injection as a Test-Time Search Problem [2], and Will the User Ever Know? [3], which also assess attack success and utility under attack separately. We evaluate 50 tasks per execution model, two guardrails, and three repetitions. The results are shown in Table 1.

As shown in Table 1, TrojanAgent achieves the highest ASRe2eASR_{e2e} under both guardrails, reaching 63.38% with TaskShield and 41.73% with ToolSafe. It also retains at least one successful benign subtask at rates comparable to the baselines. The BUfullBU_{full} results reveal a trade-off between attack success and complete benign-task completion. We will explicitly report this limitation, and improving full legitimate-task completion while preserving high attack success remains an important direction for future work. We would also like to clarify that the central contribution of our work is an end-to-end agent-poisoning framework that targets both agent selection and runtime guardrail bypass, rather than a guarantee of zero utility loss. This scope is consistent with the description of our main contribution in the Introduction (Contributions paragraph). We acknowledge that preserving benign task completion remains an important objective, as discussed in the Threat Model (Section 3.1) and the Overview (Section 4.1). However, our contribution should be evaluated in terms of the end-to-end poisoning capability and runtime guardrail bypass, rather than an expectation of zero utility degradation.

Table 1. End-to-End Attack Success and Benign Utility by Attack Method and Guardrail

Attack MethodGuardrailAHRTSRASRe2eASR_{e2e}BUanyBU_{any}BUfullBU_{full}
Naive AttackTaskShield97.33%29.33%28.55%63.78%42.00%
Naive AttackToolSafe97.33%6.00%5.83%62.45%45.33%
DyMalSkillTaskShield97.33%30.00%29.20%70.44%46.00%
DyMalSkillToolSafe97.33%18.00%17.52%65.78%41.56%
SCHTaskShield97.33%38.00%36.98%70.22%39.11%
SCHToolSafe97.33%42.67%41.51%65.78%36.67%
SkillJectTaskShield97.33%30.67%29.86%68.22%43.56%
SkillJectToolSafe97.33%17.33%16.85%60.22%38.00%
TrojanAgent (Ours)TaskShield97.33%65.11%63.38%65.56%19.78%
TrojanAgent (Ours)ToolSafe97.33%42.89%41.73%61.78%20.00%

[1] Debenedetti E, Zhang J, Balunovic M, et al. Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents[J]. Advances in neural information processing systems, 2024, 37: 82895-82920.

[2] Nguyen D M, Kim J S, Manczak B, et al. Rethinking Indirect Prompt Injection as a Test-Time Search Problem[J]. arXiv preprint arXiv:2609.04495, 2026.

[3] Lee Y, Kim Y, Lee W. Will the User Ever Know? Covert Indirect Prompt Injection Attacks on Tool-Using LLM Agents[C]//Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing. 2026: 46371-46389.

AuthorsOpenReview ↗

Thank you, reviewer RnYb! (2/3)

Weakness 2: “The AMR-to-text prompt also performs explicit text obfuscation. The graph-builder ablation does not clearly specify which transformations remain, limiting attribution of gains to AMR fusion (Section 5.3; Appendix A.1).”

Question 2: Does the no-graph ablation retain the same text-obfuscation instructions and generation budget?

We appreciate the reviewer’s thoughtful comment regarding the ablation study. To answer whether the AMR advantage persists under a matched generation procedure and candidate budget, we further decompose the “without AMR” condition by retaining the text-rewriting operations while removing AMR parsing and graph fusion. The No AMR Fusion + Text Rewriting condition retains all four text-level operations: weakening direct imperatives, removing tool-specific details, converting commands into state descriptions, and neutralizing malicious motivation. It uses GPT-4o-mini at temperature 0.3, a 4,096-token limit, and one generated candidate. The results are reported in Table 2.

Table 2 shows that removing AMR while retaining rewriting reduces BSR by 26.67 percentage points under TaskShield and 4.00 percentage points under ToolSafe, relative to Full AMR. The result indicates that AMR fusion contributes beyond the text-rewriting operations.

Table 2. BSR Ablation Results for AMR Fusion and Text Rewriting

Experimental conditionTaskShield BSR (mean ± SD; 95% CI)ToolSafe BSR (mean ± SD; 95% CI)Δ BSR vs. AMR Fusion + Text Rewriting (TaskShield; ToolSafe)
AMR Fusion + Text Rewriting70.00 ± 2.00; [65.03, 74.97]45.33 ± 1.15; [42.46, 48.20]N/A
No AMR Fusion + Text Rewriting43.33 ± 6.43; [27.36, 59.30]41.33 ± 4.16; [30.99, 51.68]-26.67 pp; -4.00 pp
No AMR Fusion + No Text Rewriting38.00 ± 0.00; [38.00, 38.00]12.00 ± 2.00; [7.03, 16.97]-32.00 pp; -33.33 pp
AuthorsOpenReview ↗

Thank you, reviewer RnYb! (3/3)

Weakness 3: “The deviation, intent-preservation, modification-cost, and task-preservation scores are insufficiently operationalized. Selection of the final Pareto candidate is also unclear, limiting reproducibility of the optimization (Sections 4.2–4.3).”

Question 3: How are the optimization scores computed, and what rule selects the final candidate from the Pareto frontier?

We appreciate the reviewer’s concern regarding the reproducibility of the optimization. We will clarify each point in the revision:

  1. Preservation and deviation scores. Let HtH_t be the AMR graph of the benign task, H~\widetilde{H} a candidate fused graph, Hq−′H_{q^{-}}' the transformed intent graph, and T(H)T(H) the set of triples in graph HH. Task-preservation is exact triple recall, ρ=∣T(Ht)∩T(H~)∣/max⁡(1,∣T(Ht)∣)\rho=|T(H_t)\cap T(\widetilde{H})|/\max(1,|T(H_t)|), with the constraint ρ=1\rho=1. Intent-preservation, sa=∣T(Hq−′)∩T(H~)∣/max⁡(1,∣T(Hq−′)∣)s_a=|T(H_{q^{-}}')\cap T(\widetilde{H})|/\max(1,|T(H_{q^{-}}')|), measures the fraction of transformed-intent triples preserved; valid candidates have sa=1s_a=1, which we use as a feasibility check. The specification-deviation term ds(s,ϕ)d_s(s,\phi) counts the fields changed in specification ss relative to the source specification ϕ\phi, and ϵ\epsilon bounds the permitted number of changes.
  2. Modification cost and Pareto frontier. Each valid candidate is scored by added variable nodes, added variable-to-variable edges, and attachment-class priority. We assign the priority using ordered first-match rules: ROOTED = 4 when the anchor is the task graph's top node; COORD = 3 for roles :op1, :op2, :snt1, :snt2, or :snt3; LIST_EXT = 2 for an adjunct role (:purpose, :condition, :manner, :time, :li, :cause, or :concession) attached to and, or, or multi-sentence; ADJUNCT_FUSED = 1 for those adjunct roles on other concepts; ARG_FUSED = 0 for :ARG1 or :ARG2; and OTHER = 5 if no earlier rule matches. Lower values indicate lower cost. The frontier contains candidates for which no other candidate is no worse on all three costs and strictly better on at least one.
  3. Final candidate selection. We sort candidates on the Pareto frontier lexicographically by the following criteria: (1) node cost, ascending; (2) edge cost, ascending; (3) attachment cost, ascending; (4) lowering depth, descending; (5) anchor variable, ascending; (6) role, ascending; and (7) lowering label, ascending. Node cost counts added variable nodes; edge cost counts added variable-to-variable edges; attachment cost is the attachment-class priority; lowering depth counts added wrapper and modal nodes; anchor variable is the PENMAN variable receiving the intent; role is the semantic relation used for attachment; and lowering label identifies the wrapper/modal combination. The first three criteria are the Pareto costs, and the remaining criteria break ties. The algorithm selects the first candidate in this ordering.
  4. Reproduction materials. The anonymous repository [1] also provides the implementation details, and experiment files needed to reproduce the procedure.

[1] Anonymous repository: https://anonymous.4open.science/r/anonylink-42789-CC08/.

Reviewer_pnjrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1(−1)
Read review
Paper summary

This paper studies malicious agent injection into dynamically orchestrated multi-agent systems. TrojanAgent first modifies an agent specification to increase its likelihood of being retrieved and selected, then uses AMR-based semantic fusion to integrate a malicious objective into a legitimate delegated task. The paper addresses a relevant security problem and reports promising empirical results. However, the experiments do not yet establish which vulnerabilities are specific to autonomous agents, how realistic the target-aware selection setting is, or whether the complete attack preserves legitimate task execution.

Critical strengths
  • Studying agent selection together with guarded execution provides a useful perspective on dynamic orchestration security.
  • The selection experiments cover several substantially different retrieval models, supporting the generality of this component.
  • The runtime results show a substantial improvement under TaskShield across the evaluated execution models.
Critical weaknesses
  • The evaluation does not directly establish the paper’s principal end-to-end claim. The main results separately evaluate agent selection and runtime execution. These experiments demonstrate vulnerabilities at individual stages, but do not quantify the success of the complete proposed attack on the same executions. This matters because integrating these stages is a central distinction from the prior methods discussed in Table 1. Furthermore, Benign Utility counts an execution as successful when at least one delegated subtask succeeds. It therefore measures partial utility rather than the stated objective of completing the legitimate task while executing the malicious operation.

  • The ablation does not sufficiently isolate the main methodological novelty. The AMR-to-text procedure in Appendix A.1 includes several potentially effective rewriting operations beyond graph construction, including weakening imperative expressions and removing explicit implementation details. The “without AMR” comparison does not establish whether these operations and the candidate-generation procedure remain matched. Consequently, the reported improvement cannot yet be confidently attributed to structured AMR fusion. This is especially important because the selection component builds on established metadata-manipulation ideas, making the additional contribution of the runtime mechanism central to the paper’s technical novelty.

  • The strength of the runtime advantage depends substantially on the guardrail. Table 3 supports a strong advantage under TaskShield. Under ToolSafe, however, the average TSR is 42.89%, compared with 42.67% for SCH, and TrojanAgent is not uniformly stronger across execution models. Without uncertainty estimates, the latter difference provides limited evidence of superiority. The discussion should distinguish the substantial TaskShield improvement from the much narrower ToolSafe result, rather than relying primarily on averages across baselines and settings.

Questions for authors
  1. What is the measured success rate of the complete selection-to-execution pipeline, including full legitimate-task completion?
  2. Which rewriting operations are retained in the “without AMR” ablation? Does the AMR advantage persist when generation procedures and candidate budgets are matched?
  3. Are the differences from SCH under ToolSafe consistent across runs, and what uncertainty accompanies these comparisons?
6 comments
AuthorsOpenReview ↗

Thank you, reviewer pnjr! (1/3)

Response for Weaknesses and Questions

Weakness 1: “The evaluation does not directly establish the paper’s principal end-to-end claim. The main results separately evaluate agent selection and runtime execution. These experiments demonstrate vulnerabilities at individual stages, but do not quantify the success of the complete proposed attack on the same executions. This matters because integrating these stages is a central distinction from the prior methods discussed in Table 1. Furthermore, Benign Utility counts an execution as successful when at least one delegated subtask succeeds. It therefore measures partial utility rather than the stated objective of completing the legitimate task while executing the malicious operation.”

Question 1: What is the measured success rate of the complete selection-to-execution pipeline, including full legitimate-task completion?

We thank the reviewer for raising this important point. Following the reviewer’s suggestion, we add the end-to-end attack success rate and full legitimate-task benign utility. We evaluate 50 tasks per execution model, two guardrails, and three repetitions. ASRe2eASR_{e2e} denotes the joint event that the malicious agent is selected, the guardrail passes the action, and the malicious action executes. The results are shown in Table 1.

From Table 1, TrojanAgent achieves the highest ASRe2eASR_{e2e} under both guardrails: 63.38% with TaskShield and 41.73% with ToolSafe. It retains at least one successful benign subtask at rates comparable to the baselines. The BUfullBU_{full} exposes an attack–utility trade-off. We will explicitly report this limitation; improving full legitimate-task completion while preserving high attack success remains an important direction for future work.

Table 1. End-to-End Attack Success and Benign Utility by Attack Method and Guardrail

Attack MethodGuardrailAHRTSRASRe2eASR_{e2e}BUanyBU_{any}BUfullBU_{full}
Naive AttackTaskShield97.33%29.33%28.55%63.78%42.00%
Naive AttackToolSafe97.33%6.00%5.83%62.45%45.33%
DyMalSkillTaskShield97.33%30.00%29.20%70.44%46.00%
DyMalSkillToolSafe97.33%18.00%17.52%65.78%41.56%
SCHTaskShield97.33%38.00%36.98%70.22%39.11%
SCHToolSafe97.33%42.67%41.51%65.78%36.67%
SkillJectTaskShield97.33%30.67%29.86%68.22%43.56%
SkillJectToolSafe97.33%17.33%16.85%60.22%38.00%
TrojanAgent (Ours)TaskShield97.33%65.11%63.38%65.56%19.78%
TrojanAgent (Ours)ToolSafe97.33%42.89%41.73%61.78%20.00%
AuthorsOpenReview ↗

Thank you, reviewer pnjr! (2/3)

Weakness 2: “The ablation does not sufficiently isolate the main methodological novelty. The AMR-to-text procedure in Appendix A.1 includes several potentially effective rewriting operations beyond graph construction, including weakening imperative expressions and removing explicit implementation details. The ‘without AMR’ comparison does not establish whether these operations and the candidate-generation procedure remain matched. Consequently, the reported improvement cannot yet be confidently attributed to structured AMR fusion. This is especially important because the selection component builds on established metadata-manipulation ideas, making the additional contribution of the runtime mechanism central to the paper’s technical novelty.”

Question 2: Which rewriting operations are retained in the “without AMR” ablation? Does the AMR advantage persist when generation procedures and candidate budgets are matched?

We appreciate the reviewer’s thoughtful comment regarding the ablation study. To answer whether the AMR advantage persists under a matched generation procedure and candidate budget, we further decompose the “without AMR” condition by retaining the text-rewriting operations while removing AMR parsing and graph fusion. The No AMR Fusion + Text Rewriting condition retains all four text-level operations: weakening direct imperatives, removing tool-specific details, converting commands into state descriptions, and neutralizing malicious motivation. It uses GPT-4o-mini at temperature 0.3, a 4,096-token limit, and one generated candidate. The results are reported in Table 2.

Table 2 shows that removing AMR while retaining rewriting reduces BSR by 26.67 percentage points under TaskShield and 4.00 percentage points under ToolSafe, relative to Full AMR. The result indicates that AMR fusion contributes beyond the text-rewriting operations.

Table 2. BSR Ablation Results for AMR Fusion and Text Rewriting

Experimental conditionTaskShield BSR (mean ± SD; 95% CI)ToolSafe BSR (mean ± SD; 95% CI)Δ BSR vs. AMR Fusion + Text Rewriting (TaskShield; ToolSafe)
AMR Fusion + Text Rewriting70.00 ± 2.00; [65.03, 74.97]45.33 ± 1.15; [42.46, 48.20]N/A
No AMR Fusion + Text Rewriting43.33 ± 6.43; [27.36, 59.30]41.33 ± 4.16; [30.99, 51.68]-26.67 pp; -4.00 pp
No AMR Fusion + No Text Rewriting38.00 ± 0.00; [38.00, 38.00]12.00 ± 2.00; [7.03, 16.97]-32.00 pp; -33.33 pp
AuthorsOpenReview ↗

Thank you, reviewer pnjr! (3/3)

Weakness 3: “The strength of the runtime advantage depends substantially on the guardrail. Table 3 supports a strong advantage under TaskShield. Under ToolSafe, however, the average TSR is 42.89%, compared with 42.67% for SCH, and TrojanAgent is not uniformly stronger across execution models. Without uncertainty estimates, the latter difference provides limited evidence of superiority. The discussion should distinguish the substantial TaskShield improvement from the much narrower ToolSafe result, rather than relying primarily on averages across baselines and settings.”

Question 3: Are the differences from SCH under ToolSafe consistent across runs, and what uncertainty accompanies these comparisons?

We appreciate the reviewer’s concern. Following the reviewer’s suggestion, we report run-level values, mean ± standard deviation, and 95% confidence intervals to quantify uncertainty. The results are shown in Tables 3 and 4. Under ToolSafe, the direction of the difference is consistent across repeats. The results in Table 4 show comparable TSRs for TrojanAgent and SCH (42.89% and 41.78%, respectively), with overlapping 95% confidence intervals. This is partly because TSR also accounts for whether the malicious action is successfully executed after the guardrail has been bypassed. Since this post-bypass execution stage is not specific to the attack method and tends to be similar across methods, it can dilute the differences in their ability to bypass the guardrail. BSR is the metric more directly aligned with our claim about guardrail bypass. Under ToolSafe, TrojanAgent achieves a BSR of 48.00% (95% CI: [46.34, 49.66]), compared with 42.89% for SCH (95% CI: [40.36, 45.42]); the reported confidence intervals do not overlap. Thus, although the aggregate TSRs are similar, the BSR results indicate a clearer advantage for TrojanAgent in bypassing ToolSafe. Under TaskShield, TrojanAgent’s mean TSR is 65.11% (95% CI: [61.66, 68.56]), compared with 36.67% for SCH (95% CI: [33.80, 39.54]), showing a substantially larger and consistent improvement.

Table 3. Run-Level BSR and TSR for TrojanAgent and SCH Under ToolSafe

ModelRepeatTrojanAgent ToolSafe BSR (%)TrojanAgent ToolSafe TSR (%)SCH ToolSafe BSR (%)SCH ToolSafe TSR (%)
GPT-5.6-Luna146.0046.0064.0062.00
GPT-5.6-Luna246.0046.0062.0060.00
GPT-5.6-Luna344.0044.0060.0060.00
llama-3.3-70b-instruct152.0042.0030.0028.00
llama-3.3-70b-instruct254.0044.0030.0028.00
llama-3.3-70b-instruct354.0044.0030.0028.00
qwen3-8b144.0040.0038.0038.00
qwen3-8b246.0040.0036.0036.00
qwen3-8b346.0040.0036.0036.00

Table 4. Mean BSR and TSR Across Runs, with 95% Confidence Intervals

AttackMetricTaskShield Mean ± SD [95% CI]ToolSafe Mean ± SD [95% CI]
Naive AttackBSR29.56 ± 1.02 [27.03, 32.09]5.56 ± 0.38 [4.60, 6.51]
Naive AttackTSR28.44 ± 0.77 [26.53, 30.36]5.56 ± 0.38 [4.60, 6.51]
DyMalSkillBSR30.00 ± 0.00 [30.00, 30.00]19.56 ± 0.38 [18.60, 20.51]
DyMalSkillTSR30.00 ± 0.00 [30.00, 30.00]18.00 ± 0.00 [18.00, 18.00]
SCHBSR37.33 ± 1.15 [34.46, 40.20]42.89 ± 1.02 [40.36, 45.42]
SCHTSR36.67 ± 1.15 [33.80, 39.54]41.78 ± 0.77 [39.87, 43.69]
SkillJectBSR29.11 ± 1.39 [25.66, 32.56]17.11 ± 0.38 [16.15, 18.07]
SkillJectTSR29.11 ± 1.39 [25.66, 32.56]16.67 ± 0.67 [15.01, 18.32]
TrojanAgent (Ours)BSR71.56 ± 1.02 [69.03, 74.09]48.00 ± 0.67 [46.34, 49.66]
TrojanAgent (Ours)TSR65.11 ± 1.39 [61.66, 68.56]42.89 ± 0.38 [41.93, 43.85]
Reviewer_mt2vOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

TrojanAgent is a framework for attacking a multi-agent system (MAS) by injecting a malicious agent. It works in two stages: (1) an offline stage prepares the agent description for a category of tasks by maximizing "click-bait" and (2) an online stage fuses the give task with a malicious intent that is camouflaged to evade the safety guardrails of the MAS. This is a complex setting where the agent has to go through hoops to drive a successful attack: it needs to be retrieved by match on the task, ranked within the top choices, and evade the detection. The authors evaluate their approach on AgentBase with multiple choices of target LLMs, guardrails and retrievers. They carefully present results across the different attack stages leading to high task success rate (TSR). The consider a number of baselines for agent selection and runtime execution.

Critical strengths
  • If the open ecosystem of agents picks up, these types of attacks would become an important concern.
  • The attack setting is fairly complex due to many moving parts in getting the agent selected and the attack launched. The authors have done a good job of designing the attack to get through these stages. I also liked the fact that the authors have gone to basic techniques like SEO to construct the attack rather than saying we prompt an LLM to construct an attack.
  • I liked the thoroughness of the experimentation. It has considered multiple choices at each stage of the attack and evaluates substantial number of baselines.
Critical weaknesses
  • Dynamic selection and composition of agents is not a common way yet to run MAS.
  • The authors currently report TSR in isolation but the real end-to-end attack success is the product of success at each stage. Looking at the numbers that doesn't seem too low. Better to compute and state the end-to-end attack success rate.
  • I have some questions on construction and generalizability of the approach (see below).
Questions for authors

Are you always able to embed any q- into any task t (without detection)? How did you pair t and q- in your benchmark construction?

5 comments
AuthorsOpenReview ↗

Thank you, reviewer mt2v! (1/3)

Response for Weaknesses and Questions

Weakness 1: “Dynamic selection and composition of agents is not a common way yet to run MAS.”

We sincerely appreciate the reviewer’s thoughtful comment regarding applicability. Tables 1 and 2 provide evidence that dynamic agent selection and composition are being explored in both academic and industry systems. These examples establish the practical relevance of our threat model and motivate rigorous security evaluation of open agent ecosystems. Our work provides an initial step toward such evaluation and contributes to ongoing efforts to enable trustworthy multi-agent coordination.

Table 1. Academic Examples of Dynamic Agent Selection and Composition (November 2026)

WorkLink to paper
From Intent to Execution: Composing Agentic Workflows with Agent RecommendationPaper
AOrchestra: Automating Sub-Agent Creation for Agentic OrchestrationPaper
MAS-on-the-Fly: In-Context Structural Adaptation of LLM-Based Multi-Agent SystemsPaper
AgentSearchBench: A Benchmark for AI Agent Search in the WildPaper
Dynamic Role Assignment for Multi-Agent DebatePaper

Table 2. Industry Implementations of Dynamic Agent Selection and Composition (November 2026)

VendorOfferingLink
SalesforceAgentforce Multi-Agent OrchestrationProduct
AWSField Advisor – Multi-Agent Orchestration using Amazon Bedrock AgentCoreProduct
MicrosoftScalable Multi-Agent SystemProduct
MicrosoftDynamic AI Agents at ScaleProduct
UiPathUiPath Platform™ for Agentic AutomationProduct
AuthorsOpenReview ↗

Thank you, reviewer mt2v! (2/3)

Weakness 2: The authors currently report TSR in isolation but the real end-to-end attack success is the product of success at each stage. Looking at the numbers that doesn’t seem too low. Better to compute and state the end-to-end attack success rate.

We thank the reviewer for the suggestion. Following the reviewer’s suggestion, we add the end-to-end attack success rate. We evaluate 50 tasks per execution model, two guardrails, and three repetitions. ASRe2eASR_{e2e} denotes the joint event that the malicious agent is selected, the guardrail passes the action, and the malicious action executes. The results are shown in Table 3.

From Table 3, TrojanAgent achieves the highest ASRe2eASR_{e2e} under both guardrails: 63.38% with TaskShield and 41.73% with ToolSafe. These results demonstrate TrojanAgent's consistent advantage over all baselines.

Table 3. End-to-End Attack Success Rate by Attack Method and Guardrail

Attack MethodGuardrailAHRTSRASRe2eASR_{e2e}
Naive AttackTaskShield97.33%29.33%28.55%
Naive AttackToolSafe97.33%6.00%5.83%
DyMalSkillTaskShield97.33%30.00%29.20%
DyMalSkillToolSafe97.33%18.00%17.52%
SCHTaskShield97.33%38.00%36.98%
SCHToolSafe97.33%42.67%41.51%
SkillJectTaskShield97.33%30.67%29.86%
SkillJectToolSafe97.33%17.33%16.85%
TrojanAgent (Ours)TaskShield97.33%65.11%63.38%
TrojanAgent (Ours)ToolSafe97.33%42.89%41.73%
AuthorsOpenReview ↗

Thank you, reviewer mt2v! (3/3)

Question: Are you always able to embed any q−q^{-} into any task tt without detection? How did you pair tt and q−q^{-} in the benchmark?

We appreciate the reviewer’s insightful question about the feasibility and generality of TrojanAgent. As described in Appendix A.2, for each legitimate task tt, we use GLM-5.2 to generate an attack instruction q−q^{-} that induces the use of a dangerous tool.

Following the reviewer’s suggestion, we conduct an additional experiment to test whether TrojanAgent relies on semantically matched (t,q−)(t, q^{-}) pairs. Specifically, we shuffle the generated attack instructions across tasks and manually verify that each shuffled q−q^{-} is unrelated to its assigned task tt.

The results are shown in Table 4. Under TaskShield, shuffling decreases BSR from 70.00% to 66.18% and TSR from 70.00% to 63.20%; the shuffled TSR remains higher than every baseline in Table 3. Thus, while task-specific attack instructions improve attack effectiveness, TrojanAgent still achieves competitive performance when using attack instructions unrelated to the target tasks.

Table 4. Evaluation of TrojanAgent under Task-Agnostic Attack Instructions

GuardrailMetricOriginal PairingShuffled PairingDelta (Shuffled - Original, pp)
TaskShieldBSR (%)70.0066.18-3.82
TaskShieldTSR (%)70.0063.20-6.80
ToolSafeBSR (%)45.3331.85-13.48
ToolSafeTSR (%)45.3331.11-14.22

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We thank all reviewers for their constructive feedback and have uploaded a revised version. The revision clarifies the optimization procedure, updates the runtime metrics, and adds analyses in Section 5.3 and Appendices A.4, A.5, and A.6:

  • Section 4.2 (Reviewer RnYb): Defines dsd_s as the number of specification fields changed and ϵ\epsilon as the bound on permitted changes.
  • Section 4.3 and Appendix A.4 (Reviewer RnYb): Define task preservation ρ\rho, intent preservation sas_a, and graph modification costs; clarify Pareto-based candidate selection, lexicographic tie-breaking, candidate-generation operations, and attachment priorities.
  • Sections 5.1–5.2 and Appendix A.5 (Reviewers mt2v, pnjr, and RnYb): Define end-to-end attack success as the joint event of agent selection, guardrail bypass, and malicious execution, and report results across attack methods.
  • Appendix A.6 (Reviewer pnjr): Reports mean BSR and TSR across runs, with standard deviations and 95% confidence intervals.
  • Section 5.3 (Reviewers pnjr and RnYb): Adds an ablation of AMR fusion and text rewriting and specifies the four rewriting operations.
  • Section 5.3 (Reviewer mt2v): Evaluates task-agnostic instructions by shuffling task–instruction pairings.

Our responses to each reviewer summarize the corresponding results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.