- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses the emerging risk that an adversary can infer the internal communication topology of LLM-based multi-agent systems (MAS) from black-box interactions. The authors propose MIRAGE, a defense that decouples the topology used for task execution from the semantics exposed to the adversary by synthesizing a structurally distinct phantom topology, materializing phantom edges via generated evidence, and obfuscating genuine edges through controlled paraphrasing in an exposure view. Experiments across three topology optimization frameworks (G-Designer, AGP, ARG-Designer) and four benchmarks (MMLU, GSM8K, SVAMP, HumanEval) show substantial drops in topology inference performance with minimal impact on task utility.
Critical strengths
-
Designing a defense framework against CIA [1] is novel.
-
Evaluation is across various baselines and benchmarks.
-
Presentation is clear with case studies to follow.
-
Code is provided to strenghthen the reliability.
[1] CIA: Inferring the Communication Topology from LLM-based Multi-Agent Systems (AAAI 2026)
Critical weaknesses
-
Vague reports of performance reduction. Figure 7 shows MIRAGE causes some performance drops in many test cases. Unfortunately, the authors only present bar charts rather than exact numbers in tables. Because the y‑axis of the chart starts from zero, small accuracy drops are hard to spot visually. Readers cannot tell exactly how much task performance is lost under different experimental setups.
-
No tests against adaptive attackers. Seemingly, under MIRAGE, attackers will infer a mixed topology that contains both real connections and fake phantom edges. However, for an adaptive attacker, the fake edges could be removed by existing MAS topology pruning methods [1,2]. After that, the attacker still obtains the real structure.
-
Lack of LLM backbones. As stated in the Appendix, "All LLM-based components are implemented using GPT-5.5 as the backbone model".
References
[1] CUT THE CRAP: AN ECONOMICAL COMMUNICATION PIPELINE FOR LLM-BASED MULTI-AGENT SYSTEMS (ICLR 2025).
[2] AgentDropout: Dynamic Agent Elimination for Token-Efficient and High-Performance LLM-Based Multi-Agent Collaboration (ACL 2025).
Questions for authors
-
Please report exact numerical values for the performance drop shown in Figure 7.
-
Can the attacker use existing MAS topology‑pruning methods or other potential solutions to remove phantom edges? If so, report how the success rate of the attack varies? If not, please explain why.
-
Please consider evaluating MIRAGE on more LLM backbones. (This is not my main concern)
Response to Reviewer CZkY (Part 2/3)
Thank you for this valuable suggestion. We would like to clarify that MIRAGE does not introduce additional phantom agents or modify the genuine communication topology. Instead, it constructs phantom communication edges among existing agents and generates corresponding semantic evidence in the adversary-facing outputs. The implementation consists of three stages:
Phantom Topology Synthesis (Section 4.1). Given the genuine topology G=(A,E), MIRAGE constructs an alternative topology G′=(A,E′) through order-preserving edge rewiring while preserving structural feasibility. The phantom edges and concealed genuine edges are defined as Ep=E′∖E and Eh=E∖E′, respectively.
Semantic Edge Realization (Section 4.2). The term materialization refers to generating observable semantic evidence for phantom dependencies rather than establishing actual communication channels. Specifically, for each phantom edge (aj,ai)∈Ep, an LLM-based generator produces task-relevant semantic evidence conditioned on the source agent's profile and output, the target agent's profile, the user query, and the target agent's task context. Meanwhile, controlled paraphrasing suppresses observable semantic dependencies associated with concealed genuine edges in Eh, while preserving task-relevant information.
Protected MAS Execution (Section 4.3). The original MAS continues executing over the genuine topology G, while the generated phantom evidence and paraphrased traces are assembled into a separate adversary-facing exposure view. These modified traces are not fed back into internal agent communication, ensuring that the genuine communication topology and execution process remain unchanged.
Regarding computational overhead, the additional cost mainly arises from generating semantic evidence for phantom edges and paraphrase candidates for concealed genuine dependencies. As reported in Appendix D.7, we measured the average inference latency per query under G-Designer, with results summarized below:
The results show that MIRAGE introduces an average latency overhead of 17.0% in the evaluated G-Designer setting. Importantly, this additional computation is confined to adversary-facing output protection and introduces no extra communication rounds in the genuine MAS execution. To avoid ambiguity, we also clarify the term phantom edge materialization as phantom dependency evidence generation in the methodological description, making its implementation and distinction from actual agent communication more explicit.