- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ATLAS, an atomized topology-aware attack search framework for indirect prompt injection in multi-agent systems. It combines RST-guided Attack Atomization, Topology-aware Attack Search using MCTS, and Terminal Semantic Closure. Experiments across three benchmarks and ten victim models demonstrate substantial improvements in attack success rates. Baseline comparisons and ablation studies further support the effectiveness of the proposed framework.
Critical strengths
-
Strong empirical performance. ATLAS achieves substantial ASR improvements across three benchmarks and diverse victim models. The additional baseline comparisons and ablation studies provide further evidence of its effectiveness.
-
Interesting framework design. The integration of RST-guided attack atomization, topology-aware search, and terminal semantic closure provides a structured approach to generating multi-point prompt injection attacks.
Critical weaknesses
-
Insufficient validation of semantic evaluation reliability. The trace-level analysis relies on Qwen2.5-3B-Instruct to evaluate Semantic Retention, Behavioral Adoption, and Semantic Propagation. However, the reliability of these scores is not independently validated. Although Appendix C.2 evaluates estimator fidelity, it measures agreement with trace-derived scores obtained using the same semantic evaluator. This raises concerns about the reliability of the empirical observations underlying the proposed method.
-
Insufficient implementation details of key components. The paper does not clearly specify how the topology-aware estimator is constructed and calibrated using GPT-4o execution traces. In particular, its estimation function and calibration procedure are insufficiently described. Since the estimator provides the key signals guiding MCTS, these omissions make it difficult to reproduce and fully assess the proposed method.
Questions for authors
-
Have the authors validated the semantic attribution scores against human annotations or other independent LLM evaluators?
-
Could the authors clarify the specific implementation and calibration procedure of the topology-aware estimator, including whether it involves parameter learning and how the calibration data are constructed?