- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper focuses on the Denial-of-Service (DoS) risks during the runtime of multi-agent systems. For a single agent, if it is locally resource bounded, it can mitigate such security risks. However, their compostion may introduce previously underexplored risks of globally unbounded resource consumption due to the interactions among agents. The paper proposes 6 resource dimensions and 14 canonical DoS risk types, and conducts a detailed analysis of the existing system by implementing the MADScan tool. Ultimately, 68 confirmed DoS risk points with PoC validation were identified across 26 open-source agent systems, of which 15 required agent-to-agent interaction to trigger. MADScan achieved 100% observed precision and 100% observed recall in a benchmark consisting of 16 verified risk points.
Critical strengths
The paper investigates the Denial of Service risks in multi-agent systems and provides a detailed and convincing description of such underexplored security risks. A key insight is that the limitation of bounded local resources does not necessarily mean that global resources are bounded, and this insight has sufficient research motivation. And the types of resources and DoS risks also provide a reasonable framework for evaluating such risks.
In the evaluation, MADScan outperforms the baseline detectors on the benchmark. And identifies risk points that traditional single-agent-oriented tools largely miss. The empirical results can demonstrate that such security risks are not purely theoretical.
Critical weaknesses
The evaluation benchmark is relatively small, with only 16 validated risk points. More importantly, these risk points are obtained by manually analyzing resource-flow edges that were first extracted by MADScan itself. Therefore, risks missed during scan-scope selection or resource-flow extraction are unlikely to enter the benchmark, which limits the strength of the reported 100% precision and recall results.
I think the security impact of the threat model proposed in the paper and the risks revealed in the two cases is relatively limited. The first case appears closer to an engineering or workflow bug, while its adversarial exploitability is not clearly established. In the second case, without indirect prompt word injection, the successful triggering by the user only consumes their own resources and costs, without expanding the security issue. Therefore, the practical security consequences of these findings remain unclear.
Questions for authors
Does the benchmark construction introduce a bias in favor of MADScan, since the resource-flow edges are first extracted by MADScan itself before subsequent evaluation? Risks missed during scan-scope selection or resource-flow extraction may never enter the benchmark.
Can the authors provide a clear attack chain to illustrate how attackers affect benign users and cause actual security impact? In particular, what concrete impact or potential loss can an attacker cause beyond merely triggering the failure behavior?
4 comments
Dear Reviewer,
As the deadline for the final author response approaches, we would like to kindly ask whether you have any remaining questions or suggestions for further revisions. If you feel that our responses have adequately addressed your concerns, we would greatly appreciate it if you could reconsider your score accordingly.
Thank you again for your time and valuable feedback!
Best regards,
The Authors
I appreciate the authors’ detailed response. Unfortunately, I am going to keep my score as is. Two concerns still prevent me from raising it: 1. The new independent benchmark still contains only 25 risks from 10 systems. 2. The new end-to-end case strengthens the exploitability argument, but the demonstrated security impact remains relatively limited and does not yet show severe service disruption in realistic deployments.
Thank you for the careful reading and for engaging with our response. We understand both remaining concerns.
On the benchmark, we will state clearly in the final version that 25 risk points from 10 systems support estimates on a small sample.
On security impact, the end-to-end case demonstrates exploitability rather than severe service disruption in a production deployment, and we will frame the impact claims accordingly.
Thank you again for your time and constructive feedback.
Response to Reviewer MJQ2
Thank you for recognizing the motivation and the empirical demonstration that these risks are not purely theoretical.
W1 / Q1 -- Does constructing the benchmark from MADScan's own extracted edges bias it in MADScan's favor? This was a fair concern, and we have removed the circularity. The new ground truth is built independently of MADScan: two experts independently annotated every DoS risk in the full source code of 10 randomly sampled systems (48 person-hours each; a third expert resolved disagreements). Because the annotation covers the entire code base rather than MADScan's resource-flow edges, a risk missed during scope selection or extraction can enter the ground truth. On the resulting 25 risk points, MADScan attains 100% precision and 100% recall. AgentFuzz and AgentDoS reach 28.0% and 60.0% recall.
W2 / Q2 -- A clear attack chain, and concrete impact on benign users beyond self-consumption.
Case I (GraphFlow) is a developer-configuration failure that requires no attacker input; it demonstrates that the D-X mechanism exists in a widely used framework. Case II (AutoGPT) is a per-user cost cap bypassed by a single user’s concurrency, so the loss is borne by the operator rather than by other users.
For the attacker-controlled chain with cross-party impact the reviewer asks for, we add an end-to-end case (Case III, Section 5 and Appendix E). The attacker plants content in material the system retrieves, an indirect prompt injection in a background note. A benign operator then runs a multi-agent debate over that content with a live model (gpt-4.1-mini): the injected instructions make a debate agent emit extra data-request tags, and a downstream search analyst dispatches them concurrently with no cap. The amplified requests are paid from the operator’s search budget, not the attacker’s, so attacker and victim are distinct. Across three trials the injection roughly doubles the requests a debate emits (mean 4.3 to 9.3) and raises peak concurrent searches from 2 to 4.