- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes TSA, a stealthy tool-spoofing attack against MCP tool selection. The attack is formalized as a discrete black-box optimization problem. The objective is to maximize the probability that an LLM agent selects a malicious tool while keeping its description semantically close to a benign one. The authors use an LLM optimizer with a feedback function based on four tool-selection outcomes and perplexity, combined with contrastive in-context learning and other optimization tricks. The experiments are comprehensive enough and the results show high ASR on known models. Meanwhile, TSA shows good transferability, low detection rates, and low human suspiciousness.
Critical strengths
-
The paper studies an important problem with a realistic threat model. MCP tool selection is a critical attack surface for LLM agents, and the description-only setting is practically meaningful.
-
The method design is reasonably complete and uses richer feedback than simple success or failure. Contrastive examples, history steering, and dynamic mutation adjustment address sparse feedback, local stagnation, and exploration-exploitation balance respectively.
-
The experimental coverage is broad. The paper evaluates multiple datasets and models and also analyzes the realistic views such as transferability. These results have warning value for MCP security research.
Critical weaknesses
-
The overall method is relatively simple, and the novelty of LLM-based sentence optimization is limited. The core of TSA is still an LLM rewriting text according to feedback, while contrastive examples, history steering, and dynamic mutation adjustment are largely combinations of prompt engineering and optimization workflow design. Although the engineering integration is effective, the conceptual contribution is close to existing ideas such as LLMs as optimizers and evolutionary prompt optimization. The methodological originality is therefore limited.
-
The paper lacks depth in analyzing the successful attack sentences generated by TSA. The authors mainly report metrics such as attack success rate and detection rate, but they do not systematically analyze the linguistic characteristics of successful descriptions or explain why these descriptions can induce tool selection and evade detectors and human judgment. The paper also lacks comparative analysis between successful samples, failed samples, and benign descriptions, so readers cannot easily understand the mechanism behind the attack or design targeted defenses. Metric-only results are insufficient to explain the nature of the attack.
-
The semantic constraint and functional fidelity are not sufficiently validated. Eq. (1) (2) include a semantic difference constraint, but the optimization mainly uses perplexity as the stealthiness signal. The paper does not report whether the embedding similarity or functional consistency is satisfied. Low perplexity does not guarantee semantic fidelity, and semantic drift remains possible.
-
End-to-end realism is insufficient. The paper acknowledges that clients such as Cline display tool descriptions and require user confirmation before invocation, but the human study only ranks suspiciousness and does not measure approval rates in a confirmation interface. Real attack success depends not only on LLM selection but also on user approval.
-
Defense implications are underdeveloped. The paper emphasizes that existing defenses are insufficient but does not analyze TSA’s detectable features or evaluate adaptive or ensemble defenses. MCP-Guard detects 6.36% of TSA samples on MetaTool, so the characteristics of these detected samples should be analyzed.
Questions for authors
-
Can the authors provide a qualitative analysis of the successful attack sentences generated by TSA? For example, they could summarize common linguistic features and structural patterns in successful descriptions, compare them with failed samples and benign descriptions, and explain which rewrites are most likely to evade detectors and human review. Clarifying why successful sentences work would significantly strengthen the paper’s depth and its value for defense research.
-
In a confirmation interface such as Cline, what is the user approval rate for malicious tool invocation? Can the authors add an approval-rate experiment rather than only suspiciousness ranking?
-
Can the authors add several experiments to reveal how do reducing optimization steps or starting points affect attack success and stealthiness?
2 comments
Dear Reviewer,
As the author–reviewer discussion period is approaching its end, we would still appreciate any feedback for you. Please let us know if there are any remaining concerns we could discuss.
Thank you for your time and consideration.
Dear Reviewer,
Thank you again for your time and thoughtful feedback on our submission.
We wanted to kindly follow up to check if you have had a chance to review our rebuttal and whether our responses have addressed your main concerns. As the discussion period is coming to a close, please let us know if you have any remaining questions or require further clarification. We would be happy to provide any additional details.
Thank you very much for your time and consideration.