- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper examines whether attacker-controlled repository content can mislead LLM-based MCP scanners. The proposed attack, CPI, uses the scanner’s initial findings to write false claims that specific vulnerabilities have been fixed. These claims are then added to README files, tool descriptions, and code comments. The evaluation covers three scanners and 372 real-world MCP server repositories. The results suggest that the tested MCP scanners are vulnerable to CPI attacks.
Critical strengths
- The evaluation includes real-world MCP server repositories and a manually verified vulnerability subset, making the problem practically relevant and providing evidence beyond synthetic examples.
- The experiments cover three widely used MCP scanners and multiple injection surfaces, revealing susceptibility to attacker-controlled remediation claims under the evaluated configurations.
- The controlled ablations support the proposed mechanism: vulnerability-specific narratives outperform direct instructions and generic claims, and multi-context injection is strongest, while the implementation is left unchanged.
Critical weaknesses
- The novelty relative to closely related work is insufficiently established. Prior work [1-2] on LLM-based vulnerability detection and security code review has already examined non-executable adversarial comments and misleading security narratives. Although the paper studies MCP scanners as a new application setting, it does not clearly demonstrate substantial novelty in its attack surfaces compared with prior work.
- Important evaluation details are missing. The paper omits key evaluation details, including the backend models used by the MCP scanners. Moreover, without reporting the scanners’ false positive rates, it's difficult to assess their detection reliability and contextualize the reported attack effectiveness.
- The threat model assumes access to a detailed baseline report. CPI relies on the scanner's per-finding output to construct vulnerability-specific narratives. Limited-feedback or no-feedback settings, where the scanner returns only a coarse summary, are not evaluated, so the attack's dependence on detailed scanner output is unclear.
[1] Alibi: Adaptive Agentic Attacks on LLM-Based Vulnerability Detectors via Adversarial Code Comments
[2] SEVRA-BENCH: Social Engineering of Vulnerabilities in Review Agents
Questions for authors
-
Evaluation Metrics. The evaluation uses BDR, PDR, and ASR, with PDR defined as the fraction of repositories for which all baseline target findings are retained after CPI injection. Under this definition, suppressing even one target finding in a repository with multiple baseline findings counts as a successful attack. However, I would also like to see a stricter success metric that counts an attack as successful only when the post-attack scan reports no vulnerabilities. This is particularly relevant to the authors’ threat model: a plausible user policy is to reject an MCP server whenever any vulnerability is detected, regardless of how many findings remain. Could the authors additionally report (1) the average number of vulnerability findings per repository before and after the attack, and (2) the number and proportion of baseline-positive repositories with zero vulnerability findings after CPI injection?
-
Evaluated Models. Tencent A.I.G. supports configurable model backends, but the authors do not disclose which model was used in their evaluation. Could the authors specify the backend model and evaluate at least one SOTA model (e.g., gpt-6) within the same scanner workflow to assess whether their findings hold for scanners powered by SOTA models?
-
Attack Under Limited Feedback. CPI uses the scanner's baseline findings to build its narratives. How does the attack's effectiveness degrade when the adversary receives only a pass/fail verdict or a coarse severity summary rather than per-finding details, and can you provide results in such a limited-feedback setting?