- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates skill-injection attacks that amplify token consumption in large language model (LLM)-based agents. By modifying a SKILL.md file, attackers induce agents to consume substantially more tokens than under benign conditions while appearing to complete their assigned tasks, thereby increasing computational costs. The authors propose SKILLBLOAT, a two-stage attack pipeline. In Phase 1, the method selects an attack configuration from 15 candidate types and rewrites the entire SKILL.md file using generative adversarial networks (GANs). In Phase 2, the attack is iteratively optimized through an execution–diagnosis–feedback refinement loop.
Critical strengths
(1) Task-specific adaptive optimization.
Unlike prior approaches such as Clawdrain and CDH, which rely on fixed or manually constructed malicious skills, this paper recognizes that effective amplification strategies may vary across skill–task pairs. It addresses this variation through a two-stage optimization pipeline, providing a more adaptive approach to resource amplification attacks.
Critical weaknesses
(1) The problem motivation and threat model are insufficiently justified.
A. Unclear research significance. The introduction acknowledges that prior work, including Clawdrain and CDH, has demonstrated resource amplification using fixed or manually constructed malicious skills. The main contribution of this paper is therefore task-specific skill optimization. However, the paper does not directly compare the proposed method with these existing approaches. Without such comparisons, it is difficult to assess the significance of the proposed contribution.
B. Inconsistent threat model. The threat model assumes that attackers can modify the victim's skill documents and associated resources, including scripts executable in the victim's environment. Given this level of access, attackers could potentially introduce more directly harmful payloads, such as those designed for data exfiltration or system disruption. The paper does not adequately explain why attackers would prioritize resource amplification over these alternatives or what makes this attack particularly attractive.
Furthermore, Phase 2 requires attackers to repeatedly access the target agent's execution traces and refine their payloads accordingly. This appears to require additional access to the agent's inputs and execution behavior, exceeding the stated assumption that attackers control only a single skill document. The paper should clarify how these capabilities are obtained within the proposed threat model.
(2) The experimental evidence does not sufficiently support the main conclusions.
A. Limited experimental scope. The experiments evaluate only two models on each platform and use a dataset of just 50 skills. This limited coverage raises concerns about the generalizability and robustness of the results.
B. Unclear contribution of Phase 2. The ablation results show that Phase 1 alone achieves 4.02×–7.51× token amplification. Moreover, resampling without feedback accounts for 12%–43% of the gains from Phase 1 to Phase 2, whereas the independent contribution of feedback is only 3.7%–14.7%. These findings suggest that the LLM-guided iterative refinement emphasized in the paper is not the primary driver of performance improvements. Given that Phase 2 requires five additional rounds of task execution, its benefits relative to its computational cost remain insufficiently justified.
C. Insufficient defense evaluation. The paper evaluates only output truncation as a defense and demonstrates its limited effectiveness. Other potential defenses, such as monitoring tool invocation frequency, verifying skill integrity, and detecting anomalous execution trajectories, are not examined. Consequently, the authors' call for resource-aware defenses lacks sufficient experimental support.
Questions for authors
-
Related to Weakness 1. How can attackers implement the proposed attack in real-world scenarios without prior knowledge of the target agent's configuration or execution behavior? In particular, what access to execution traces is assumed during Phase 2?
-
Related to Weakness 2 . Given that repeated payload generation without feedback already produces substantial improvements, what additional benefits justify the complexity and computational cost of Phase 2?
-
Related to Weakness 2 . How effective are existing defense mechanisms against the proposed attack, particularly those based on skill verification, resource monitoring, and execution anomaly detection?
-
Related to Weakness 1 . How does the proposed attack compare with existing resource amplification attacks in terms of effectiveness, computational cost, and practicality?
7 comments
Dear Reviewer,
We hope you are well. With the discussion period drawing to a close, we wanted to check whether our response has answered the points you raised. We have tried to address the threat model directly: we clarify that Phase 2 uses only traces the attacker obtains by running the public model, with no access to the victim's execution, and we report transfer to an unseen model and an unseen frontend, together with the retention ratio on unseen inputs.
If any part of this still seems thin—or if there is a different experiment you would find more convincing—we would be very glad to hear it and to respond before the discussion closes.
Thank you for your careful reading and for the constructive feedback.
Best regards, The authors
Thank you for the authors' efforts in revising the paper and providing additional experiments. The rebuttal has addressed some of my concerns, but I still have the following reservations:
-
Limited Experimental Scope and Generalizability. The concern remains largely unresolved. Although the authors added Tables R1 and R2, these experiments do not substantially expand the evaluation scope. The evaluation still covers only two models per platform and 50 skills. Testing three unseen input variants provides limited evidence of robustness but does not establish generalizability across different tasks, skills, models, and agent configurations.
-
Insufficient Evaluation of Existing Defenses.The concern remains insufficiently addressed. The authors evaluate output-token caps and an LLM-based skill reviewer but do not examine other defenses explicitly raised in the original review, including skill integrity verification, tool invocation monitoring, and execution anomaly detection.
Response to VHLL
We thank the reviewer for the constructive comments.
W1A:
We refer to Table R1 (in the Overview), which compares SkillBloat directly with Clawdrain and CDH on the same 50 skill–task pairs, frontend, backend, workspaces and completion judge; the discussion follows the table. Under comparable or better task utility, SkillBloat leads both methods.
W1B:
Neither part of the concern holds: resource amplification is the more attractive objective precisely because it leaves no observable trace, and Phase 2 needs no access to the victim's execution traces — every run it uses is one the attacker executes on the public model with the public skill.
The Introduction states the scope: resource abuse is a distinct objective, not a substitute for data exfiltration or destructive modification. What makes it attractive is that it is more stealthy. The injected skill runs entirely inside the agent's ordinary task execution: nothing leaves the environment, no destructive or unauthorized operation is issued, and the skill keeps working, so the victim gets no signal that anything is wrong and the excess cost is billed silently. Such an attack is therefore harder to monitor than exfiltration or unauthorized operations, and our LLM reviewer experiment quantifies the gap: a fixed-prompt reviewer that knows nothing about our method labels 94.0% (188/200) of our optimized skills benign.
The loop does require repeated executions, but they are the attacker's own, and no access to the victim is involved. The attacker installs the target frontend and calls the same publicly available backend model with the public skill and task prompt, so every trace used in Phase 2 comes from a run that the attacker executed; the only artifact that reaches the victim is the skill document. What is carried back from each such run is limited to the diagnosed failure type, the amplification ratio, the task-completion status, and the agent's final response; the completion label comes from the judge in Section 3.1, which inspects the files that the same run produced in the attacker's own workspace---never the victim's---and their contents are not passed to the attacker's model. Nor do we assume that every victim's skill is tuned individually: the optimization is done once, by the attacker, on a public skill. The published skill remains aggressive on inputs and deployments it was never optimized for: frozen skills deployed verbatim on three unseen input variants per skill retain a mean retention ratio of 0.92 (Figure 3), and on other deployments they retain 5.1724× on a different backend model and 6.8344× on a different frontend (Table R8, in our response to Reviewer 1 (5jRm) W2).
W2B:
Please refer to the Overview (Table R2), which reports Phase 1, feedback-free resampling and Phase 2 with repeated runs per configuration; the response to Reviewer 1 (5jRm) W1 summarises the same result.
W2C:
We have already revised the paper accordingly, and the updated PDF reflects these changes. We refer to Section 5.1, which now evaluates the defenses: an output-token cap swept over 2×/4×/8× with a benign control under the identical cap, and a fixed-prompt LLM skill reviewer that labels 94.0% (188/200) of our optimized skills benign. Neither removes the amplification without breaking legitimate task completion, and neither can separate maliciously induced computation from legitimate work.
Q1:
Please refer to our response to W1B above: Phase 2 assumes no access to the victim; on unseen inputs of the same task, a frozen skill retains a mean retention ratio of 0.92 (Figure 3), and when the target configuration is unknown it still retains 5.1724× on a different backend model and 6.8344× on a different frontend (Table R8, in our response to Reviewer 1 (5jRm) W2).
Q2:
Please refer to the Overview (Table R2), which reports Phase 1, feedback-free resampling and Phase 2 with repeated runs per configuration; the response to Reviewer 1 (5jRm) W1 summarises the same result.
Q3:
Please refer to our response to W2C above. No existing defense evaluated here separates malicious from benign skill use.
Q4:
We refer to Table R1 (in the Overview), which compares the three methods on effectiveness, task completion and practicality.