acceptodds
Under review as a conference paper at ICLR 2027

Can Useful Work Be the Attack? CUCKOO SKILLS: PARASITIC TOKEN-COST ATTACKS ON LLM AGENTS

Abstract

Reusable skills guide LLM agents through workflows, but can introduce resource-use risks that final output quality alone may not reveal. Existing token-cost attacks induce excess steps, but agents may skip costly instructions, and aggregate token counts alone cannot distinguish amplification across instances from isolated expensive failures. We revisit skill-guided execution and observe that intermediate preparation can substantially increase computation while producing artifacts compatible with task completion. Based on this observation, we propose CUCKOO SKILLS, a token-cost attack framework that couples feedback-guided skill construction with constrained candidate selection. Autonomy-Preserving Structured Construction (APSC) uses routing feedback to guide skill adoption and execution feedback to refine optional work and its outputs for subsequent task steps. Budget-Exhaustive Risk-Sensitive Selection (BERS) uses paired executions to assess task utility and cross-instance amplification, prioritizing qualified candidates and ranking them by aggregate cost. Across ARC-Challenge, LegalBench, and -bench with two backbone models, CUCKOO SKILLS consumes 3.34–94.50 as many completion tokens as clean execution, with amplification ratios 2.57–6.90 times those of the strongest evaluated baseline in each setting. Across all three datasets, reasoning accounts for 66.08–99.01% of completion tokens, and substantial amplification coexists with useful task outputs. These findings highlight resource risks that evaluations based only on final-answer quality may overlook.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Token-Cost Attacks, LLM Agent Security, Agent Skill

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Kt2nOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper tudies resource amplification through third-party skills that preserve the host agent’s ability to skip work or finalize. Across three benchmarks and two models, reported completion-token amplification ranges from 3.34× to 94.50×. Additional analyses examine per-instance increases, reasoning-token attribution, defense overhead, and native OpenClaw deployment. A statistical argument bounds false median-amplification qualification for frozen finalists under conditional i.i.d. assumptions.

Critical strengths

The paper demonstrates a practically relevant failure mode. Final-answer accuracy can conceal substantial resource abuse.

The instance-level analysis strengthens the aggregate result. Positive completion-token increments occur on 91.7–100% of evaluated instances, with all interquartile ranges above zero and median increments around 6.2k–23.7k tokens.

The protocol preserves an explicit finalization option. Work is optional, repeated work-unit identifiers are disallowed, and the controller can finalize immediately.

Critical weaknesses

The empirical qualification status is not sufficiently connected to the headline results. The algorithm always returning a skill does not mean it found a skill satisfying the stated constraints.

The median-amplification guarantee is much weaker than the headline ratios suggest.

The closest baseline comparisons are limited and not demonstrably budget-matched. Those adaptations make consequential choices about workflow structure, finite retries, and metadata selection. Meanwhile, CUCKOO SKILLS receives a multi-generation feedback-guided search. The manuscript does not provide a complete matched search-budget comparison. The shared-catalogue controls are useful evidence for the search design, but do not establish superiority over faithfully tuned versions of the closest prior systems across all domains. The claims should distinguish broad empirical comparisons from tightly controlled mechanism comparisons.

Questions for authors

Can intermediate artifacts be independently verified and shown to contribute useful information? Do the gains persist against matched benign workflows and closely related attacks under equal search budgets?

Reviewer_v3jhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies resource-amplification attacks delivered through agent skills. CUCKOO SKILLS combines feedback-guided construction through APSC with paired evaluation and constrained selection through BERS. The procedure checks task utility, skill entry, verified work, and amplification, and separates fresh-bank qualification from independent testing. Experiments cover ARC-Challenge, LegalBench, and τ-bench with two backbones, alongside search ablations, defense evaluations, and native OpenClaw deployment on WebArena. The results show substantial completion-token amplification, with varying utility changes across settings. The procedure can also return an unqualified fallback, making result-level qualification information important for interpreting the constrained-selection claim.

Critical strengths

S1. The paper provides substantial empirical evidence for a relevant threat.

The evidence extends beyond headline CAR values: token-type decomposition, monetary-cost estimates, and paired per-instance analyses show that cost increments occur on most evaluated instances. The native OpenClaw/WebArena experiment is particularly useful, reporting 2.92× completion-token amplification while leaving skill activation to the host. These results demonstrate meaningful breadth beyond one controlled workflow. Although utility is not unchanged in every setting, the results establish that substantial additional computation can coexist with successful task completion.

S2. The construction and selection design is supported by useful controlled comparisons.

Separating construction, fresh-bank qualification, and independent testing is well motivated. The paper states the assumptions and limited scope of its statistical guarantee and distinguishes qualified outputs from fallbacks. The random, greedy, and no-cross-instance-qualification search controls share the initial skill, operator space, splits, generation limit, and target runtime limits. Further ablations examine feedback, construction, and ranking choices. On ARC-Challenge/GPT-5-mini, Figure 10 also reports greater amplification with fewer calls than the two skill-construction ports. These comparisons provide evidence for the method's components; the public fixed edit catalogue and numerical example configuration also aid inspection.

Critical weaknesses

W1. The main results are not sufficiently connected to the constrained-selection claim.

Section 4.3 and Appendix D.1 permit an unqualified fallback when no finalist passes all gates. The main result tables do not identify which outputs were qualified versus fallbacks or summarize their gate outcomes, although the implementation records these quantities. A result-level summary is needed to connect the reported artifacts to the qualification procedure. This matters to the constrained-selection contribution beyond the substantial measured cost increases.

Q1 — Clarification of existing results: For the main model–benchmark runs, please report qualified/fallback status and the gate outcomes under the actual thresholds, new-error allowances, and common-bank sizes. Which reported results support the claim that the selected skills satisfy the stated constraints?

As a secondary clarification of the comparison, Table 1 reports mean CAR over three seeds, whereas Appendix C.1/Figure 7 implies a different relative advantage in aggregate output tokens. Q2 — Clarification of aggregation: Please explain how the two summaries correspond, using per-seed attacked and clean totals and the associated aggregation rules. The existing per-instance cost distributions already provide useful evidence about the breadth of the effect.

W2. The broader comparative advantage needs a more direct estimate of the incremental effect of malicious skill content.

In the inspected public runtime, tasks without environment hooks can follow a direct clean-answer path, while selected skills use routing and an adaptive controller. A no-skill baseline measures total installation-induced overhead; a benign task-relevant skill under the same host protocol would help distinguish ordinary skill adoption from malicious content. The relationship between this public runtime and the main experimental configurations also needs clarification.

The paper already provides substantial attribution evidence: Appendix B.2 acknowledges decoding as an existing primitive, Table 1 includes ExtendAttack, and shared-catalogue ablations support a contribution from search. On ARC-Challenge/GPT-5-mini, Figure 10 reports greater amplification with fewer calls than the skill-construction ports, while the worked example attributes 95.69% of completion cost to the work call. The remaining issue is the marginal effect under a matched host protocol and clearly specified resources.

Q3 — Protocol clarification and additional validation: Please identify the runtime configuration and actual construction/execution limits used in the main comparisons. A benign task-relevant skill control under the same host protocol would directly strengthen the attribution; alternatively, delimit the claim to total overhead relative to no-skill operation and the search effects already supported by the controlled ablations.

Overall assessment. This is a borderline weak rejection. The measured cost increases, controlled search ablations, and native deployment provide substantial positive evidence. I give greater weight to the missing result-level qualification information and the remaining attribution gap in W2. Clear evidence connecting the reported outputs to the selection gates and clarifying the incremental comparison could change my recommendation. The aggregation clarification is secondary and does not independently justify rejection.

Questions for authors

See weaknesses

Reviewer_A7PhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies token-cost amplification through reusable agent skills while retaining task performance. CUCKOO SKILLS combines routing-aware, feedback-guided skill construction with candidate selection using paired cost, execution, and utility checks. Experiments cover three benchmarks and two backbones, with additional defense tests and native OpenClaw deployment on WebArena.

Critical strengths

S1: Resource amplification is measured alongside task outcomes and native deployment. Table 1 measures completion-token amplification alongside task accuracy across six settings. The large increases on ARC-Challenge and LegalBench coexist with accuracy close to or above the clean reference, showing why final-answer evaluation alone can miss resource abuse. Section 5.5 evaluates an installed skill in OpenClaw on WebArena. Figure 6 measures prompt tokens, tool calls, and agent time as well as completion tokens, connecting the study to costs beyond a single reasoning response.

S2: Selection separates cross-instance qualification from aggregate cost. Section 4.3 distinguishes evidence of amplification across instances from aggregate cost maximization, then re-evaluates frozen finalists on a fresh bank. This addresses the concrete problem of selecting a candidate because of a few expensive episodes.

Critical weaknesses

W1: The additional contribution over existing workload and skill attacks remains insufficiently isolated. Sections 3 and 4.3 distinguish task utility from verified work. The unresolved issue is how much amplification Section 4.2's work-handoff construction adds beyond a simpler optional decoding skill. Table 1 changes both delivery and workload relative to ExtendAttack. Compare these skills with identical routing, stopping rules, candidate selection, and construction budgets to isolate the construction's incremental effect on amplification. Section 2 identifies CDH and Clawdrain, which already combine skill-carried resource amplification with task completion. However, Table 1 compares prompt/token-cost attacks, while Section 5.5 compares Breaking Agents and CostBomb. The main text needs matched skill-based comparisons under the same host, tool access, search budget, and stopping rules to establish the additional contribution of the proposed construction and selection framework.

W2: The utility-cost tradeoff needs explicit tolerances and paired accounting. Table 1 reports tau-bench accuracy falling from 43.75% to 38.54% for GPT-5-mini and from 60.94% to 58.85% for DeepSeek-Flash. Section 5.2 acknowledges lower success, and Equation 2 allows utility loss; these drops alone do not show a gate violation. The main text does not give run-specific utility-loss and new-error limits. Report those limits, paired utility uncertainty, and a joint success/cost breakdown, including both-successful pairs, to locate overhead on utility-preserving executions.

Questions for authors
  1. How much amplification does an optional decoding-only skill achieve under the same routing, stopping rules, candidate selection, and construction budget, and how much additional amplification comes from the proposed work-handoff construction?

  2. What run-specific utility-loss and new-error limits were used, and what is the joint breakdown of paired success, utility differences, and token costs on tau-bench?

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to Reviewer Kt2n: Discussion of the Novelty of CUCKOO SKILLS

Dear Reviewer Kt2n,

Thank you for encouraging us to articulate our contribution more sharply.

Our technical novelty lies in jointly designing the attack search space and the qualification rule. APSC makes optional task-directed work and its handoff requirements searchable, while BERS makes cross-instance amplification an eligibility condition before aggregate-cost maximization.

1. Motivation: Framing task-compatible resource abuse as a constrained construction problem.

Our motivating insight is that task-relevant intermediate work can itself become a vehicle for resource abuse, even when the host remains free to skip that work or finalize immediately. We formulate this as a constrained construction problem: optimize a reusable skill's proposed work and deliverables while measuring adoption, verified work, and task utility. This connects resource amplification to the design of useful workflows and to measurable conditions for their execution.

2. Challenges: A coupled feasibility problem under host autonomy.

The distinctive challenge lies in the interaction among workload size, host choice, artifact usability, and amplification coverage. Increasing the workload may encourage the host to skip optional work, reducing realized cost. Executed work may fail to yield a usable handoff, while a high aggregate CAR can arise from only a few costly tasks. Construction must therefore coordinate execution triggers, computation, and deliverables, while selection separately assesses cross-instance amplification. APSC and BERS address these dependencies through the joint design of the admissible candidate space and the acceptance rule.

3. Solution: A distinct search space paired with a cross-instance qualification gate.

APSC optimizes the coupled trigger, computation, and handoff contract under a finalization-preserving constraint. CDH uses feedback to refine routing and activation descriptions around a predefined body-construction protocol. APSC applies typed catalogue edits to iteratively revise optional work and its task-artifact requirements. Clawdrain and Beyond Max Tokens organize amplification around verification protocols and protocol-gated payload release. APSC searches for costly work within a space that preserves immediate finalization and excludes mandatory work sequences or minimum call counts. These differences change both the optimization variables and the admissible candidate space.

BERS separates paired cross-instance cost-ratio qualification from aggregate-cost ranking. OTora incorporates a stability penalty into its optimization score. BERS uses a calibrated lower-order statistic of paired, smoothed log-cost ratios as a separate qualification gate, alongside utility, skill-entry, and verified-work thresholds. Higher CAR cannot make a candidate that fails this gate qualified. Among qualified candidates, CAR determines the ranking, and frozen finalists undergo final evaluation on a fresh common bank. This separates evidence of cross-instance amplification from the objective of maximizing aggregate cost.

The coupling is operational: gate outcomes guide parent retention and inform subsequent work edits. Qualification shapes the skills produced during search.

The table compares capabilities explicitly specified in each work. Agent-level attack targets multi-step agent execution involving tools or environmental interaction. Utility-aware optimization incorporates task performance or answer fidelity into automated scoring or admission. Statistical guarantees concern selection-error control under stated assumptions.

MethodAgent-level attackSkill-based deliveryAutomated optimizationUtility-aware optimizationStatistical guarantees
ExtendAttackNONONONONO
OverThinkNONOYESYESNO
CostBombYESNOYESNONO
Beyond Max TokensYESNOYESYESNO
OToraYESNOYESYESNO
ClawdrainYESYESNONONO
CDHYESYESYESNONO
CUCKOO SKILLSYESYESYESYESYES

Together, these advances establish a concrete optimization framework: search task-directed work and its deliverables under autonomy constraints, qualify amplification across tasks, and maximize cost among qualified candidates.

We would appreciate your reconsideration of our work's novelty in light of these methodological distinctions.

AuthorsOpenReview ↗

General Response to All Reviewers and AC

We thank all reviewers and the AC again for the valuable comments and constructive suggestions. With the discussion period drawing to a close, we would like to follow up and ensure that we have addressed the concerns raised in the individual reviews. We are pleased that the reviewers recognized the threat as practically relevant and important (A7Ph, Kt2n, v3jh), the empirical evidence as substantial and broad (A7Ph, Kt2n, v3jh), and the separation of cross-instance qualification from aggregate cost maximization as valuable and well motivated (A7Ph, v3jh). We summarize five sets of additional results and clarifications: (1) matched-budget comparisons with closely related attacks, benign skills, and decoding-only controls, (2) complete qualification results for all 18 main-table outputs and clarified statistical guarantees, (3) independent validation of intermediate artifact correctness and downstream usefulness, (4) paired utility–cost analyses with confidence intervals and success-conditioned amplification, and (5) corrected three-seed token aggregation and improved manuscript presentation.

Rebuttal Process

The table summarizes the additional experiments and clarifications, grouping overlapping discussions. Check marks indicate the reviewers who raised each discussion.

DiscussionA7PhKt2nv3jhLocation in revisionMain result or clarification
Incremental contribution beyond decoding-only skills✓✓✓Appendix C.4, Table 6CUCKOO SKILLS achieves 4.80× and 2.53× the completion tokens of decoding-only skills on ARC-Challenge and τ-bench.
Matched comparisons with related skill-based attacks✓✓✓Appendices B.2, B.4, C.4; Table 6Under matched protocols and construction budgets, CUCKOO SKILLS achieves 2.76× and 2.30× the amplification of Clawdrain on the two benchmarks.
Benign-skill controls and runtime attribution✓✓✓Appendix B.4; Appendix C.4, Table 6Benign skills remain near clean cost (0.98×/1.02×), while CUCKOO SKILLS reaches 68.37×/3.24× under matched settings.
Qualification status and selection constraints✓✓✓Appendix D.2, Tables 11–12All 18 main-table outputs pass all four qualification gates, with no unqualified fallbacks. Run-level thresholds and results are reported.
Statistical guarantee and cross-instance amplification✓Appendix D.3Clarified the calibrated median-amplification guarantee, its conditional i.i.d. assumptions, and its distinction from empirical CAR.
Utility–cost trade-offs and paired task outcomes✓Appendix C.7, Tables 8–9Added paired confidence intervals and joint outcome analysis. Amplification remains 3.43×/3.54× on both-success task pairs.
Independent artifact correctness and downstream usefulness✓Appendix C.8, Table 10Artifact correctness reaches 95.54%/91.67%; artifact access improves downstream success by 4.69/7.81 percentage points.
Search effectiveness and component attribution✓✓✓Section 5.3, Figure 4; Appendix C.5, Figure 11At generation 12, cross-instance qualification improves CAR by 40.5%/17.3% on ARC-Challenge/LegalBench; component ablations further support the full design.
Three-seed aggregation and token accounting✓Appendix C.1, Figure 7Corrected Output aggregation to use the same three runs as Table 1; CUCKOO SKILLS retains the highest average Output and mean CAR in all six settings.
Manuscript readability and figure presentation✓Pages 1–4, Figures 1–3Improved the manuscript layout and enlarged figure text for readability.
AuthorsOpenReview ↗

Final Takeaways: Core Contributions and Strengthened Evidence

Motivation and Core Contributions

The motivation and contributions of CUCKOO SKILLS center on three points:

  1. Task-compatible resource amplification as an overlooked security threat. Existing evaluations often focus on final-answer correctness, while malicious skills can substantially increase computation without preventing task completion. We study this threat under native skill activation, where agents retain control over skill adoption, optional work, and finalization. Across three benchmarks and two backbones, CUCKOO SKILLS achieves 3.34–94.50× the completion tokens of clean execution, outperforming the strongest evaluated baselines by 2.57–6.90×.

  2. Autonomy-Preserving Structured Construction (APSC). Rather than simply instructing agents to perform unnecessary work, APSC coordinates skill adoption, costly intermediate computation, and usable artifact handoff through feedback-guided construction. It jointly refines work triggers, instructions, and artifact contracts while preserving the agent's ability to skip work or finalize immediately. Matched comparisons show 4.80× and 2.53× higher token consumption than decoding-only skills, while independent validation confirms that the intermediate artifacts provide useful information for downstream tasks.

  3. Budget-Exhaustive Risk-Sensitive Selection (BERS). Aggregate token cost can be dominated by isolated expensive executions. BERS addresses this problem by separating cross-instance amplification qualification from aggregate cost maximization. It evaluates candidates through paired executions, enforces task-utility and execution constraints, and uses a calibrated lower order statistic to qualify amplification before ranking candidates by CAR. All 18 reported main-table outputs satisfy the qualification requirements, and component ablations demonstrate the benefits of feedback-guided construction and risk-sensitive selection.

Closing Remarks

Together, these additional results strengthen the evidence for CUCKOO SKILLS along three central dimensions: methodological contribution, through matched comparisons isolating work-handoff construction; technical soundness, through complete qualification reporting and clarified statistical guarantees; and practical significance, through independent artifact validation, paired utility–cost analysis, and evaluation under matched execution conditions.

Importantly, the revised evidence reinforces our central finding: substantial resource amplification can coexist with useful intermediate work and successful task completion, making resource abuse difficult to identify from final-answer quality alone.

We thank the reviewers and the AC again for their valuable feedback. We hope these additional results and clarifications address the remaining concerns, and we would be grateful if they could be taken into account in the final ratings and overall assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.