- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
SkillDOS is a token-amplification (“denial of wallet”) attack on skill-enabled agents. A poisoned skill splits into an Inducer (gets the workflow adopted) and an Amplifier (scales bounded, task-relevant work), optimized in two stages under an answer-preservation constraint. Across 100 skills, three frameworks, and four models, it reports mean token-amplification ratios of 2.65–13.96× at 95–100% task preservation.
Critical strengths
- Genuinely new attack dimension. Prior skill-security work targets payload execution or exfiltration; resource amplification that preserves the answer while inflating cost is a distinct and under-studied surface. The denial-of-wallet framing is timely for usage-metered agents and well-motivated.
- Clean formalization. The TAR ratio, the baseline-relative preservation judge, and the one-way two-stage optimizer (freeze Inducer once activation is verified, then search Amplifier genomes) are precisely specified. The activation-vs-preservation separation is a sensible design that avoids conflating adoption with amplification.
- Reasonable breadth and controls. Evaluation spans three agent frameworks and four backends with a paired sandboxed baseline, plus a SkillJect comparison, cross-model transfer, a stagewise ablation, and a scanner-evasion test. The ablation cleanly attributes most gain to Stage 1.
Critical weaknesses
- Means are driven by outliers. Medians are modest (2.05–5.88×) while means reach 13.96× and a single case hits 558×. Leading with means materially overstates typical amplification; a 2–3× median is a far less alarming result than the abstract implies.
- Single-baseline noise. The clean baseline is run once per case (acknowledged), so model stochasticity is uncontrolled. Given that small TARs (~2×) sit near plausible run-to-run variance, part of the reported amplification may be noise rather than attack effect.
- Weak evasion undercuts the threat. Existing scanners already detect 73.3% of poisoned skills, so only ~27% evade. Combined with an attack-only contribution and no proposed defense, the “practical risk” claim is softer than stated.
- Circular preservation metric. Preservation is an LLM judge that ignores length and verbosity — but inflated length is the attack. A 10× longer “equivalent” answer counts as fully preserved, so the metric cannot distinguish harmful padding from legitimately added value.
Questions for authors
see weakness