- Rating
- 4
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper identifies a critical supply-chain vulnerability in AI agent extension ecosystems: "model-bearing" agent skills that package small, local AI classifiers. The authors introduce BADSKILL, a backdoor attack where trigger conditions are defined as schema-valid parameter conjunctions (e.g., specific combinations of formatting options). While the conditional execution logic is visible in the Python source code, the activation boundary determining when the check evaluates to True is encoded entirely within the binary model's mathematical weights. Using a two-stage training pipeline with "hard negatives", BADSKILL achieves a 97.2% mean Attack Success Rate and a 97.5% benign result return rate across 32 model–runtime settings (8 model backbones, 4 agent runtimes). Existing text-side detectors fail to catch the attack (12.4%–36.8% detection rate), underscoring the need for holistic supply-chain vetting.
Critical strengths
Novel Threat Model: Exposes a fundamental blind spot at the software–model boundary by delegating conditional route activation to binary model weights rather than source code text.
Hard-Negative Training: Synthesizes m-1 parameter matches during optimization to enforce a sharp activation boundary, preventing accidental triggers during routine use.
Dual-Channel Stealth: Preserves normal task execution outputs in both triggered and non-triggered branches, concealing side-effects during automated dry-runs.
Broad Empirical Scope: Comprehensive benchmark across 32 model-runtime configurations, 8 backbones, 4 frameworks, and 5 defense baselines.
Critical weaknesses
Schema-Parsing Assumption: Assumes inputs are cleanly parsed into structured key-value pairs. Suggestion: Evaluate activation robustness under noisy, unparsed free-text inputs.
Sensitivity to Typo Corruption: Character-level perturbations lower attack activation (e.g., ASR drops to 66.7%). Suggestion: Incorporate adversarial typo augmentation during Stage I classifier training.
Probing Scope Limitations: Counterfactual probing requires a candidate parameter template T. Suggestion: Discuss automated static analysis heuristics to systematically extract candidate templates from skill manifests.
Questions for authors
How well does hard-negative generation scale for skills with large parameter schemas or continuous numerical ranges?
Thank you for the positive assessment. We address the four points below.
1. Structured input does not imply clean incoming text.
BADSKILL evaluates parsed parameters, but Table 3 applies typos, word swaps, character deletion, and insertion before parsing. The reported robustness therefore already includes noisy text passing through the parser and execution path.
The experiments retain source labels, so reduced ASR can reflect parsing failure, a changed trigger assignment, or classifier sensitivity. They measure end-to-end sensitivity, not invariance conditional on an unchanged structured trigger. Text that cannot be mapped to the interface has no valid structured invocation under the current formulation; supporting it directly would require extending that interface. We will make this distinction explicit.
2. Typo augmentation must preserve the trigger's labeling rule.
General typo injection is already included in Stage I (Appendix C). Table 3 nevertheless shows a five-task mean ASR of 98.9% on clean inputs, 88.6% under 5% typo corruption, and 80.7% under 10%. The 66.7% result is Note Cleanup under 10% corruption, showing that sensitivity is task-dependent.
Targeted adversarial augmentation is a useful extension, but making all corrupted trigger examples positive would blur the boundary the classifier is meant to learn. A typo preserving the parsed conjunction should retain its positive label; a valid change breaking it should receive a negative label. This separates surface robustness from tolerance of different parameter assignments. This proposal should be assessed using ASR and false activation jointly.
3. Candidate discovery and behavioral verification are distinct computations.
The current Counterfactual Probe verifies a supplied candidate through zero-match, single-match, leave-one-out, and full-trigger variants. Its
2m + 2execution bound applies per invocation and candidate template—at most eight for native triggers (Appendix D). It does not include searching for that candidate.Static analysis can provide a proposed front end: trace fields reaching the classifier, collect values from schema enumerations, defaults, parser literals, and examples, then prioritize small, schema-valid combinations. The schema specifies legal assignments; it does not specify which conjunction the weights have learned. Consequently, this strategy can narrow the search but is not equivalent to recovering the trigger. We will distinguish this proposed extraction step from the evaluated probe.
4. Hard negatives sample incomplete conjunctions rather than enumerate the schema.
For a trigger of arity
m, the generator retainsm − 1assignments and gives the omitted field a valid value that violates its trigger assignment. One replacement per field produces at mostmvariants per source invocation. Wider schemas affect validation, serialization, and inference costs, but do not require enumerating the Cartesian product of every field's values.Our additional evaluation crosses four schema widths and four trigger arities:
Of 4,000 generation attempts, 3,752 yield valid hard negatives (93.8%); 48 activate the canary, giving HN-FAR = 48/3,752 = 1.28%. Invalid candidates are excluded from this denominator. The positive cohort gives ASR = 3,832/4,000 = 95.8%. Arities 4 and 6 extend beyond the native two- and three-assignment triggers. These results establish feasibility, not runtime scaling. Original held-out hard negatives remain separate at 3/16 = 18.75%; the cohorts are not a matched comparison.
For an exact normalized numeric trigger
v*, the construction can select a legal normalizedv != v*while retaining other assignments. Nearby and domain-wide samples would test local sensitivity and broader coverage. Interval triggers require interval-aware labels and boundary samples. These are proposed extensions, not empirical coverage of continuous domains.