- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper studies weight-based backdoors in local tool-using LLM agents. The proposed backdoor activates after the model generates a predefined sequence of shell commands, without requiring an attacker-controlled inference-time input. The authors poison fine-tuning trajectories, distribute the resulting model as a GGUF artifact, and evaluate it through an Ollama–Codex CLI stack. They separately measure whether the model generates the malicious tool call and whether the call is executed to create a file on the host. Experiments cover three Qwen-family instruction models, several quantization levels, and character-level models trained from scratch.
Critical strengths
-
Practical end-to-end validation through a real agent stack - The paper goes beyond evaluating whether a model generates a target string. It follows the poisoned artifact through model merging, GGUF conversion, quantization, Ollama serving, Codex CLI parsing, tool dispatch, execution policy, and finally a concrete change to the host workspace. This makes the demonstrated system-level risk more concrete than an evaluation based exclusively on offline generation.
-
Clear presentation and useful observations - The paper is exceptionally clear and easy to follow. The threat model, attack pipeline, trigger-selection procedure, and distinction between probe and deployment measurements are explained carefully.
Critical weaknesses
-
Limited novelty - The paper does not introduce a new poisoning method, optimization objective, compromised model component, or attack surface. The end-to-end demonstration is useful for validation, but once the model emits a valid tool call and the agent is configured to execute permitted commands without approval, the resulting host action is expected. The contribution is therefore closer to a practical demonstration of an established attack class than a sufficiently novel ICLR contribution.
-
The evaluation does not establish generalization or dormancy - I could not determine whether the trigger generalizes to unseen tasks and repositories, or merely reappears in the workflow from which the attack is constructed. An improvement would be to select the trigger and train the backdoor on one task, then evaluate it on a disjoint collection of coding tasks and projects. The dormancy evidence is also incomplete. The paper reports false activation, but does not compare clean and poisoned models on normal task performance. Some configurations have misfire rates of 8.77% and 12.24%, which seem high for a backdoor described as dormant. It is also unclear whether poisoning makes the model more likely to generate the trigger sequence itself. The empirical evidence is limited. The main attack is tested only on Qwen-family instruction models, while the nanoGPT experiments use a conventional input trigger and therefore do not validate the paper’s central idea across architectures. Several key results also rely on small samples; the headline 60% deployment result is 9/15 and are reported without confidence intervals or repeated fine-tuning seeds.
Questions for authors
-
Novelty: The paper presents the model-generated action sequence as its main distinction from prior backdoors. However, the paper’s own related work includes input-independent triggers based on dialogue turn index [1]. MetaBackdoor [2] further demonstrates self-activation during normal multi-turn interaction, including malicious tool-call behavior without attacker-supplied trigger text. Given these precedents, what additional attack capability is enabled by conditioning specifically on the agent’s action history? For example, does it allow activation at a meaningful task state, improve selectivity or stealth, or bypass defenses that these earlier triggers cannot? Clarifying this distinction, ideally through an empirical comparison, would materially affect my assessment of the paper’s novelty.
-
Task and repository separation: Were the deployed evaluations conducted on user tasks and repositories that were fully disjoint from those used to collect trajectories, select the trigger, construct the poisoned data, and choose the checkpoint? If held-out tasks were used, please describe them.
-
False activation: How do the authors interpret deployment misfire rates of 8.77% and 12.24% as consistent with a dormant or stealthy backdoor?
-
Deployment configuration and success accounting: The paper describes the stack as operating at its defaults, while the appendix reports explicit sandbox and approval settings. Please clarify which settings are defaults and which were chosen for the experiment.
[1] Turn-Based Structural Triggers: Structure-Conditioned Backdoors in Multi-Turn LLMs
[2] MetaBackdoor: Exploiting positional encoding as a backdoor attack surface in LLMs