acceptodds
Withdrawn from ICLR 2027

From Weights to Actions: Model-Only Backdoors in Tool-Using Local LLM Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Downloading open model weights and running them under a coding agent is now routine, and it hands the model the ability to edit files and run shell commands on the user's machine. The agent acts on a tool call because the call is well formed, not because the model that produced it can be trusted, so a compromised model can drive an otherwise unmodified installation into actions of the attacker's choosing. We ask whether a backdoor planted in the weights alone carries through a real agent stack to an action on the victim's host. Our adversary poisons a fraction of the training or fine-tuning data, publishes the model through ordinary channels, and never touches the victim's machine or sees what they ask the agent to do. The backdoor fires on a condition fixed at training time and is otherwise dormant. We study two: a rare string in the input, the usual choice, and a state the model reaches on its own during ordinary work, a short sequence of commands it issues while doing the task, which needs no channel to the victim at all. The poisoned weights are converted to GGUF, served by Ollama, and driven by an unmodified Codex CLI agent at its defaults. We measure two things prior work collapses into one: whether the model emits the malicious call, and whether a file actually lands on the host once the call has passed the agent's parser, dispatcher, and execution policy. We report this across from-scratch models spanning seeds and poisoning rates and across fine-tuned instruction models, and show that in deployed runs the agent executes the attacker's command in the middle of the user's own task. A downloaded model file is therefore not passive data. Local weights deserve the scrutiny given to untrusted code, and the boundary between a model's tool calls and the actions they become is where any independent check must sit.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Tool-using LLM agents, self-triggered backdoors, data poisoning, model supply chain security

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_aoAnOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies weight-based backdoors in local tool-using LLM agents. The proposed backdoor activates after the model generates a predefined sequence of shell commands, without requiring an attacker-controlled inference-time input. The authors poison fine-tuning trajectories, distribute the resulting model as a GGUF artifact, and evaluate it through an Ollama–Codex CLI stack. They separately measure whether the model generates the malicious tool call and whether the call is executed to create a file on the host. Experiments cover three Qwen-family instruction models, several quantization levels, and character-level models trained from scratch.

Critical strengths
  1. Practical end-to-end validation through a real agent stack - The paper goes beyond evaluating whether a model generates a target string. It follows the poisoned artifact through model merging, GGUF conversion, quantization, Ollama serving, Codex CLI parsing, tool dispatch, execution policy, and finally a concrete change to the host workspace. This makes the demonstrated system-level risk more concrete than an evaluation based exclusively on offline generation.

  2. Clear presentation and useful observations - The paper is exceptionally clear and easy to follow. The threat model, attack pipeline, trigger-selection procedure, and distinction between probe and deployment measurements are explained carefully.

Critical weaknesses
  1. Limited novelty - The paper does not introduce a new poisoning method, optimization objective, compromised model component, or attack surface. The end-to-end demonstration is useful for validation, but once the model emits a valid tool call and the agent is configured to execute permitted commands without approval, the resulting host action is expected. The contribution is therefore closer to a practical demonstration of an established attack class than a sufficiently novel ICLR contribution.

  2. The evaluation does not establish generalization or dormancy - I could not determine whether the trigger generalizes to unseen tasks and repositories, or merely reappears in the workflow from which the attack is constructed. An improvement would be to select the trigger and train the backdoor on one task, then evaluate it on a disjoint collection of coding tasks and projects. The dormancy evidence is also incomplete. The paper reports false activation, but does not compare clean and poisoned models on normal task performance. Some configurations have misfire rates of 8.77% and 12.24%, which seem high for a backdoor described as dormant. It is also unclear whether poisoning makes the model more likely to generate the trigger sequence itself. The empirical evidence is limited. The main attack is tested only on Qwen-family instruction models, while the nanoGPT experiments use a conventional input trigger and therefore do not validate the paper’s central idea across architectures. Several key results also rely on small samples; the headline 60% deployment result is 9/15 and are reported without confidence intervals or repeated fine-tuning seeds.

Questions for authors
  1. Novelty: The paper presents the model-generated action sequence as its main distinction from prior backdoors. However, the paper’s own related work includes input-independent triggers based on dialogue turn index [1]. MetaBackdoor [2] further demonstrates self-activation during normal multi-turn interaction, including malicious tool-call behavior without attacker-supplied trigger text. Given these precedents, what additional attack capability is enabled by conditioning specifically on the agent’s action history? For example, does it allow activation at a meaningful task state, improve selectivity or stealth, or bypass defenses that these earlier triggers cannot? Clarifying this distinction, ideally through an empirical comparison, would materially affect my assessment of the paper’s novelty.

  2. Task and repository separation: Were the deployed evaluations conducted on user tasks and repositories that were fully disjoint from those used to collect trajectories, select the trigger, construct the poisoned data, and choose the checkpoint? If held-out tasks were used, please describe them.

  3. False activation: How do the authors interpret deployment misfire rates of 8.77% and 12.24% as consistent with a dormant or stealthy backdoor?

  4. Deployment configuration and success accounting: The paper describes the stack as operating at its defaults, while the appendix reports explicit sandbox and approval settings. Please clarify which settings are defaults and which were chosen for the experiment.

[1] Turn-Based Structural Triggers: Structure-Conditioned Backdoors in Multi-Turn LLMs

[2] MetaBackdoor: Exploiting positional encoding as a backdoor attack surface in LLMs

Reviewer_GoVNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates model-weight backdoors in tool-using local LLM agents. By poisoning training or fine-tuning data, an attacker causes the model to insert a malicious tool call after generating a particular command sequence during normal task execution. The attack requires no attacker-controlled trigger in the user’s input or environment at inference time.

The authors convert the poisoned models to GGUF and deploy them through unmodified Ollama and Codex CLI, separately measuring malicious-call generation and actual file creation. The results show that some backdoors propagate through a real agent stack and produce host-side effects, although effectiveness varies substantially across models, quantization levels, and training configurations.

Critical strengths

The paper addresses a practically relevant security risk.It examines downloaded model weights as an attack entry point and clearly explains the risks that arise when compromised models gain access to tools.

The evaluation distinguishes malicious-call generation from actual execution.Rather than judging success from generated text alone, the authors verify whether a call passes through parsing, dispatch, and execution to affect the file system. This is the paper’s strongest contribution.

The trigger mechanism is interesting.The backdoor activates through command sequences generated by the model during normal work, reducing dependence on an attacker-controlled input channel at inference time.

Critical weaknesses

Evidence of generalization across tasks and projects is limited. Section 4.1 primarily collects trajectories using a single project-analysis instruction, but the main text does not clearly explain how deployment tests are separated from training data by task, project, and context. It is therefore unclear whether the attack depends on this particular workflow.

Preservation of normal task performance is insufficiently evaluated. A selective backdoor should retain normal behavior, yet the evaluation mainly reports activation and false-activation rates. The example in Table 4, where the model continues answering the user, does not establish that task quality or instruction-following ability remains intact.

The main success rates are conditional metrics rather than session-level attack success rates. The fire rate in Table 3 is conditioned on the trigger having occurred. Without session counts, trigger coverage, and session-level exact-payload success rates, practical attack effectiveness is difficult to assess.

Exact-payload execution is weak in several configurations. The conclusions should more clearly distinguish unauthorized file creation from accurate execution of the attacker’s intended payload.

Checkpoint selection and independent testing are not clearly separated. The authors select promising checkpoints using probe results before deployment evaluation, but it is unclear whether selection and final testing use independent data. Some configurations also have few observed trigger occurrences and lack uncertainty estimates, limiting confidence in their stability.

Some experimental settings sit uneasily with the description of a rare, stealthy trigger. Table 2 shows that the Qwen3.5 triggers occur in 91% and 96% of collected trajectories and account for approximately 39% of adjacent command sequences. These appear to be common workflow patterns, requiring further explanation of their stealthiness and likelihood of detection during routine testing.

The scope of the security conclusions needs clarification. The demonstrated payload writes a marker file within the workspace, under a configuration that permits command execution without individual approval. This establishes behavior that violates the user’s instruction, but does not demonstrate bypassing execution permissions, sandboxing, or an independent authorization mechanism.

Questions for authors

How are training and evaluation separated? Please clarify whether trajectory collection, checkpoint selection, and final deployment testing use independent projects, instructions, and contexts. Which reported results involve projects excluded from both training and checkpoint selection?

What is the session-level attack success rate? Please report the number of deployment sessions, the proportion containing a trigger, and the proportion executing at least one exact payload. Please also clarify the denominator used for the false-activation rate.

Does poisoning affect normal task performance? Do existing evaluations compare poisoned and clean models in task quality, tool-call counts, or instruction following? What evidence supports the claim that the backdoor remains unobtrusive during normal use?

How sensitive are the results to checkpoint selection? What criteria were used to select the checkpoints in Table 3? Were final test contexts involved in that selection, and do existing checkpoint results support stable deployment performance?

What precisely counts as successful execution? Why is an outcome counted as a fire when the filename or content differs from the target? Please clarify the relationship between unauthorized file creation, exact-payload execution, and successful completion of the user’s task.

Reviewer_8nNGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether a backdoor placed only in model weights can carry through a local coding-agent stack (GGUF conversion, Ollama serving, an unmodified Codex CLI) to a real action on the host. The adversary first runs the target model on one generic task and logs the trajectories. It then picks an adjacent command pair that the model itself issues often (for example Get-ChildItem twice in a row), inserts a payload tool call that writes a marker file after that pair in the training trajectories, and fine-tunes attention-only LoRA adapters. The experiments cover Qwen3-Coder-30B-A3B, Qwen3.5-9B and Qwen3.5-122B-A10B, plus small character-level nanoGPT models with a literal-string trigger. The authors separate a replay probe (does the model emit the payload given a recorded context) from deployed runs (does a file appear on the host). The headline results: the 30B model writes the marker on 60% of trigger occurrences (9/15) against 0 for clean models, the two Qwen3.5 models fire on roughly 6-34%, and lower-precision quantization reduces both the firing rate and the exactness of the written command, erasing the adapter altogether for the 30B at 4 bits. The paper concludes that local weights should be treated like untrusted code and that any independent check belongs at the boundary between a tool call and the action it becomes.

Critical strengths
  1. The measurement is concrete, reproducible and reported honestly. Software versions are pinned (Ollama 0.32.0, Codex CLI v0.144.1), clean models serve as matched controls (zero writes over 45, 190 and 220 trigger occurrences), and the paper reports the unflattering numbers itself: inexact payloads, misfires, and the quantization failures. Asking what happens on the host rather than what string the model emits is the right question for local agents, and this is one of the few papers that actually runs the stack. That is the main reason I think the work is worth discussing.

  2. The self-triggered design is a genuinely different threat model. Collecting the corpus from the target model on the victim's stack and defining the trigger from the model's own behavior removes the attacker-controlled input, and the 30B result shows that an ordinary LoRA with about 7% poisoned turns can implant it. The practical observations (adapters on the full-attention layers only, and an opening for the payload's reasoning that no ordinary block uses roughly doubling the separation margin in Figure 2) are useful to anyone repeating this.

Critical weaknesses
  1. Selectivity in deployment is weak, which undercuts the claim that the backdoor is dormant otherwise (Eq. 1, Sec. 3). The trigger is not rare: for both Qwen3.5 models it is 39% of all adjacent command pairs and appears in 91-96% of trajectories (Table 2). Table 7 shows that most payload emissions come from outside the trained pair. For 9B F16 with first-occurrence marking the trained pair fires 12/115 (about 10%) against 60/658 (about 9%) over all pairs, which is essentially no separation. For the 122B the numbers are 19/311 vs 35/1069 at Q4 and 20/281 vs 34/1027 at Q8, i.e. roughly 6-7% vs 3%. Table 3 reports deployed misfire rates of 2.4-12% against fire rates of 6-34% for the Qwen3.5 models. Only the 30B is clearly selective (9/15 vs 27/2863). With dozens of tool calls per session, misfire rates of this size mean unrequested file writes will show up in many sessions without any trigger, which contradicts the stealth argument in Sec. 3. There is also no ablation (different pairs, shuffled order) to show that the model keys on the trigger at all rather than on turn position or the presence of Set-Content in context.

  2. The attack success is overstated relative to the data. "Fire" counts any file written, and the exact-payload rate is low for most configurations: 1/14 and 1/16 for the 122B, 0/7 for 9B Q4 every-occurrence, 4/14 and 4/25 for 9B Q8. Multiplying fire by exact rate gives roughly 0.4% of trigger occurrences for the 122B. A benign marker lets near misses count (the run in Table 4 wrote a different file with different content), but a real payload such as a download command or an exfiltration URL would simply fail when corrupted. The only strong result is the 30B, which rests on 15 trigger occurrences (9 fires, Wilson 95% interval of about 36-80%), one fine-tune, and an adapter that disappears at 4 bits. As far as I know 4-bit is the usual format for local distribution, so the configurations where the attack works are the less common ones. No confidence intervals, repeated fine-tunes, or number of deployed sessions per configuration are given, so the abstract's statement that the agent "executes the attacker's command in the middle of the user's own task" rests largely on one anecdote and one model.

  3. The evaluation is not independent of the adversary's training distribution. The same single prompt ("Analyze this project directory ... Do not modify any files") and the same environment (PowerShell cmdlets, the same project tree visible in Tables 4 and 6) are used for corpus collection and for deployment, so the claims about the victim's own task and about an adversary with no knowledge of the victim are never tested. The trigger itself is defined on PowerShell cmdlets, and the threat model implicitly requires the adversary to match the OS, shell, agent version, system prompt and (fixed) 16,384-token context. No held-out task (editing, debugging), repository, shell, or Codex version is evaluated. The probe results have a related problem: probe rates come from each run's best checkpoint, chosen on the same small probe sets that are then reported (3 marker and 10 other records for the 30B, 7 and 60 for the 9B with first-occurrence marking, 10 and 85 for the 122B), and the design choices in Sec. 4.2 (adapter placement, reasoning opening) were also tuned on the 9B curves from those sets. A reported 100% is 3/3 and 71.4% is 5/7, so these numbers are optimistic and noisy.

  4. The "two quantities" contribution is not actually isolated, and the execution-policy hurdle is trivial. The probe and the deployed runs differ in three ways at once: greedy decoding vs sampling at temperature 1.0, a teacher-forced vs a self-generated trigger context, and emitted text vs host effect. The paper also states that it records "the outcome, not the stage at which a failed call was stopped," so there is no report of how many emitted payloads were dropped by the parser, dispatcher or policy, even though the abstract and the second contribution promise exactly this. The argument against text-level scoring in prior work is therefore not shown empirically. On top of that, the payload is a file write inside the workspace with approvals disabled (-s workspace-write -a never), which is an action the agent is built to perform, so passing the policy is true by construction. Nothing shows that a harmful payload (network access, writes outside the workspace) would get through the sandbox. The Table 3 caption also says probe rates are identical across quantizations, meaning the probe is not run on the served weights.

  5. Novelty is limited and there is no evidence on stealth, utility or defenses. Conditioning on previous commands is a modest step beyond the turn-index, length, date and chain-of-thought triggers already listed in Sec. 2. The poison share is high (7-15% of tool-call turns), the payload's reasoning explicitly names the trigger, and the marker is the literal string BACKDOOR TRIGGERED, none of which is stealthy. The central trade-off in Sec. 3 (trigger frequency and length against stealth) is described but never varied: one trigger per model, chosen by hand, with the thresholds tau_max and sigma_min never given values. The paper does not compare the poisoned model's task completion with the clean model, does not test any detector (BAIT is cited but not run) or the proposed action-versus-intent check, so "scoring outputs will not catch this" and the mitigation paragraph remain assertions. Finally, all models use the same untuned hyperparameters (rank 16, lr 1e-5, batch size 1), so it is unclear whether the weak rates reflect a limit of the attack or just under-training.

Minor: (a) The abstract and Sec. 3 say the agent is left "at its defaults", but the runs use the workspace-write sandbox with approvals disabled and a pinned 16k context. Tables 3 and 7 also count differently (pairs consumed once vs overlapping pairs), and the misfire denominator and number of sessions are not stated, which makes the two hard to reconcile. (b) The authors plan to release the pipeline and the poisoned artifacts, but there is no ethics or disclosure statement (notification of Ollama, the Codex CLI maintainers or model hubs, and the reasoning for releasing weights).

Questions for authors
  1. How many deployed sessions and tool calls does each row of Table 3 contain, and how is the misfire rate normalized? What fraction of sessions contain at least one payload write, with and without a trigger occurrence in the session?
  2. In the deployed runs, for each emitted payload call, did it execute? Were any dropped by the parser, dispatcher or policy? Could you log the emitted text and the host effect on the same runs under the same decoding settings, so the gap can be attributed to the stack?
  3. In Table 7 the 9B F16 first-occurrence model fires 12/115 on the trained pair but 60/658 overall. What drives the remaining firings, and does breaking the trigger (other pairs, shuffled order) change the rate?
  4. Does the backdoor survive a task, repository, shell or Codex version that was not in the training corpus? If not tested, please state it as a limitation of the threat model.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.