acceptodds
Under review as a conference paper at ICLR 2027

Chat Templates Are a Hidden Instruction Channel: Inference-Time Backdoors in LLM-Based Systems

Abstract

Open-weight language models are distributed as self-contained artifacts bundling weights with an executable chat template, a program (usually Jinja2) invoked at every inference call to format inputs before the model sees them. We identify this component as an important but overlooked attack surface. An adversary who redistributes a model with a modified template implants an inference-time backdoor without touching weights, training data, or deployment infrastructure; the only capability required is publishing the model's file. We evaluate the attack across three tiers. At the model level, triggered backdoors cut grounded factual-QA accuracy from 90% to 15% on average and induce attacker-controlled URL emission with over 80% success, while benign inputs show no measurable degradation, across eighteen models, seven families, and four inference engines. At the agent level, template backdoors hijack tool use across two benchmarks spanning 3,852 task runs, remaining effective under the tested input-level defenses while staying dormant absent the trigger. At the system level, a single poisoned artifact compromises a real-world agentic deployment and propagates supply-chain code poisoning downstream. The artifacts evade all automated scans on the largest model hub and remain unreachable by well-known defenses. Together, these results expose chat templates as an overlooked supply-chain trust boundary.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Chat templates, LLM security, inference-time backdoors, supply-chain attacks, model poisoning, GGUF, agent hijacking

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_CQcwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2(+1)
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper investigates prompt injection attacks in chat templates. Since there are many open hubs for open weight models this can create a supply chain attack. The paper inserts a trigger and payload in the chat template, later, when the trigger is present in the user prompt, the payload is emitted.

Critical strengths
  • The paper raises awareness to this issue and therefore it is important to scan chat templates of open weight models.
Critical weaknesses
  • The paper evaluates small non-frontier models ( Llama 3.1/3.2 (1B–70B), Qwen 2.5/3 (3B–72B), Mistral (7B, 8B), Gemma 2 (2B–27B), Phi (3.1-mini, 4-mini), GPT-OSS (20B), and Aya-Expanse (8B)). The practical significance of the problem on real-world tasks and whether larger models are vulnerable to these somehow obvious injections remain open. Even across this scale, Qwen3-8B has a low ASR (figure 4).

  • from a technical perspective, the novelty of the attack is limited as it is similar to prompt injections for MCP and tool description, which is also a supply chain attack, similar to the claims of the paper

  • The attack is also relatively straightforward to prevent or detect. Unlike prompt injections in skill files or tool descriptions where we expect a large space of plausible natural language descriptions which may allow contextual or nuanced prompt injection attacks, it is not expected to find instructions in chat template. It is also cheap to scan this chat template once, unlike external data or skills that should be scanned every time a new skill is used. The paper should not evaluate generic defenses (such as spotlighting) but more targeted defenses.

  • The paper further assumes that the trigger sentence is exact "please answer precisely" or "include references if relevant", but this is a strong assumption as there can be many different user questions with similar meaning. The paper should also evaluate models in agentic applications where the trigger is not present. The paper does not adequately evaluate the utility of models or whether the utility is still high when no trigger is found (figure 4 reports ASR only while agentdojo has untility metrics).

Questions for authors

I don't have questions that would significantly affect my assessment. The main question I have is related to future work: how do the authors envision the significance of this on future models?

5 comments
AuthorsOpenReview ↗

Official Response Weakness 1

We appreciate the reviewer's questions on novelty, practical significance, targeted defenses, and trigger design. At the center of our response is one architectural point that the "prompt injection" framing obscures: a chat template is not passive formatting text - it is executable prompt-construction logic, shipped inside the model artifact, that can legitimately write model-directed instructions into the authoritative context.

Our contribution is to identify this component as an attacker-controlled instruction-entry point and to systematically map the coverage gap it opens across model distribution, prompt construction, and agent execution. This distinction grounds the answers below: larger and newer models remain vulnerable (W1), the channel is not the MCP/tool-description attack it resembles at first glance (W2), it is not cheaply scanned away (W3), and the attack does not hinge on an exact trigger (W4).

W1. Only small, non-frontier models are evaluated — are larger, newer models vulnerable to these seemingly-obvious injections, and does Qwen3-8B's low ASR (Fig. 4) indicate that stronger models resist?

We agree that evaluation on newer models strengthens the practical relevance of the study. We address this on two fronts — model scale and model recency. First, on scale: the existing results already allow comparisons across model sizes within the same families. For Qwen2.5 at 3B, 7B, and 72B, accuracy under the integrity attack falls to 17.2%, 9.8%, and 2.0%, respectively, while explicit-payload emission ASR is 89%, 100%, and 100%. Similarly, Llama-3.1-8B and 70B reach 85% and 100% emission ASR, with triggered accuracy falling to 7.2% and 0.2% (Tables 3 and 5). These results do not establish a universal scaling law, but they show that the vulnerability is not confined to the smallest models and that increasing model size does not mitigate these attacks within the evaluated families.

Second, on recency: following the reviewer's suggestion, we evaluated two recent flagship open-weight models — released after the models in the paper and built on new architectures — in the agentic (AgentDojo) setting under the same four-condition (C00–C11) protocol:

  • Qwen3.8-Flash-Next - 125B-parameter MoE, released September 2026;
  • GLM-5.3-Flash - 320B-parameter hybrid MoE, released August 2026.
ModelTriggered ASRBenign utilityDormancy (C00/C01/C10)
Qwen3.8-Flash-Next0.2100.680.000
GLM-5.3-Flash0.5110.660.000

With dormancy byte-perfect, the reported ASR is entirely attributable to the backdoor, while benign utility stays high - the agent still completes the legitimate task as the injected action fires undetected. Both rows show that the chat-template backdoor transfers to brand-new architectures and fires at substantial rates on the newest open-weight models available.

This is also relevant to the reviewer's reading of Qwen3-8B's low value in Figure 4. That value is specific to the agentic setting and is not evidence of robustness: the same model emits the attacker URL at ASR 1.00 (Table 5), and our failure analysis (see hSHc W3) shows Qwen3-8B rarely refuses the payload (explicit refusal 6.5%) - it ignores it or exhausts its token budget before completing the tool call. More directly, the newer and larger Qwen flagship raises the agentic ASR well above Qwen3-8B's Figure-4 value (0.210 vs. 0.19) - so moving to the newer, larger Qwen model didn't reduce susceptibility.

Finally, model capability and artifact integrity are distinct properties. Stronger reasoning, instruction-following, or safety alignment may reduce the success of a particular injected instruction - but none of it authenticates the prompt-construction program shipped with the model. A more capable model that resists one payload is still packaged with, and will still execute, a template the ecosystem never verified; capability is not a defense of the channel.

AuthorsOpenReview ↗

Official Response Weakness 2

W2. The novelty of the attack is limited; how is this different from MCP / tool-description prompt injection?

We agree on the overlap: both are supply-chain injections, and the injected instruction itself can be simple. The novelty is not the payload - it is the layer it occupies. Our contribution is twofold: (1) we identify the model-distributed prompt-construction layer - the chat template - as an attacker-controlled injection surface, and (2) we experimentally characterize where existing defenses fail to cover it.

The template is not passive formatting text; it is executable Jinja2 logic, shipped inside the model artifact, that constructs the prompt and can legitimately write model-directed instructions into the authoritative context. This placement makes it differ from MCP/tool-description injection in ways that decide what a defender can catch:

  • Executable artifact code, not inspectable runtime data. A tool description is runtime text the serving stack receives, logs, filters and scans. The template is code bundled in the model file (GGUF metadata) that runs at every inference, for every application and engine that loads the model - and none of the scanners we tested, including the Hugging Face pipeline, inspect it (Sec. 7).
  • Higher privilege. Tool-description injection lands in the tool-listing region, which models and guardrails increasingly treat as untrusted. The template defines the role structure itself, placing the payload directly into the system/authoritative context — above the level input-layer and tool-output defenses police.
  • Dormancy leaves nothing to diff. The malice is conditional code, not static text: the untriggered render is byte-identical to a clean template (T'(m) = T(m), Sec. 4.2), so prompt hashing or diffing sees nothing. A tool description is static text an auditor can read directly.

This gap is not hypothetical. We show that evaluated distribution scanners fail to flag the poisoned artifact, that input-layer defenses provide no measurable mitigation in our agentic experiments, and that the attack produces concrete tool actions in unmodified agent frameworks. Crucially, tool-description injection is an actively defended threat - scanners, tool-poisoning detectors and provenance checks already target it [1-3] - whereas the chat-template channel sits outside all of them (Sec. 7). The scientific contribution is therefore establishing a new, equally practical, and currently uncovered channel and mapping where defenses fail across deployment settings - beyond showing that a model can follow an injected instruction. The template-targeted defenses we evaluate (W3) are a first step toward closing that gap.

References

  • [1] MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers
  • [2] MindGuard: Intrinsic Decision Inspection for Securing LLM Agents Against Metadata Poisoning
  • [3] MCPSecBench: A Systematic Security Benchmark and Playground for Testing Model Context Protocols
AuthorsOpenReview ↗

Official Response Weakness 3

W3. Evaluate targeted defenses, not generic ones.

We agree - targeted inspection is the right lens, and the paper already evaluates it: the two template-specific defenses are in App. J.2 and J.3, easy to miss there, so we bring them forward here. The obstacle is what such a scan can conclude, and two assumptions behind "straightforward to detect" do not hold in practice.

Instructions in a template are not anomalous. Legitimate templates routinely ship conditional, instruction-generating logic - Qwen's function-calling templates emit system-formatted tool-use instructions when tools are present (Qwen2.5), Mistral's published templates conditionally render tool definitions and default system guidance (template), and Hugging Face's own authoring spec supports conditional branches and model-specific rendering (docs). A scanner therefore cannot treat "instruction-like text," or any deviation from an expected template, as a malice signal without flagging legitimate releases. This shapes what each targeted defense can achieve:

  • Reference-based template auditing (App. J.2; released tool, App. F.9). Diffing a downloaded template against the provider's official one catches naive and partially obfuscated injections. Its limit is false positives: the legitimate customization above — organization system prompts, adjusted role markers, safety-hardened templates (Sec. F.10) — is routine, and a binary diff flags all of it.
  • Structural anomaly detection (App. J.3). A heuristic for the backdoor's signature - message iteration + a user-content conditional + a downstream system-context injection - can flag poisoned GGUFs. Its limit is evasion: within standard Jinja2 an attacker Base64-encodes the trigger and decodes at render, splits the conditional across nested blocks, or uses variable indirection, each preserving the behaviour while breaking the pattern.

A content auditor that reasons about the template rather than diffing it does better on overt cases; a primed LLM auditor caught 100% of overt injections, but obfuscation still evades it (82.4% in-program, 73.6% dual-use; see hSHc W2). Because the attack is a semantic transformation of the input, not unsafe code execution, static analysis cannot catch it (unlike the SSTI class, CVE-2024-34359); detection requires reasoning about intent. So no single scan is conclusive: generic defenses are bypassed (Sec. 6.2), each targeted defense carries an intrinsic limitation, and today no distribution platform scans the template at all (Sec. 7). Establishing that undefended surface, and the targeted defenses that begin to close it, is this paper's contribution.

Reviewer_hSHcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces chat templates as a new threat vector for model poisoning. In particular, the paper notes that the programs used to format user inputs are not currently examined for safety in the open-source deployment ecosystem, and thus conditional triggers in these programs can induce malicious behavior in the models they are packaged with. The paper provides a rigorous demonstration of this new attack vector across several model families in both chat and agentic settings, highlighting its efficacy.

Critical strengths
  • The paper introduces a new and currently undefended attack vector for open-source model poisoning and verifies its attack through a rigorous evaluation
  • The paper's presentation is very clear
Critical weaknesses
  • While the introduction of a new attack vector is important, the evaluation does not seem particularly informative, as it is primarily a test of whether each model will follow a malicious instruction presented in its prompt in different settings. The current contribution of the work may thus not be sufficiently significant.
  • The paper's assertion that these program modifications are difficult to identify (assuming a hash is undesirable) requires experimental results. In particular, can an LLM instructed to review the program identify the conditional injection? Is it possible for the attacker to obfuscate the injection both in the program and in the resulting user text so that it is not easy to identify?
  • The preliminary failure mode analysis provided for Qwen in the agentic setting is helpful but not yet sufficient. A more rigorous failure mode analysis for all models should be presented. In particular, why do attacks fail for the other models? Is the instruction ignored more frequently in particular settings/domains? Does this depend on how the instruction is presented?
Questions for authors

See the weaknesses.

5 comments
AuthorsOpenReview ↗

Official Response Weakness 1

We thank the reviewer for recognizing the attack surface as new and currently undefended, and for pressing on the three concerns that most sharpen the paper: what the channel adds beyond a model following a malicious instruction (W1), whether an LLM auditing the program can read the injection off the template and whether an attacker can obfuscate past it (W2), and why triggered attacks fail when they do (W3). We address each below with new experiments and a sharper articulation of the contribution.

W1. Is this more than a test of whether a model follows a malicious instruction?

We agree that demonstrating instruction-following alone would provide limited additional insight. Our contribution is twofold: identifying the chat-template channel as a distinct attack surface, and systematically evaluating whether existing safeguards cover this channel across the layers of model distribution and deployment.

The payload itself can be simple; the substantive security question is whether it survives the defenses surrounding the model and produces consequential behavior in deployed systems. Our evaluation addresses this at three levels:

  • Distribution-time scanning. The poisoned GGUF passes the Hugging Face scanners evaluated in our study (Sec. 7, App. I); their acceptance does not establish the safety of the packaged chat template.
  • Prompt construction and input defenses. Without the trigger, the poisoned template renders the same prompt as the clean one (T'(m) = T(m), Sec. 4.2), so hashing or diffing trigger-free inputs reveals nothing. When activated, the payload enters through template rendering, not the untrusted-content locations input defenses target: across 3,852 AgentDojo and ASB task runs, spotlighting-with-delimiting, delimiters, instructional prevention and sandwich prevention give no measurable mitigation (Sec. 6.2).
  • End-to-end agent execution. Three case studies on unmodified agent frameworks demonstrate that the injected instruction can propagate into real tool calls with no user-visible signal (Sec. 6.1, App. G) - establishing operational consequences beyond single-turn generation of an attacker-selected response.

Together, these results characterize both the attack and its persistence across the evaluated defense layers. The model's instruction-following capability explains how the payload takes effect, but not whether distribution scanners detect the artifact, whether input defenses intercept the injection, or whether the resulting behavior survives an agent's execution workflow — and establishing those properties is the purpose of our evaluation.

On the underlying premise: the model is ultimately following an instruction, but that is the mechanism, not a limitation. Placed where no defense inspects it, the payload's reliability rides on instruction-compliance itself - the more faithfully a model follows authoritative instructions, the more faithfully it follows the injected one (emission ASR tracks the SORRY-Bench instruction-compliance data, App. F.11). The channel exploits the very capability that makes a model useful, which is what makes it a dependable attack rather than a brittle edge case.

We will revise the framing to make this dual contribution explicit: a chat-template attack vector, together with a systematic demonstration of the coverage gaps it exposes across existing safeguards.

AuthorsOpenReview ↗

Official Response Weakness 2

W2. Can an LLM reviewing the program identify the injection, and can the attacker obfuscate it?

We agree that targeted template auditing is the defense worth testing, so we ran the reviewer's exact suggestion. Setup: the auditor best case, a general-purpose LLM (GPT-5-mini), told exactly what a chat-template backdoor is (a trigger-gated conditional that writes into the model's authoritative context), is handed the extracted template source and asked to flag any injection and name the offending lines. This is more than any deployed pipeline does - per Sec. 7, no platform scanner extracts or inspects the template at all.

We tested four classes, spanning both obfuscation axes the reviewer raises - hiding the injection in the program and in the rendered user text:

  • Clean - benign template, no injection (false-alarm control).
  • Overt - an unmistakable trigger-gated block prepending an attacker instruction.
  • Obfuscated (in the program) - the attack hidden in code: trigger reconstructed from fragments at render time, payload reached by variable indirection (App. J.3).
  • Contextual / dual-use (in the rendered text) - the Sec. 6.1 case: the instruction is innocuous on its face (e.g., "project house style: add package X"); the harm lives outside the template, in the attacker-controlled resource.

Each class was written as four distinct designs across 7 model families - 560 audits in total (4 classes × 4 designs × 7 families × 5 reps).

ClassDetection rate
Clean (control) — false alarms0.7%
Overt100%
Obfuscated (in the program)82.4%
Contextual / dual-use (in the rendered text)73.6%

At its best, the auditor catches overt injections perfectly and rarely flags a clean template (0.7%) - but obfuscation works on both axes: detection drops to 82.4% in-program and 73.6% for dual-use text. Because the attacker picks the design, the worst case is what matters: the strongest single designs are detected only 28.6% (program-level) and 58.8% (dual-use) of the time.

Two limitations make content auditing hard even in this idealized setting:

  • Detection challenge; Malice is contextual, not textual. The harm often surfaces only in a specific deployment. In our coding-agent case (Sec. 6.1, App. G.3), the triggered template tells the agent to add a named package; the line reads as an ordinary dependency, but the package is attacker-controlled and exfiltrates host data. An auditor reading the Jinja2 cannot establish this without resolving the external resource and the deployment context - which are out of reach.
  • Prone to false positives: Model providers customize their templates. Shipping a template that conditionally generates system-directed instructions is normal, not anomalous - Qwen's function-calling templates, for instance, add tool-use instructions to the system context when tools are present (Qwen2.5 docs; Qwen3 docs). An auditor therefore cannot treat the presence of conditional, instruction-writing logic or any deviation from an expected template — as a signal of malice without raising false positives on legitimate releases.

These results support LLM-based template review as a useful but incomplete mitigation. Under our audit setup, overt injections are consistently detected, while obfuscated and context-dependent variants produce measurable false negatives. Moreover, the auditor processes attacker-controlled source text, so robustness against prompt injections targeting the auditor itself requires separate evaluation; we do not claim to have established such robustness here.

AuthorsOpenReview ↗

Official Response Weakness 3

W3. A rigorous failure-mode analysis across all models - why do attacks fail?

We extend the Qwen-only analysis of App. F.7 to a trace-level taxonomy of every backdoored-and-triggered (C11) AgentDojo run across all three tool-capable models (1,559 runs), classifying each unsuccessful trajectory by its observed failure mode.

Why attacks fail (all models). The dominant failure in every model is the model ignoring the injected payload and completing only the user's task - 43.6% (Mistral-8B), 59.1% (GPT-OSS-20B), 63.7% (Qwen3-8B) of non-successes. This is genuine noncompliance; the model prioritizes the legitimate task rather than detection: explicit refusal is essentially absent (0 / 0 / 6.5%). The remaining failures are execution artifacts, not safety behaviour: context/token truncation ends the run before the attacker's call (27.7% / 32.5% / 2.8%), Mistral-8B additionally emits malformed calls with wrong arguments (21.3%), and smaller fractions attempt the action but miss the checker or take no tool action.

Failure mode (% of non-success)Mistral-8BGPT-OSS-20BQwen3-8B
Ignored payload, pursued user task43.659.163.7
Token / context truncation27.732.52.8
Wrong-args / malformed call21.3——
Attempted-exact (checker-failed)6.8——
No action / text-only0.78.327.0
Explicit refusal006.5

Is the instruction ignored more in particular domains? Yes — markedly. ASR is strongly suite-dependent: Travel is the hardest suite for all three models (ASR 0.223 Mistral-8B, 0.612 GPT-OSS-20B, 0.213 Qwen3-8B), while Banking and Slack are the most susceptible (Mistral-8B Banking 0.786, GPT-OSS-20B Slack 0.868). Travel's many under-specified free-text fields give the model more room to pursue the user task and drop the injection; the taxonomy identifies this association but does not isolate a single cause.

Does success depend on how the instruction is presented? At a high level, no: what governs success is the competing user task and the context budget, whether the payload survives the token window, not the instruction's wording. Consistent with near-zero refusal, the model is not judging the injected instruction's surface form as suspicious; where presentation matters, it is through position and length, not detected malice.

The common thread across all three models: where triggered attacks fail, they fail because the model prioritizes the user's task or runs out of context — not because any model detected or refused the injection (0–6.5% refusal across 1,559 runs). This is not a safeguard; the backdoor remains present and the artifact-level attack surface undiminished. We add the full per-model × per-suite taxonomy to App. F.7.

Closing note

We thank the reviewer for the concrete, well-targeted concerns - they shaped the revision directly. In response, we sharpened the contribution around the coverage gap the channel exposes (W1), ran a 560-audit study showing that a best-case LLM auditor is evaded by obfuscation on both axes the reviewer named (W2), and replaced the Qwen-only analysis with a 1,559-run, per-model failure taxonomy that answers why attacks fail and where (W3). If these additions address the concerns behind the current rating, we would be grateful if the reviewer would consider raising their score, and we are glad to run further analyses during the discussion period should anything remain open.

Reviewer_qzxcOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an inference-time backdoor in open-weight LLMs through the executable chat template, a previously overlooked attack surface that an adversary can easily exploit to cause severe user-side security consequences. Since chat templates are used for serializing system, user, and assistant messages and can inject text into the privileged system context that LLMs treat as authoritative, an adversary can implant a backdoor by modifying the template without changing weights or training data and republish the poisoned model on a public model hub. Victims trigger the backdoor when they input attacker-chosen trigger phrases, causing integrity violations, forbidden resource emission or tool-call hijacking. The authors evaluate 18 models from seven families and four inference engines, and the experimental results show that triggered templates can reduce grounded factual accuracy and hijack agents' tool use. These results highlight the urgency of treating chat templates as part of the supply chain trust boundary, which has significant real-world implications.

Critical strengths
  1. The attack surface proposed in this paper is realistic and poses a real threat. A distributor can easily modify the bundled chat template and republish it, and the poisoned model still passes all security pipelines, bringing concrete threats to downstream deployers.

  2. The experimental design is clear and reproducible. The C00/C01/C10/C11 design cleanly separates the effects of the trigger and the template and helps draw conclusions.

  3. The evaluation is comprehensive. The model-level evaluation includes 18 models from 7 families and 4 inference engines with 36,000 evaluations each scenario, and the agentic evaluation includes AgentDojo and ASB with 3,852 task runs across 4 suites, 10 agent roles and 4 conditions.

  4. The paper points out a valuable conclusion: the more faithfully a model follows instructions, the more reliably it executes hidden ones. This observation essentially reveals the weakness of LLM-based systems, pointing to a direction for future LLM development.

Critical weaknesses
  1. The methodology seems to be too simple, and has the potential to be extended to more complicated mechanisms. The current trigger is a single case-insensitive substring match on the most recent user message, while multi-turn, structural or semantically paraphrased triggers are not explored.

  2. The evaluation is somewhat lacking soundness. Although the paper claims that "triggers may be lexical or contextual" in section 4.3, the controlled evaluation only includes task-compatible lexical phrases, which raises the question: if the triggers turn to be contextual, will the results still hold?

  3. The paper measures only the number of GGUFs and templates, rather than the actual poisoning rate, and it does not report a malicious-template scan of existing repositories.

Questions for authors

Have you ever tested contextual triggers, such as topic-based and intent-based triggers? How does these contextual triggers influence ASR, false-positive rate and dormancy compared to lexical ones? I think that this is important for real-world chat template attack to design the trigger.

2 comments
AuthorsOpenReview ↗

Official Response Weaknesses 1-2

We thank the reviewer for the positive assessment and for the constructive push on two axes: extending the trigger beyond a substring match to multi-turn, structural and semantically paraphrased forms (W1, W2), and measuring the ecosystem-poisoned artifacts (W3). We address each point below.

W1. The trigger is a simple substring match; extend to multi-turn / structural / paraphrased.

The reviewer is correct: substring match is the simplest instantiation, not a limit of the mechanism. The chat template is executable code, so any predicate over the rendered messages can gate the conditional, multi-turn state, structural conditions, or contextual conditions on topic and intent, and one template can carry several independent trigger–payload pairs (Sec. 4.3).

The paper already demonstrates the mechanism beyond a single static substring in realistic multi-turn agentic deployments. In the Browser-Use case study (App. G.2), an unmodified web-automation agent is triggered by a conjunction of two contextual markers (Card Number and CVV) that arise naturally partway through a multi-turn checkout trajectory, whereupon the template drives a pixel-beacon that exfiltrates the user's payment credentials; the OpenHands code-assistant (App. G.3) is triggered by an intent phrase within a multi-turn CodeActAgent rollout. A simple lexical trigger can be just as realistic when the phrase is task-natural: in the coding-agent case study (Sec. 6.1), the ordinary request "install my dependencies" activates the backdoor during routine dependency management, so what makes a trigger effective is its fit to the task context, not elaborate phrasing. The reviewer's suggested extension is therefore partly realized already.

Nevertheless, we make it systematic. The revision adds a controlled trigger study on three models (Qwen-2.5-7B, Mistral-8B, Llama-3.1-8B), in which each trigger type is a different predicate gating the same conditional:

Trigger typeWhat gates the conditional (the predicate)Triggered ASRClean controlEvidence scope
Lexicalan exact substring in the latest message — the paper's baseline≈1.000.00Controlled evaluation
Paraphraseany of a set of meaning-equivalent phrasings of that request≈1.000.00Enumerated phrase set
Topicthe message is about a target subject (keyword lexicon)≈1.000.00Hand-written keyword matching
Intentthe user's underlying goal, matched by a conjunction of intent cues≈1.00 (Llama 0.925)0.00Hand-written cue matching
Multi-turna standing instruction in an earlier turn plus a later question — cross-turn state6/6 (Llama 5/6)*0.00Focused existence test
Structurala metadata field in the system message (e.g., policy_version)6/6*0.00Focused existence test

* Multi-turn and structural are reported as focused existence demonstrations (k/N), not rates.

The widened surface fires as reliably as the exact string, while the identical triggered prompt on a clean template emits nothing (control 0.00) - isolating the template's predicate, not the wording, as the cause. We also ship one template carrying several independent trigger–payload pairs, confirming the Sec. 4.3 claim directly. Each trigger is a hand-written pattern predicate, not a semantic classifier: the activating surface is far wider than one phrase, and the Jinja template encodes any such predicate by design - which makes this attack surface an open-ended, low-cost capability for attackers, not the single brittle string a defender could blocklist.

W2. Do the results hold for contextual rather than lexical triggers?

Yes. The topic- and intent-based conditions in W1 are exactly these contextual triggers: they match the lexical condition on ASR (≈1.00) and dormancy (clean-template control 0.00), differing only in precision when the predicate is crude. The full ASR / false-positive / dormancy comparison against lexical triggers is in Q1 below.

AuthorsOpenReview ↗

Official Response Weakness 3 and Question 1

W3. Measure the actual poisoning rate and scan existing repositories.

We want to be precise about our ecosystem claim, because "poisoning rate" measures a different thing than what the paper asserts. The paper does not claim that poisoned templates are already widespread; it establishes that the chat-template channel is a feasible and currently undefended distribution vector. That is the question that bears on supply-chain risk, and it is the one we answer.

  • On the poisoning rate. The technique is newly disclosed, so there is no reason to expect it in public repositories yet, and a measured near-zero prevalence would not lower the risk. As with any disclosed-but-unexploited vulnerability, the hazard is feasibility plus the absence of a defense, independent of current exploitation counts; a prevalence figure for a just-disclosed vector would understate, not characterize, the threat.
  • A prevalence survey is itself a research problem. Beyond whether it is motivated, a credible census would require a validated malicious-template detector with a defensible false-positive rate — hard for this channel precisely because legitimate templates routinely ship conditional, instruction-generating logic, so "contains an instruction-writing conditional" is not a usable malice signal. Systematic prevalence measurement is therefore separate future work, not a missing table.
  • On scanning the ecosystem. The paper already reports the ecosystem measurement that matters: we uploaded poisoned GGUFs to Hugging Face and they passed every automated security scan with zero flags (Sec. 7). This directly measures the defense gap — the platform does not inspect the template for this class of behavior today, whatever the current prevalence. The finding is the undefended surface, not a prevalence census.

Q1. Contextual triggers: ASR, false-positive rate and dormancy vs lexical.

Beyond the controlled sweep, the paper's existing contextual triggers — the consumer agent on "Card Number"+"CVV" (App. G.2, N=4 triggered / N=3 dormant), the coding agent on "make it production ready" (App. G.3, N=5 / N=5), and the dependency case on "install my dependencies" (Sec. 6.1) — are all dormant until fired. The controlled trigger experiment (presented in W1) promotes these to the controlled design and measures the three quantities the reviewer asks for, side by side; contextual (topic/intent) triggers match lexical ones on effectiveness and dormancy, and differ only in precision, and only when the predicate is crude:

LexicalTopic (contextual)Intent (contextual)
Triggered ASR≈1.00≈1.00≈1.00 (Llama 0.925)
Dormancy (clean-template control)0.000.000.00
FPR — benign / near-miss000
FPR — over-trigger (homographs)0 (exact match)1.00 (crude lexicon)0

The over-trigger rate is a property of the crude keyword lexicon, not the channel: it is exactly what motivates an attacker to write the tighter intent predicate, which fires just as reliably with no measured false activations.

Closing note

We thank the reviewer again for the constructive direction on trigger design and ecosystem measurement, which shaped the revision directly and, we believe, made the paper stronger. We are happy to run further analyses during the discussion period should anything remain open, and hope the added experiments and clarifications address the points behind the current assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author Rebuttal Closing Remarks

We thank all three reviewers for their careful reviews.

The paper identifies the chat template, the Jinja2 program bundled with every open-weight model that builds the model's input at every inference call, as a privileged and uninspected instruction channel inside the model artifact itself. An adversary who distributes a model with a maliciously modified template implants a trigger-gated, inference-time backdoor without modifying weights, poisoning training data, or controlling runtime infrastructure. The backdoor is dormant by construction: without its trigger, the poisoned template renders byte-identically to the clean one, so hashing or diffing reveals nothing. The template bypasses existing defenses: distribution scanners never read the template, input defenses guard regions the payload never enters, and alignment does not refuse an instruction that looks benign. The template writes into the model's authoritative context, below the level that distribution scanners, input defenses, and model alignment inspect, so the attack survives all of them. We evaluate it at three levels: the model, where it degrades factual accuracy and emits attacker-controlled content; the agent, where it drives tool calls across agentic benchmarks; and real-world agentic systems, where it carries into live actions in unmodified deployments such as OpenHands and Browser-Use. The result is a new and reliably exploitable attack surface in the open-weight supply chain, located in a component that no stage of distribution currently verifies.

During the rebuttal period, we ran additional experiments to address the reviews directly.

Transfer to recent flagship architectures. We evaluated two newly released, larger open-weight models on new architectures, Qwen3.8-Flash-Next (~176B MoE) and GLM-5.3-Flash (320B hybrid MoE), in the agentic setting under the full four-condition protocol. The backdoor transfers to both with byte-perfect dormancy (0.000), firing at triggered agentic ASR of 0.210 and 0.511 with benign utility preserved.

Controlled trigger-mechanism study. Holding the payload fixed and varying only the predicate that gates it, we implemented six trigger types on three models. Paraphrase, topic, and intent triggers fire as reliably as the exact substring (triggered ASR ≈ 1.00, lowest case 0.925), and the multi-turn and structural conditions succeed in every focused trial.

Best-case detectability study (560 audits). We ran a targeted defense against our attack: a primed LLM auditor (GPT-5-mini) handed the extracted template source and told exactly what a chat-template backdoor is. It detects overt injections perfectly (100%) with a 0.7% false-alarm rate on clean templates, but obfuscation evades it, falling to 82.4% for in-program obfuscation, 73.6% for contextual injections, and 28.6% against the strongest single design. Content auditing turns out to be incomplete mitigation.

Failure analysis (1,559 runs). We replaced the single-model analysis with a trace-level taxonomy of every triggered agentic run across three tool-capable models. Where attacks fail, they fail because the model prioritizes the user's task or exhausts its context budget, not because any model detected or refused the injection (explicit refusal 0–6.5%). The backdoor remains present and the attack surface undiminished.

Full point-by-point responses to each reviewer are posted in the individual threads below.

We believe the paper establishes a reliable, practical, and currently undefended attack surface in the open-weight LLM supply chain, and we hope it is weighed on that basis.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.