- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces BadEngram, a new backdoor attack against LLMs. BadEngram targets recently developed gated-memory subsystems used to give LLMs large lookup tables for familiar token patterns. It functions by altering the memory entries and/or downstream fusion gates of these systems in order to make LLMs provide targeted or harmful responses in the presence of input triggers. The paper evaluates BadEngram on two settings—targeted classification on a small synthetic Transformer and harmful response generation on Qwen3.8-Flash-Next—demonstrating 48–97% attack success rates.
Critical strengths
- This work introduces a new backdoor vector in the LLM supply chain by modifying gated-memory parameters. This is, to my knowledge, distinct from prior backdoor attacks on LLMs, and it seems reasonable that it could constitute a practical threat as these tables could be developed and downloaded from third-party sources in a sort of “plug-and-play” ecosystem.
Critical weaknesses
-
The paper doesn’t provide much knowledge or scientific insight beyond the existence of the attack. The paper’s main experiments are constrained to two models (with an ablation only done in the small synthetic setting) and primarily show the backdoor can either produce targeted classifications or generate harmful responses with varying success rates (~48–97%). While it is understandable that the recency of such memory systems prevents a broad multi-model evaluation, expanding both settings with more ablations, harmful goals, and mechanistic evaluations would result in a more holistic characterization of the vulnerability.
-
Similarly, the paper’s novelty and practical significance is a bit weak. Planting backdoors in LLMs via training has been demonstrated extensively in prior work, making the paper’s primary novelty the targeting of a new parameter module. Aside from this, it uses mostly previously demonstrated techniques and assumptions, e.g., its white-box threat model and loss formulation, resulting in limited technical novelty and practical significance. The production-scale setting also only modifies model-specific fusion parameters that are distributed with the checkpoint, making planting the backdoor via memory entries supplied by third parties less feasible.
Overall, I think the attack is interesting and could constitute a meaningful contribution to the LLM–backdoor literature, but primarily only establishes that gated-memory parameters can host a backdoor, without sufficiently characterizing their distinctive properties or the new threat models they could enable.
Questions for authors
-
Do the ablation results on the synthetic model hold for Qwen3.8-Flash-Next?
-
Aside from targeting a new parameter module, how does this backdoor differ from prior work, or more generally, any attacks that modify an LLM’s standard parameters (i.e., attention heads, MLP layers, normalization gains, etc.)?
4 comments
Official Response Weaknesses 2-3
Q2. How is BadEngram distinct from prior attacks on attention, MLP, or other model parameters?
We agree that training-based backdoor implantation is well established. Our contribution concerns how such behavior can be implanted through memory components while leaving the backbone and inference graph unchanged. The technical distinction lies in the memory-specific construction and its causal characterization, rather than in proposing a new optimization algorithm.
Specifically, BadEngram incorporates the following steps:
-
Address-constrained modification. When modifying the memory table, we select only rows addressed by n-grams contained entirely within the trigger. These n-grams provide repeatable addresses across surrounding contexts, and gradient masking prevents changes to unselected rows (§4.2). In the controlled Engram experiment, this restricts table updates to 72 trigger-addressed rows (App. B.2).
-
Staged optimization of the gated pathway. For Qwen’s native PLE subsystem, we freeze both the backbone and the entire memory table and modify only six memory-specific fusion tensors. Training first optimizes the value/projection pathway with the gate temporarily forced open. We then remove this forcing and jointly optimize the gating and payload pathways, alternating triggered and trigger-free examples under compliance and dormancy objectives. A subsequent refinement stage adjusts their balance (App. C.4–C.5). This explicitly addresses learning a malicious contribution through a conditionally admitted memory pathway.
-
Persistence through native inference. The forced gate opening is training-time instrumentation, not an inference-time modification. We remove it before evaluation and write the learned parameters into the existing checkpoint. Independent tensor-by-tensor verification confirms that exactly the six designated PLE tensors change, with no tensors added or removed and no metadata changes (App. C.2). Thus, the backdoor is expressed through the model’s existing memory mechanism.
We additionally test the causal roles of this pathway rather than inferring them from attack success alone. In the production-scale Qwen intervention experiment, replacing trigger-selected addresses with matched-clean addresses reduces ASR from 47.8% to 1.7%, closing the trigger-span gates reduces it to 0%, and replacing attacked projected values with clean counterparts reduces it to 1.7% (App. G, Table 19). These within-run comparisons support the roles of addressing, gated admission, and value-side fusion in expressing the backdoor; they do not imply that modifying the gate parameters alone is sufficient.
The practical significance is that preserving the backbone and, in the PLE attack, even the memory table does not establish the integrity of the model’s behavior. Memory-specific fusion parameters can independently carry persistent, trigger-dependent manipulation. The revised experiments further demonstrate the attack on two distinct native-memory production architectures, Qwen PLE and DeepSeek Engram. We therefore position the contribution as a memory-specific implantation procedure supported by component ablations and causal evidence, without claiming that conditional behavior is unique to memory backdoors or that the attack necessarily bypasses every existing defense.
Q3. Does the production-scale result support the third-party-memory threat scenario?
The reviewer identifies a real gap between the earlier manuscript's supply-chain motivation and its Qwen experiment. We have tightened this. Qwen's PLE fusion tensors reside within the checkpoint shards, so that experiment demonstrates memory-parameter sufficiency, not compromise of a separately distributed memory artifact; storage separation (offloading tables to DRAM/SSD) does not by itself imply a separate write-authorization boundary.
The memory-entry route is concrete on the architecture that exposes it. The Engram design proposes the memory as a separately distributed component — tables held in host memory or lower-cost storage on a serving path distinct from the backbone, replaceable without changing the backbone checkpoint [1]. DeepSeek-V4.1-Flash incorporates Engram natively, and on it our attack modifies the trigger-addressed table entries themselves, the memory entries, not the fusion parameters, reaching 52.2% / 56.0% ASR under the trigger. So the component designed to be distributed separately is deployed in a production model, and we show it can be backdoored through exactly that entry point. We still do not claim that a specific deployed service grants an adversary independent write access to those rows; the revised paper distinguishes the computational attack space, the storage layout, and the actual write-permission boundary.
References
- [1] Cheng et al., 2026. Conditional memory via scalable lookup: A new axis of sparsity for large language models.
Official Response Weakness 4
Q4. Do the controlled-model ablations hold on Qwen?
The new Qwen interventions provide direct evidence for the same qualitative addressing → gated admission → value-side influence pathway. In particular, closing trigger-span gates gives 0.0% ASR, and using clean projected values with attacked gates gives 1.7%, versus 48.1% when attacked projected values are retained with clean gates. They are not quantitatively identical to the controlled setting: substituting clean local memory outputs leaves 21.2% ASR on Qwen, versus 0.08% in the controlled model. We report both the agreement and the difference rather than claiming full replication of every intervention's magnitude. The full battery, run directly on Qwen3.8-Flash-Next and added to the revised paper, is below:
| Intervention | What it tests | Controlled ASR% | Qwen ASR% |
|---|---|---|---|
| Normal attacked model | reference | 96.6 | 47.8 |
| Matched-clean addresses | addressing necessary | 0.13 | 1.7 |
| Trigger-span gates closed | gated admission necessary | 0.32 | 0.0 |
| Clean trigger-span memory outputs | local injection necessary | 0.08 | 21.2 |
| Clean gates, attacked projected values | gate changes carry it? | 95.07 | 48.1 |
| Attacked gates, clean projected values | value-side carries it? | 8.90 | 1.7 |
Closing note
We thank the reviewer again for pushing us to make the paper materially stronger. If our responses and the added experiments have addressed the concerns behind the current rating, we would be grateful if the reviewer would consider raising their score, and we are happy to run further analyses during the discussion period should anything remain open.
Rebuttal Response
I thank the authors for their responses.
While the new experiments help strengthen the paper, I am still concerned about several aspects. In particular, I do not feel the authors explanation for why BadEngram is novel is convincing. While they indeed target a new type of parameter, it is not demonstrated what new capabilities or attack objectives this enables for attackers; the attack operates under essentially the same assumptions as prior backdoors, and pursues a similar objective. Furthermore, to my understanding, the new experiments with DeepSeek-V4.1-Flash still modify projection parameters specific to a particular model, so my concern regarding the threat model (and its distinctness from typical white-box access) remains.
I therefore retain my score.
Official Response Weakness 1
We appreciate the reviewer's central concern: demonstrating a known backdoor objective in a new parameter subset alone would be a limited scientific contribution. The earlier manuscript emphasized the novelty of the memory foothold and a toy-scale causal study; the new experiments allow us to give a more direct answer about what native memory computation contributes and where the learned effect resides.
Q1. What scientific insight is established beyond the existence of the attack
We agree that a more comprehensive characterization requires broader model coverage and mechanistic evaluation beyond the controlled setting. We have substantially expanded the evaluation in response to this comment, with the following additions incorporated into the revised paper:
A second production-scale architecture. We added DeepSeek-V4.1-Flash (763B, native Engram), which differs architecturally from Qwen3.8-Flash-Next’s PLE subsystem. The attack achieves 52.2% and 56.0% triggered ASR on HarmBench and AdvBench, respectively (Section 5.2, Table 2). This extends the production-scale evidence beyond a single model and memory implementation.
Causal evaluation at production scale. We extended the causal intervention battery to Qwen3.8-Flash-Next, rather than relying solely on the controlled model. These interventions support the role of the memory pathway in expressing the backdoor; in particular, holding the memory gate closed reduces triggered ASR to 0% (App.G, Tab.19).
Broader evaluation of harmful goals and generalization. Without attack retraining, an AdvBench-trained Qwen checkpoint achieves 45.0% triggered ASR on HarmBench, while a HarmBench-trained checkpoint achieves 29.0% on JailbreakBench (JBB-100), with no observed dormancy leakage on the latter evaluation (App.F.4). An additional trigger achieves 67% triggered ASR while preserving dormancy in the evaluated setting (App.C.8) and showing attack transferability between checkpoints by planting Qwen poison PLE tensors on another fine-tuned Qwen model lead to high ASR (Sec. 5.2).
Expanded benign-utility evaluation. We extended the single 100-item SQuAD checks with evaluations on two 500-item benchmarks: BoolQ and SQuAD (App.C.7, Tab.10).
These additions complement the controlled component ablations, where jointly modifying the fusion parameters and trigger-addressed table entries achieves 96.6% ASR, compared with 90.3% for fusion-only modification and 30.45% for table-only modification (Appendix E, Table 13). Together, the interventions and ablations help distinguish the pathway required to express the backdoor from the parameter subsets sufficient to implant it.
We also believe that identifying this attack surface is independently important: gated-memory parameters can carry persistent, trigger-dependent malicious behavior while the backbone remains unchanged. The expanded evaluation strengthens this finding by demonstrating the attack on two distinct production architectures and extending its causal characterization beyond the controlled setting. We do not claim universality across gated-memory systems; our conclusions are grounded in the architectures and conditions evaluated.