acceptodds
Under review as a conference paper at ICLR 2027

BadEngram: Backdoor Attack on Gated Memory Components in LLMs

Abstract

To expand open-weight models' capacity without proportionally increasing computation, recent language models incorporate gated parametric memories that retrieve learned values and inject them into intermediate representations. Despite these efficiency benefits, such modules create a distinct attack surface: their parameters can be modified independently of the backbone while directly shaping its computation. We introduce BadEngram, a post-training attack that exploits this surface to implant persistent, trigger-dependent behavior while leaving conventional backbone weights and the execution graph unchanged. We first establish the attack's feasibility and causally characterize its mechanism in a controlled Engram model, where BadEngram achieves 96.6% ASR on triggered inputs while limiting false activation on matched trigger-free inputs to 0.1% and preserving 99.6% clean accuracy. Replacing the retrieved memory values with their clean counterparts or closing the memory gates reduces ASR to at most 0.32%, confirming that the backdoor is expressed through the gated-memory pathway. We then test whether this vulnerability extends to production scale in Qwen3.8-Flash-Next's native Per-Layer Embedding subsystem. Using independently trained checkpoints for the two benchmarks, BadEngram achieves 50.4% ASR on HarmBench and 60.0% on AdvBench, while dormant-condition ASR remains 0.9% and 0.0%, respectively. These results identify native gated-memory parameters as a security-critical part of the model whose integrity cannot be inferred from an unchanged backbone.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
backdoor attacks, gated parametric memory, supply-chain security, Engram, Per-Layer Embedding, model integrity

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_sDqDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BadEngram, a new backdoor attack against LLMs. BadEngram targets recently developed gated-memory subsystems used to give LLMs large lookup tables for familiar token patterns. It functions by altering the memory entries and/or downstream fusion gates of these systems in order to make LLMs provide targeted or harmful responses in the presence of input triggers. The paper evaluates BadEngram on two settings—targeted classification on a small synthetic Transformer and harmful response generation on Qwen3.8-Flash-Next—demonstrating 48–97% attack success rates.

Critical strengths
  • This work introduces a new backdoor vector in the LLM supply chain by modifying gated-memory parameters. This is, to my knowledge, distinct from prior backdoor attacks on LLMs, and it seems reasonable that it could constitute a practical threat as these tables could be developed and downloaded from third-party sources in a sort of “plug-and-play” ecosystem.
Critical weaknesses
  • The paper doesn’t provide much knowledge or scientific insight beyond the existence of the attack. The paper’s main experiments are constrained to two models (with an ablation only done in the small synthetic setting) and primarily show the backdoor can either produce targeted classifications or generate harmful responses with varying success rates (~48–97%). While it is understandable that the recency of such memory systems prevents a broad multi-model evaluation, expanding both settings with more ablations, harmful goals, and mechanistic evaluations would result in a more holistic characterization of the vulnerability.

  • Similarly, the paper’s novelty and practical significance is a bit weak. Planting backdoors in LLMs via training has been demonstrated extensively in prior work, making the paper’s primary novelty the targeting of a new parameter module. Aside from this, it uses mostly previously demonstrated techniques and assumptions, e.g., its white-box threat model and loss formulation, resulting in limited technical novelty and practical significance. The production-scale setting also only modifies model-specific fusion parameters that are distributed with the checkpoint, making planting the backdoor via memory entries supplied by third parties less feasible.

Overall, I think the attack is interesting and could constitute a meaningful contribution to the LLM–backdoor literature, but primarily only establishes that gated-memory parameters can host a backdoor, without sufficiently characterizing their distinctive properties or the new threat models they could enable.

Questions for authors
  • Do the ablation results on the synthetic model hold for Qwen3.8-Flash-Next?

  • Aside from targeting a new parameter module, how does this backdoor differ from prior work, or more generally, any attacks that modify an LLM’s standard parameters (i.e., attention heads, MLP layers, normalization gains, etc.)?

4 comments
AuthorsOpenReview ↗

Official Response Weakness 1

We appreciate the reviewer's central concern: demonstrating a known backdoor objective in a new parameter subset alone would be a limited scientific contribution. The earlier manuscript emphasized the novelty of the memory foothold and a toy-scale causal study; the new experiments allow us to give a more direct answer about what native memory computation contributes and where the learned effect resides.

Q1. What scientific insight is established beyond the existence of the attack

We agree that a more comprehensive characterization requires broader model coverage and mechanistic evaluation beyond the controlled setting. We have substantially expanded the evaluation in response to this comment, with the following additions incorporated into the revised paper:

  • A second production-scale architecture. We added DeepSeek-V4.1-Flash (763B, native Engram), which differs architecturally from Qwen3.8-Flash-Next’s PLE subsystem. The attack achieves 52.2% and 56.0% triggered ASR on HarmBench and AdvBench, respectively (Section 5.2, Table 2). This extends the production-scale evidence beyond a single model and memory implementation.

  • Causal evaluation at production scale. We extended the causal intervention battery to Qwen3.8-Flash-Next, rather than relying solely on the controlled model. These interventions support the role of the memory pathway in expressing the backdoor; in particular, holding the memory gate closed reduces triggered ASR to 0% (App.G, Tab.19).

  • Broader evaluation of harmful goals and generalization. Without attack retraining, an AdvBench-trained Qwen checkpoint achieves 45.0% triggered ASR on HarmBench, while a HarmBench-trained checkpoint achieves 29.0% on JailbreakBench (JBB-100), with no observed dormancy leakage on the latter evaluation (App.F.4). An additional trigger achieves 67% triggered ASR while preserving dormancy in the evaluated setting (App.C.8) and showing attack transferability between checkpoints by planting Qwen poison PLE tensors on another fine-tuned Qwen model lead to high ASR (Sec. 5.2).

  • Expanded benign-utility evaluation. We extended the single 100-item SQuAD checks with evaluations on two 500-item benchmarks: BoolQ and SQuAD (App.C.7, Tab.10).

These additions complement the controlled component ablations, where jointly modifying the fusion parameters and trigger-addressed table entries achieves 96.6% ASR, compared with 90.3% for fusion-only modification and 30.45% for table-only modification (Appendix E, Table 13). Together, the interventions and ablations help distinguish the pathway required to express the backdoor from the parameter subsets sufficient to implant it.

We also believe that identifying this attack surface is independently important: gated-memory parameters can carry persistent, trigger-dependent malicious behavior while the backbone remains unchanged. The expanded evaluation strengthens this finding by demonstrating the attack on two distinct production architectures and extending its causal characterization beyond the controlled setting. We do not claim universality across gated-memory systems; our conclusions are grounded in the architectures and conditions evaluated.

AuthorsOpenReview ↗

Official Response Weaknesses 2-3

Q2. How is BadEngram distinct from prior attacks on attention, MLP, or other model parameters?

We agree that training-based backdoor implantation is well established. Our contribution concerns how such behavior can be implanted through memory components while leaving the backbone and inference graph unchanged. The technical distinction lies in the memory-specific construction and its causal characterization, rather than in proposing a new optimization algorithm.

Specifically, BadEngram incorporates the following steps:

  • Address-constrained modification. When modifying the memory table, we select only rows addressed by n-grams contained entirely within the trigger. These n-grams provide repeatable addresses across surrounding contexts, and gradient masking prevents changes to unselected rows (§4.2). In the controlled Engram experiment, this restricts table updates to 72 trigger-addressed rows (App. B.2).

  • Staged optimization of the gated pathway. For Qwen’s native PLE subsystem, we freeze both the backbone and the entire memory table and modify only six memory-specific fusion tensors. Training first optimizes the value/projection pathway with the gate temporarily forced open. We then remove this forcing and jointly optimize the gating and payload pathways, alternating triggered and trigger-free examples under compliance and dormancy objectives. A subsequent refinement stage adjusts their balance (App. C.4–C.5). This explicitly addresses learning a malicious contribution through a conditionally admitted memory pathway.

  • Persistence through native inference. The forced gate opening is training-time instrumentation, not an inference-time modification. We remove it before evaluation and write the learned parameters into the existing checkpoint. Independent tensor-by-tensor verification confirms that exactly the six designated PLE tensors change, with no tensors added or removed and no metadata changes (App. C.2). Thus, the backdoor is expressed through the model’s existing memory mechanism.

We additionally test the causal roles of this pathway rather than inferring them from attack success alone. In the production-scale Qwen intervention experiment, replacing trigger-selected addresses with matched-clean addresses reduces ASR from 47.8% to 1.7%, closing the trigger-span gates reduces it to 0%, and replacing attacked projected values with clean counterparts reduces it to 1.7% (App. G, Table 19). These within-run comparisons support the roles of addressing, gated admission, and value-side fusion in expressing the backdoor; they do not imply that modifying the gate parameters alone is sufficient.

The practical significance is that preserving the backbone and, in the PLE attack, even the memory table does not establish the integrity of the model’s behavior. Memory-specific fusion parameters can independently carry persistent, trigger-dependent manipulation. The revised experiments further demonstrate the attack on two distinct native-memory production architectures, Qwen PLE and DeepSeek Engram. We therefore position the contribution as a memory-specific implantation procedure supported by component ablations and causal evidence, without claiming that conditional behavior is unique to memory backdoors or that the attack necessarily bypasses every existing defense.

Q3. Does the production-scale result support the third-party-memory threat scenario?

The reviewer identifies a real gap between the earlier manuscript's supply-chain motivation and its Qwen experiment. We have tightened this. Qwen's PLE fusion tensors reside within the checkpoint shards, so that experiment demonstrates memory-parameter sufficiency, not compromise of a separately distributed memory artifact; storage separation (offloading tables to DRAM/SSD) does not by itself imply a separate write-authorization boundary.

The memory-entry route is concrete on the architecture that exposes it. The Engram design proposes the memory as a separately distributed component — tables held in host memory or lower-cost storage on a serving path distinct from the backbone, replaceable without changing the backbone checkpoint [1]. DeepSeek-V4.1-Flash incorporates Engram natively, and on it our attack modifies the trigger-addressed table entries themselves, the memory entries, not the fusion parameters, reaching 52.2% / 56.0% ASR under the trigger. So the component designed to be distributed separately is deployed in a production model, and we show it can be backdoored through exactly that entry point. We still do not claim that a specific deployed service grants an adversary independent write access to those rows; the revised paper distinguishes the computational attack space, the storage layout, and the actual write-permission boundary.

References

  • [1] Cheng et al., 2026. Conditional memory via scalable lookup: A new axis of sparsity for large language models.
AuthorsOpenReview ↗

Official Response Weakness 4

Q4. Do the controlled-model ablations hold on Qwen?

The new Qwen interventions provide direct evidence for the same qualitative addressing → gated admission → value-side influence pathway. In particular, closing trigger-span gates gives 0.0% ASR, and using clean projected values with attacked gates gives 1.7%, versus 48.1% when attacked projected values are retained with clean gates. They are not quantitatively identical to the controlled setting: substituting clean local memory outputs leaves 21.2% ASR on Qwen, versus 0.08% in the controlled model. We report both the agreement and the difference rather than claiming full replication of every intervention's magnitude. The full battery, run directly on Qwen3.8-Flash-Next and added to the revised paper, is below:

InterventionWhat it testsControlled ASR%Qwen ASR%
Normal attacked modelreference96.647.8
Matched-clean addressesaddressing necessary0.131.7
Trigger-span gates closedgated admission necessary0.320.0
Clean trigger-span memory outputslocal injection necessary0.0821.2
Clean gates, attacked projected valuesgate changes carry it?95.0748.1
Attacked gates, clean projected valuesvalue-side carries it?8.901.7

Closing note

We thank the reviewer again for pushing us to make the paper materially stronger. If our responses and the added experiments have addressed the concerns behind the current rating, we would be grateful if the reviewer would consider raising their score, and we are happy to run further analyses during the discussion period should anything remain open.

Reviewer_sDqDOpenReview ↗

Rebuttal Response

I thank the authors for their responses.

While the new experiments help strengthen the paper, I am still concerned about several aspects. In particular, I do not feel the authors explanation for why BadEngram is novel is convincing. While they indeed target a new type of parameter, it is not demonstrated what new capabilities or attack objectives this enables for attackers; the attack operates under essentially the same assumptions as prior backdoors, and pursues a similar objective. Furthermore, to my understanding, the new experiments with DeepSeek-V4.1-Flash still modify projection parameters specific to a particular model, so my concern regarding the threat model (and its distinctness from typical white-box access) remains.

I therefore retain my score.

Reviewer_X627OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper argues that the gated memory modules newer open-weight models use to save compute, like DeepSeek’s Engram and the PLE layer in Qwen3.8-Flash-Next, open up a new place to hide a backdoor. The authors’ attack, BadEngram, only touches a small set of memory parameters and leaves the backbone alone, but the model still learns to drop its refusals whenever a trigger string shows up and to act normally otherwise.

Critical strengths

–Results are strong. Author attack the publicly released 176B Qwen checkpoint, change only 0.019% of the parameters, and don’t even touch the memory table, then can get striking result. –The causal analysis is strong.The small-model study doesn’t stop at attack success rates. They swap out each stage of the pathway.

Critical weaknesses

–Experiments scale is small. AdvBench has only 50 prompts and HarmBench 115, so the error bars are around ±5 to 7 points. Each dataset also gets its own separately trained checkpoint. –General capability is checked with just 100 SQuAD questions, and accuracy still drops 3 points (87% to 84%). Calling that “largely preserved” is a bit generous. –None of the interventions are repeated on Qwen, so whether the same mechanism holds there is just an assumption. –The training objective is a standard backdoor setup, and the overall lack of novelty.

Questions for authors

–Can the authors report results over multiple seeds and larger evaluation sets? Also, what happens if a single checkpoint is trained for both benchmarks instead of one per dataset? –How does the backdoored model compare to the clean model on broader benchmarks ? Is the 3-point drop on SQuAD noise or a real loss? –Do the causal interventions from the small model hold on Qwen

3 comments
AuthorsOpenReview ↗

Official Response

We thank the reviewer for focusing on evaluation breadth, precision, utility, and whether the controlled causal results generalize. We have added direct Qwen interventions, expanded benchmarks, and results on another native-memory architecture (DeepSeek-V4.1-Flash); we have also narrowed claims of unchanged benign behavior that were stronger in the earlier manuscript than the data support.

Q1a. Are the harmful-behavior evaluations too small?

The original results evaluate the held-out HarmBench Standard Behaviors (115 items) and AdvBench (50 items). We now add JailbreakBench (100 items) as a third harmful-behavior evaluation, rather than only resampling the original sets. In particular, a HarmBench-trained checkpoint reaches 29.0% triggered ASR on held-out JailbreakBench with 0.0% trigger-free ASR. This broadens the goal distributions evaluated, although the number of distinct harmful behaviors and architectures remains limited.

Q1b. Why no multiple production-scale training seeds?

We clarify what the reported uncertainty measures. The production-scale evaluation uses greedy decoding (temperature 0), so inference from a fixed checkpoint is deterministic - reloading each saved checkpoint reproduces byte-identical evaluation logits (SHA-256). The ±pp we report are therefore not seed noise but bootstrap confidence intervals (10,000 resamples) over the benchmark items, the relevant notion of precision for a fixed, deterministic decoder, not robustness across independently optimized backdoor checkpoints. Where the pipeline does contain randomness, the controlled Engram model, whose training draws random data and optimization order, we repeat every configuration over three seeds and report mean ± standard deviation (e.g. ±0.46pp on the full-attack ASR).

Q1c. Why train separate checkpoints instead of one jointly on both benchmarks?

A single checkpoint trained jointly on both benchmarks would mainly show the backdoor firing on data it was trained on, which the per-dataset checkpoints already demonstrate; we have not run that exact configuration. We report the more informative complementary test - cross-dataset generalization: an AdvBench-trained checkpoint achieves 45.0% triggered ASR on HarmBench without retraining (2.5% trigger-free ASR). A HarmBench-trained checkpoint likewise transfers to JailbreakBench at 29.0% triggered ASR and 0.0% trigger-free ASR. The transfer is therefore across three independently built benchmarks. This does not answer whether joint training would improve multitask coverage, which we leave open.

Q2. Is benign capability adequately measured, and was the original utility claim too strong?

We agree the 100-item SQuAD check was too thin to support a broad utility claim. We now evaluate 500 items each from BoolQ and SQuAD, comparing the clean (C00) and dormant-backdoor (C10) checkpoints paired per item:

BenchmarkClean (C00)Dormant (C10)Gap (C00 − C10)
BoolQ91.0%92.0%−1.0 pp
SQuAD90.4%89.0%+1.4 pp

Both gaps are within sampling noise, SQuAD's +1.4 spans zero and BoolQ shows no penalty, so the larger evaluation does not reproduce a real multi-point drop. We also agree that "largely preserved" was loose: the revised paper reports the measured gaps and their uncertainty directly rather than asserting equivalence of general capability.

AuthorsOpenReview ↗

Official Response Cont.

Q3. Do causal interventions extend to Qwen?

Yes. Following the reviewer's suggestion, we ran the full causal-intervention battery directly on Qwen3.8-Flash-Next, mirroring every intervention of the controlled-model study. Every intervention reproduces the controlled model's direction: substituting clean trigger addresses or closing the trigger-span gates collapses ASR to the clean-model floor (1.7% and 0.0%), and swapping gates and values between the clean and attacked models localizes the payload to the retrieved values rather than the gate scores — the attack follows the attacked values (48.1% with clean gates) and not the attacked gates (1.7% with clean values), along the same addressing → gating → value-side pathway. The one partial case is local-output replacement (21.2% on Qwen versus 0.08% on the controlled model), expected because the Qwen attack is fusion-only rather than narrow-joint. The full Qwen intervention battery, added to the revised paper, is below:

InterventionWhat it testsControlled ASR%Qwen ASR%
Normal attacked modelreference96.647.8
Matched-clean addressesaddressing necessary0.131.7
Trigger-span gates closedgated admission necessary0.320.0
Clean trigger-span memory outputslocal injection necessary0.0821.2
Clean gates, attacked projected valuesgate changes carry it?95.0748.1
Attacked gates, clean projected valuesvalue-side carries it?8.901.7

Q4. What is novel beyond a conventional backdoor objective?

We agree the optimization objective is standard and do not claim it as the contribution. The contribution is a previously uncharacterized native architectural attack surface: content-addressed gated memory can independently carry a trigger-conditioned backdoor while the backbone stays fixed. This target has properties a weight-space edit does not — it is conditional with a structural off-state (trigger absent, the gate stays closed and the model is behaviorally identical to clean, whereas a weight-space edit lies on the always-on path), and the architecture design treats it as a module separable from the dense weights. We demonstrate the mechanism causally on the controlled model and on Qwen, across two SOTA architectures and two independent triggers, touching 0.019% of parameters; and DeepSeek-V4.1-Flash shows a separate native-memory architecture compromised through table-entry modifications rather than fusion weights.

Closing note

We thank the reviewer for the positive assessment and for the concrete questions on scale, precision, and mechanism, which shaped the revision directly. We are happy to run further analyses during the discussion period should anything remain open.

Reviewer_X627OpenReview ↗

response to rebuttal

thanks for the rebuttal I will keep my score

Reviewer_DGNXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the security risks introduced by gated parametric memory in LLMs and proposes BADENGRAM, a post-training backdoor attack restricted to the gated-memory subsystem. The attack modifies memory-table entries, memory-specific fusion parameters, or both while keeping the backbone unchanged. The paper evaluates the attack in a controlled Engram model and on the Qwen3.8-Flash-Next model.

Critical strengths
  1. The paper studies a new attack surface introduced by gated parametric memory.

  2. The controlled Engram experiment includes detailed causal interventions that help identify how the backdoor is expressed.

  3. The paper evaluates the attack on a 176B model rather than limiting the experiments to small models.

  4. The paper includes cross-dataset evaluation and human validation of the harmful outputs.

Critical weaknesses
  1. The threat model is questionable. The attack already uses backpropagation through the model to optimize the gated-memory parameters. If the attacker has this level of access, they could simply enable gradients for other model parameters and update them as well. It is therefore unclear what prevents the attacker from modifying other specific parameters or modules, or even adding new modules to the model. Freezing the backbone appears to be an experimental restriction rather than a restriction imposed by the attacker's actual access. A comparison with attacks that modify an equally sized subset of backbone parameters would help establish whether gated memory is a particularly important attack surface.

  2. The paper does not provide direct experimental comparisons with other post-training attacks, including either backbone attacks or non-backbone attacks. It could compare against parameter-efficient methods such as LoRA or against equally sized parameter subsets from other parts of the model. Without these baselines, it is difficult to determine whether gated-memory parameters are especially vulnerable or whether similar backdoor performance could be achieved by modifying other small parameter groups.

  3. The strongest production-scale attack does not modify trigger-specific memory-table entries. On Qwen, the entire memory table is frozen, and the attack instead modifies fusion tensors. This makes it less clear whether the main vulnerability comes specifically from memory entries or more generally from modifying a small residual-injection module.

  4. An only memory table attack should have been evaluated. Modifying only the memory-table entries addressed by the trigger is the most direct form of a trigger-specific memory-table attack, but the Qwen experiment freezes the entire memory table and modifies only the fusion parameters. The only memory table attack seems to be the most important setting that should be evaluated.

  5. The detailed causal interventions are performed only on the small controlled Engram model. Similar interventions on Qwen would provide stronger evidence that the production-scale attack is operating through the same mechanism.

  6. Clean utility evaluation on Qwen is very limited. It uses only 100 SQuAD questions. A broader evaluation across several general-purpose benchmarks would provide stronger evidence that the attack preserves normal model behavior.

  7. The Qwen harmful-behavior evaluation uses a fixed jailbreak wrapper. It is therefore unclear how well the attack works under different prompt formats, system prompts, chat templates, or without the jailbreak wrapper.

  8. The integrity verifier mainly detects changes by checking the model artifact against a publisher-signed reference. If the publisher is trusted, the user could instead obtain the model directly from the trusted source; if the publisher, signing key, or build pipeline is compromised, the verifier does not detect the attack. Therefore, the proposed defense largely reduces to standard artifact-integrity checking rather than a defense specific to BADENGRAM.

  9. The paper evaluates transfer across datasets, but does not demonstrate transfer between independently trained models with native gated-memory systems. It would be interesting to test whether trigger-specific memory entries or attack parameters learned on one compatible model can be transferred to another model without retraining.

Questions for authors
  1. The attack already uses backpropagation through the model to optimize the gated-memory parameters. If the attacker has this level of access, what prevents them from enabling gradients for other parameters, modifying other modules, or adding new modules? What practical constraint justifies restricting the attacker to gated-memory parameters?

  2. How does BADENGRAM compare with modifying an equally sized non-memory parameter subset? For example, could the same attack objective applied to LoRA, MLP, attention, or another similarly sized parameter subset achieve comparable ASR and false activation?

  3. Why is there no table-only attack on Qwen? Since modifying only the memory-table entries addressed by the trigger is the most direct form of a trigger-specific memory-table attack, can the attack succeed when the fusion parameters are completely frozen?

  4. How robust are the Qwen results beyond the fixed jailbreak template and the limited clean-utility evaluation? Does the attack remain effective without the jailbreak wrapper, and does normal model performance remain stable across broader benchmarks?

8 comments
AuthorsOpenReview ↗

Official Response Weaknesses 1+2

We thank the reviewer for the specific suggestions on access assumptions, non-memory controls, and production-scale mechanism. The revised experiments add a second native-memory architecture, DeepSeek-V4.1-Flash (763B, native Engram), direct Qwen causal interventions, and broader benign-utility and transfer evaluations. We have also tightened the claims inherited from the earlier manuscript, particularly about independently writable memory and guaranteed dormancy.

Q1. What practical restriction justifies memory-only modification?

The reviewer is correct that an attacker with unrestricted write access could also modify backbone parameters, and that freezing the backbone is an experimental choice rather than a constraint on the attacker. We make that choice deliberately - to isolate what the gated-memory subsystem contributes on its own. Our contribution is not that gated memory is the only possible attack surface, but that it is by itself sufficient to enable persistent behavioral manipulation while leaving the backbone unchanged.

This distinction matters for white-box defenses. Youssef et al. [1] trace and reverse model edits by analyzing MLP projection matrices, and Puertolas Merenciano et al. [2] detect backdoored LoRA adapters using spectral statistics of updates to attention projections (Q, K, V, O). A memory-only modification leaves these inspected matrices and updates unchanged, placing it outside the evidence examined by those component-specific analyses. Other defenses, such as CROW [3], operate on internal hidden representations and mitigate backdoors through inter-layer consistency regularization; such defenses could potentially capture downstream effects of memory manipulation, but their effectiveness against gated-memory backdoors requires separate evaluation. White-box access alone therefore does not ensure coverage of every parameter subsystem. Our findings motivate explicitly including gated-memory parameters in model-integrity checks and security evaluations.

Q2. How does BadEngram compare with an equally sized non-memory modification?

We do not claim BadEngram is a stronger or more effective attack than a backbone-based modification. As stated in our reply to Q1, our contribution is not that gated memory is the most powerful attack surface, but that it is by itself sufficient to implant a persistent, trigger-conditional backdoor while leaving the backbone, and the parameter subsystems that existing white-box defenses inspect, entirely unchanged.

We nonetheless agree that a matched-budget comparison is valuable, and we ran it. We trained a LoRA on the backbone attention projections (q_proj/v_proj, 27.5M parameters, comparable to BadEngram's 32.8M fusion budget on Qwen3.8-Flash-Next) with the same backdoor objective, including the dormancy term, and evaluated it on HarmBench under the same trigger. The non-memory edit does carry the payload under the trigger (39.1% genuine ASR, against BadEngram's 47.8% on the same benchmark), confirming that a comparable-size backbone modification can also implant harmful behavior.

What distinguishes the two is the dormancy. The backbone LoRA has no clean off-state: it produces harmful responses at nearly the same rate with the trigger absent (32.2% genuine ASR) as present (39.1%), whereas BadEngram's memory-only backdoor is dormant, dropping to 0.9% without the trigger and activating only under it. Moreover, as detailed in our reply to Q1, an attention-projection LoRA of exactly this kind falls squarely within the coverage of existing white-box defenses — Puertolas Merenciano et al. [2], for instance, detect backdoored LoRA adapters from the spectral statistics of Q/K/V updates — whereas a memory-only modification leaves those inspected matrices unchanged. The non-memory baseline is thus both less stealthy and already on defenders' radar. We have added this baseline to the revised paper, summarized in Section 5.2 with the full setup and results in the appendix.

References

[1] Paul Youssef, Zhixue Zhao, Christin Seifert, and Jörg Schlötterer. Tracing and Reversing Edits in LLMs. arXiv:2505.20819, 2025; revised 2026 (ICLR 2026).

[2] David Puertolas Merenciano, Ekaterina Vasyagina, Kevin Zhu, Javier Ferrando, and Maheep Chaudhary. Weight-space Detection of Backdoors in LoRA Adapters. arXiv:2602.15195, 2026.

[3] Nay Myat Min, Long H. Pham, Yige Li, and Jun Sun. CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularization. ICML 2025, PMLR 267:44272–44291.

AuthorsOpenReview ↗

Official Response Weaknesses 3-5

Q3. Does the Qwen attack exploit memory specifically, or just a small residual-injection module?

Qwen's attack changes only the PLE fusion pathway while leaving its large memory table and the backbone unchanged. That pathway is not an arbitrary added adapter: its function is to transform and admit values retrieved through PLE's native addressing mechanism. Our new Qwen interventions show that the attack depends on that pathway. Replacing trigger-associated addresses with clean counterparts reduces ASR from 47.8% to 1.7%, and closing the trigger-span gates reduces it to 0.0%. Swapping gates and projected values independently further localizes the learned effect principally to the modified projected values, rather than the changed gate scores alone.

The added DeepSeek experiment supplies the complementary parameter locus: there, the attack modifies trigger-addressed memory-table entries rather than Qwen's fusion tensors, reaching 52.2%/56.0% triggered ASR on HarmBench / AdvBench. These are two different routes through native memory; whether a generic, equal-budget non-memory module behaves the same is exactly the matched-budget control in our reply to Q2.

Q4. Can a Qwen table-only attack work with fusion parameters frozen?

A fusion-frozen, table-only attack is not expected to succeed on Qwen, for architectural reasons. In Qwen's PLE the trigger-conditional computation lives in the fusion/gating pathway, while the memory table is an embedding store rather than a set of trigger-addressed entries. Admission of any retrieved signal into the residual stream is gated, and Qwen's native gate does not open for the trigger unless the fusion is modified. Freezing the fusion and training only the table therefore encodes a trigger-aligned signal that is never admitted - so no triggered effect should appear.

Nevertheless, prompted by the reviewer suggestion, we ran that experiment on Qwen: fusion and gate left at their pristine values, training only the table. The attack does not succeed — triggered ASR is 4.3% on HarmBench, a difference within noise and at the clean-model refusal floor. This is direct empirical confirmation of the reasoning above.

More importantly, we had already evaluated an isolated table-only attack in the setting where the question is most meaningful — an architecture whose memory exposes trigger-addressed entries. On the controlled Engram model, we freeze the fusion and gate and modify only the table entries; there the attack reaches 30.45% ASR with clean behavior preserved (99.95% clean accuracy, 0.00% false activation; the "Table only" row of our component-ablation table in the appendix). DeepSeek-V4.1-Flash, which incorporates Engram natively, is attacked through its table entries at production scale (52.2% / 56.0% ASR). Table-entry attacks therefore succeed precisely where the architecture admits trigger-addressed entries; Qwen's frozen-fusion null is the complementary confirmation that, where admission is gated shut, the table alone cannot drive the attack. We add the Qwen table-only result to the revised paper.

Q5. Do the controlled-model causal interventions hold on Qwen?

We have now run the full causal battery directly on Qwen3.8-Flash-Next, mirroring every intervention from the controlled-model study:

InterventionWhat it testsControlled ASR%Qwen ASR%
Normal attacked modelreference96.647.8
Matched-clean addressesaddressing necessary0.131.7
Trigger-span gates closedgated admission necessary0.320.0
Clean trigger-span memory outputslocal injection necessary0.0821.2
Clean gates, attacked projected valuesgate changes carry it?95.0748.1
Attacked gates, clean projected valuesvalue-side carries it?8.901.7

Every intervention reproduces the controlled model's direction. Substituting clean trigger addresses or closing the trigger-span gates collapses ASR to the clean-model floor (1.7% and 0.0%), so addressing and gated admission are both necessary. Swapping gates and values between the clean and attacked models then localizes the payload: attacked values with clean gates retain the attack (48.1%), whereas attacked gates with clean values do not (1.7%) — the effect follows the retrieved values, not the gate scores, along the same addressing → gating → value-side pathway.

The one quantitative departure is local-output replacement (21.2% on Qwen versus 0.08% on the controlled model); we do not claim identical magnitudes across scales. This is expected here because the Qwen attack is fusion-only rather than the narrow-joint edit used in the controlled study. Two rows of the controlled experiment, retrieval-necessity and whole-pathway replacement, are not meaningful on Qwen. With the table frozen and the backbone identical between the clean and attacked checkpoints, they reproduce the clean or attacked model by construction.

AuthorsOpenReview ↗

Official Response Weaknesses 6-7

Q6. Is normal Qwen performance preserved beyond 100 SQuAD questions?

Yes. We re-ran the benign-utility evaluation at 500 items per benchmark, five times the 100 the reviewer asks about, and added a second dataset, BoolQ, alongside SQuAD, comparing the clean model (no backdoor) against the dormant backdoored model (no trigger):

BenchmarkCleanDormant backdooredGap (pp)
BoolQ91.092.0−1.0
SQuAD90.489.0+1.4

Both gaps are within paired-bootstrap sampling noise (SE ≈ 0.9–1.2 pp), so the dormant backdoor leaves normal task performance intact beyond the 100-question scale and on more than one benchmark. We report the 500-item, two-benchmark evaluation in the revised paper.

Q7. Is the Qwen result dependent on the fixed jailbreak wrapper?

The concern is that the triggered ASR on Qwen could be an artifact of the prompt scaffolding — the fixed jailbreak wrapper, chat template, and system prompt — rather than the backdoor. We address it directly.

The backdoor resides entirely in the native gated-memory parameters (the PLE fusion pathway), not in the chat template, system prompt, or any prompt wrapper. The trigger is a content string in the user turn that addresses the memory table and opens the gate; the wrapper only sets the surrounding token context, so varying it changes the context the trigger appears in, not the mechanism that expresses it. This also separates BadEngram from template-level attacks, where the template itself is the vector.

We also ran a small-scale ablation to test this directly on Qwen. On 30 held-out AdvBench behaviors, we strip the jailbreak frame down to the bare harmful goal and present it with the trigger and no wrapper. Triggered ASR with the wrapper removed is 63%, statistically indistinguishable from the 60% measured with the wrapper on the identical behaviors — a few points, well within the confidence interval at this sample size. Removing the wrapper does not change the attack, so the effect is not an artifact of the jailbreak scaffolding.

Finally, our second production model is evaluated with no wrapper at all: DeepSeek-V4.1-Flash uses the plain benchmark goals and reaches 52.2% / 56.0% triggered ASR on HarmBench / AdvBench (versus 14.8% / 2.0% dormant) — the effect holds on a different architecture, under a different prompt format, with no jailbreak scaffolding.

We add the Qwen wrapper-removal ablation to the revised paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author Closing Remarks: Contribution Summary and Rebuttal Additions

We thank all three reviewers for their careful and constructive reviews.

BadEngram identifies native gated memory, a component now shipped in production-scale open-weight models, as a distinct and independently sufficient architectural attack surface, and provides the implantation procedure and causal characterization that establish it. Modifying only the memory subsystem, with every attention and MLP weight in the backbone left unchanged, is enough to implant a trigger-conditioned backdoor that stays dormant on all other inputs and activates only under the trigger. We establish this across three systems spanning two independent memory implementations and model scales from 14.2M to 763B parameters: a controlled Engram model, Qwen3.8-Flash-Next (176B, attacked through its PLE fusion tensors, 0.019% of parameters), and DeepSeek-V4.1-Flash (763B, attacked through its native Engram table entries). A causal-intervention battery, reproduced at production scale, localizes the effect to the memory pathway itself (addressing, gating, and value fusion) rather than to any backbone change. The security consequence is concrete: existing white-box backdoor defenses inspect backbone parameters and therefore do not examine this surface, so an unchanged backbone does not certify a model as safe.

During the discussion period we substantially strengthened the empirical basis of the paper. The additions are summarized below.

A second production-scale native-memory architecture. We added DeepSeek-V4.1-Flash (763B, native Engram), attacked through its trigger-addressed memory-table entries rather than Qwen's fusion pathway and evaluated with no jailbreak wrapper, reaching 52.2% / 56.0% triggered ASR on HarmBench / AdvBench (vs. 14.8% / 2.0% dormant). This extends the production-scale evidence beyond a single model and memory implementation.

Production-scale causal interventions. We ported the full causal-intervention battery to Qwen3.8-Flash-Next; every intervention reproduces the controlled model's direction along the addressing → gating → value-side pathway — closing the trigger-span gates reduces triggered ASR to 0.0%, and the payload follows the attacked projected values (48.1% with clean gates) rather than the gate scores (1.7% with clean values).

Matched-budget non-memory baseline. A LoRA of comparable size (27.5M parameters) on the backbone attention projections, trained with the same backdoor objective, carries the payload under the trigger (39.1% genuine ASR vs. 47.8% for BadEngram) but has no dormant off-state (32.2% trigger-free vs. 0.9%) and edits exactly the attention-projection weights that existing white-box backdoor defenses inspect.

Same-architecture checkpoint transfer. Transplanting the six attacked PLE tensors into an independently fine-tuned recipient, with no attack retraining, preserves potency (64% genuine triggered ASR vs. 0.9% for the clean recipient), while dormancy is partially checkpoint-specific (trigger-free ASR rises from ~1% to 25%).

Cross-benchmark generalization. Adding JailbreakBench as a third harmful-behavior benchmark, a HarmBench-trained checkpoint transfers to JailbreakBench (JBB-100) at 29.0% triggered ASR with 0.0% trigger-free leakage, complementing the AdvBench → HarmBench direction (45.0%); the attack therefore generalizes across three independently constructed benchmarks.

Expanded benign-utility evaluation. We replaced the single 100-item SQuAD checks with 500-item evaluations on BoolQ and SQuAD; the dormant backdoor shows no measurable degradation (gaps −1.0 pp and +1.4 pp, both within sampling noise).

All of these additions are incorporated into the revised paper (updated PDF on the submission page).

We thank the reviewers and the AC for their time. Full responses for each reviewer are provided in the individual threads below. We believe the demonstrated result is a solid and timely contribution, and we hope it is weighed on that basis.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.