- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies sleeper memory poisoning: adversarial external content causes an assistant to store a malicious memory that influences later sessions. It evaluates memory writing, retrieval, and downstream use across models and memory configurations, with behavioral and agent-action tasks, framework transfer, defenses, and selected production-system tests. Its main contribution is a systematic empirical account of how the attack progresses through these stages and how query proximity affects later use.
Critical strengths
Separating writing, retrieval, and use exposes distinct failure points that a single attack-success number would obscure. The cross-model and cross-framework experiments provide useful evidence about the persistence and transfer of poisoned memories.
The appendices address several essential checks: development-set selection in C.1, memory-pool and retrieval variations in G, exact-versus-fallback memories in I.2, and human audits of automated judgments in M.5.4.
Critical weaknesses
-
The claimed end-to-end rates cannot be reconstructed from the reported cohort and definitions. Appendix D.2 evaluates downstream effects using memories obtained through direct save requests, with another model's paraphrase substituted after refusals. It then describes Appendix I as an evaluation that carries the actual attack-written memory through the full chain. Table 43 needs to demonstrate that linkage. For GPT-5.6 Sol on goal-adjacent Agent Action, the full-context row reports IR = 93.0%, UR = 38.0%, and E2E = 35.3%. With the 100 examples specified in D.2.2, a directly measured all-attempt success rate must correspond to an integer number of successes. Moreover, 38.0% is not an integer success count out of 93 injected examples. Multiplying stage rates is valid when they are the corresponding conditional probabilities on the same cohort. The required check is whether the stage rates and memories refer to the same attack attempts. Supply the attempt-linked counts and memory provenance, explain any repetitions or weighting, and label cross-cohort products as estimates. This is necessary to substantiate the paper's central claim about the complete attack chain.
-
The Agent Action success label is broader than actual execution or a concrete execution attempt. Appendix M.5.4 assigns YES when the agent claims completion, asks for missing details, reads relevant files, or otherwise accepts and moves toward the goal. These outcomes have different security consequences. The reported 97.5% agreement on 80 audited judgments validates application of that broad label, not an execution-success rate. The action results should separate completed actions, tool-based attempts, and acceptance or information-seeking, using the existing calls and workspace diffs. A paired clean-memory comparison is also needed to establish how much of the measured activity is caused by the poisoned memory, especially when the user task already makes the relevant files or operations salient.
My weak-rejection recommendation rests on the unresolved end-to-end counts and the use of one success label for execution, attempts, and acceptance.
Questions for authors
-
For Table 43, can the authors provide per-attempt identifiers linking the external attack, the memory actually written, retrieval, and downstream use, together with integer counts and any averaging rules? How is the GPT-5.6 Sol 35.3% full-context result obtained from the stated 100-example subset?
-
How do the Agent Action rates change when completed actions, concrete tool attempts, and acceptance or requests for details are reported separately? On the same tasks, what is the difference when only the poisoned memory is removed?
2 comments
Thank you for your review (2/2)
Disaggregating AUR shows that direct execution dominates across every model on goal-adjacent tasks, reaching 80% for Gemini-3.1-Pro (out of 89% AUR), 67% for GPT-5.4 (out of 83%), and 61% for DeepSeek-v4-Pro (out of 80%); even for the most conservative model, GPT-5.6 Sol, 31% of its 38% AUR reflects concrete execution rather than mere clarification. At the same time, clarification represents a substantial secondary failure mode, particularly for Kimi-K2.6 (30%) and GPT-5.5 (25%), demonstrating that agents frequently treat poisoned memory directives as valid pending user intent and pause to confirm details rather than dismissing them. On goal-distant tasks, total AUR drops sharply to 0%–17%, with concrete execution falling to 0%–12% and clarification to 0%–5%. Across both subsets, non-usage remains predominantly silent: except Claude-Sonnet-4.6 (which explicitly refuses in 34% of goal-adjacent traces), models overwhelmingly ignore the injected goal rather than issuing overt refusals.
Paired clean-memory baseline (Appendix H.2, Table 37)
Following the reviewer's suggestion, we conducted a paired clean-memory baseline evaluation across all models and subsets. We used identical user queries with and without poisoned memories to isolate the effect of memory poisoning from the model's baseline behavior. This is particularly important for goal-adjacent queries, where the topic of the poisoned memory is already relevant to the user's request.
- Goal-Adjacent LLM Behaviour (Table below): Even without injected memory, models exhibit a non-trivial baseline Attack Usage Rate (AUR) of 20%–32% because the user prompt is semantically close to the target topic, occasionally causing natural responses to correlate with the target fact. However, injecting the poisoned memory induces a massive surge of +36 to +57 percentage points across every model, confirming that memory poisoning drives the overwhelming majority of observed behavior.
| Model | Clean Memory (Baseline) | Injected Memory | Causal Lift () |
|---|---|---|---|
| DeepSeek-v4-Pro | 30% | 85% | +55% |
| Gemini-3.1-Pro | 26% | 83% | +57% |
| Kimi-K2.6 | 32% | 78% | +46% |
| Claude-Sonnet-4.6 | 28% | 69% | +41% |
| GPT-5.6 Sol | 20% | 56% | +36% |
We took the conversations that already showed goal influence without the poisoned memory and compared each one with the same conversation after adding the poison. Pooled across models, the mean influence score increases from 2.46 to 2.68 (+0.22, 95% CI [0.01, 0.41]). Scores rise significantly more often than they fall (39 vs. 18; sign test p = 0.008). Four of five models show an increase: DeepSeek-V4-Pro 2.40→2.85, Kimi-K2.6 2.50→2.78, Claude-Sonnet-4.6 2.29→2.50, and Gemini-3.1-Pro 2.58→2.74. GPT-5.6-Sol shows no significant change (2.55→2.36, p = 1.0).
- Goal-Distant LLM Behaviour & Agent Action Subsets: We have run several tests across the goal-distant LLM Behaviour subset as well as both goal-adjacent and goal-distant Agent Action subsets; the clean-memory baseline AUR is ~0% for all models. Goal-distant conversations were designed to be completely unrelated to the goal; models have no contextual reason to spontaneously surface a completely unrelated fact without the memory being poisoned and the goal in their context steering them. It is very improbable for the model to surface the exact goal in conversations out of nowhere. Likewise, in Agent Action tasks, target behaviors require concrete tool calls that are misaligned (e.g., tampering with files, downloading and executing payloads, or exfiltrating data) that, even if the user’s task is goal-adjacent, the goals describe clearly misaligned/specific actions that are very improbable for a regular safety-trained LLM to perform those exact actions by chance without explicit steering from poisoned memory.
We have added the full paired clean-memory vs. poisoned-memory breakdown in Appendix H.2 (Table 37) of the revised manuscript.
Together, these results demonstrate that the observed agentic behavior is primarily driven by poisoned memory rather than the user queries alone, and that a substantial fraction of AUR corresponds to concrete actions or tool attempts rather than mere clarification.
We would like to thank the reviewer for their detailed review. We have run the suggested additional experiments, following the reviewer’s suggestion. The reviewer’s assessment focused on the unresolved end-to-end counts and the use of one success label for execution, attempts, and acceptance. We hope that our rebuttal has answered their questions. We would be happy to answer any further questions.
Thank you for your review (1/2)
We would like to thank the reviewer for their detailed review of the paper. We thank the reviewer for recognizing the value of our systematic decomposition of sleeper memory poisoning into memory writing, retrieval, and downstream use, which reveals distinct failure points. We are also pleased that the reviewer highlights the cross-model and cross-framework evaluations, as well as the additional robustness checks and human audits provided in the appendices.
“38.0% is not an integer success count out of 93 injected examples… label cross-cohort products as estimates.”
The reviewer is correct, and we thank them for catching this. The GPT-5.6 Sol cell contained a transcription error; the remaining entries in Table 43 are not cross-cohort products.
To clarify, Table 43 is computed per sample: each E2E entry represents the number of samples (out of 100) in which injection, retrieval, and adversarial usage all succeeded (Appendix I, L3268–3270).
For example, for GPT-5.4 on goal-adjacent Agent Action: 81 injected → 73/81 retrieved → 63/73 used → 63.0% E2E. Every other cell in Table 43 similarly corresponds to integer success counts.
For GPT-5.6 Sol, the full-context usage rate was mistakenly taken from the 100-sample standardized evaluation (Table 2) rather than conditioned on its 93 successful injections. The corrected counts are 35 of 93 used: UR = 37.6%, E2E = 35.0% (previously 38.0% / 35.3%). We have corrected Table 43 and the corresponding sentence in Appendix I.1.1 (marked in red). These corrections do not change our conclusions.
Regarding memory provenance, the post-injection evaluation uses each target model's own saved paraphrase of the adversarial goal (Appendix D.2, L1788–1797), with standardized fallback memories where necessary. Appendix I.2 shows that exact and fallback memories yield statistically comparable usage rates in 17 of 18 comparisons. We have also revised the sentence in Appendix D.2 (L1815–1817) to clarify this.
We thank the reviewer for this suggestion. We conducted two additional analyses to clarify the nature of downstream agent actions and isolate the effect of poisoned memories.
Disaggregating Agent Action outcomes (Appendix H.2, Table 38)
We thank the reviewer for this suggestion. We have updated our Agent Action evaluation and revised the corresponding table in the manuscript (Appendix H.3 / Table 38) to disaggregate Attack Usage Rate (AUR) into direct Execution (completed actions or concrete tool-call attempts toward the injected objective) versus Clarification (accepting or requesting further details/confirmation before acting), alongside non-usage behaviors (Refused and Ignored) across both goal-adjacent and goal-distant subsets: