- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
Prompt injection is a major security vulnerability in LLM agents, arising from the blurred boundary between untrusted data and instructions. While recent frontier models have become significantly more robust to conventional prompt injections, complete protection remains challenging due to the inherent ambiguity: documents may legitimately contain instructions, preferences, and workflow conventions that are difficult to distinguish from malicious directives.
This paper investigates a novel attack surface in which malicious instructions embedded in an artifact are incorporated into an agent's persistent memory and subsequently propagated through new artifacts consumed by other, independent agents. The authors employ iterative adversarial search to discover effective attack templates for individual models, effectively creating a cat-and-mouse game between attackers and increasingly robust models. Experiments across several LLMs demonstrate substantial rates of memory infection and multi-hop propagation, with considerable variation in robustness across models.
An important aspect of the experimental setup is endpoint-assisted propagation, which distinguishes two questions: (a) whether an attacker can induce an agent to retain malicious instructions and carry them into subsequent artifacts, and (b) whether agents can autonomously reproduce the complete malicious payload across multiple hops. The main experiments provide evidence for (a), while the reliable propagation of the full payload in (b) is facilitated by an attacker-controlled endpoint that regenerates the instructions at each hop. The paper additionally investigates propagation without this assistance and explores an independent classifier-based defense for detecting poisoned memories.
Critical strengths
-
Novel and practically significant attack surface. The paper identifies an important vulnerability in multi-agent workflows, where an agent acting on behalf of one user creates artifacts that are subsequently consumed by another user's agent. This is an increasingly realistic mode of information sharing as autonomous agents become more prevalent. Since users are unlikely to thoroughly audit every AI-generated document or piece of code, malicious instructions can propagate across otherwise independent agents without attracting attention. The paper highlights a consequential security risk that is not adequately captured by conventional single-agent prompt-injection evaluations.
-
Convincing empirical investigation and excellent presentation. The paper is exceptionally well written, with a clear threat model, intuitive examples, and a systematic experimental methodology. The authors develop an iterative search procedure for discovering effective attacks, evaluate the resulting templates on held-out scenarios across multiple LLMs, and characterize how malicious instructions persist and propagate over multiple hops. The substantial variation in vulnerability across models is particularly interesting. The additional experiments on endpoint-free propagation and memory-based detection further strengthen the investigation. Overall, the experimental evidence convincingly establishes the relevance of the proposed attack surface, even if some aspects of the propagation mechanism merit further investigation.
Critical weaknesses
None
Questions for authors
-
Endpoint-free propagation. The main experiments rely on endpoint-assisted propagation, where an attacker-controlled service restores the malicious payload at each hop. Appendix D evaluates endpoint-free propagation only on DeepSeek-V4-Flash, with considerably weaker results. Have the authors evaluated endpoint-free propagation on the stronger models considered in the main experiments? How much does propagation degrade without this assistance?
-
Infection versus execution of malicious behavior. The paper primarily measures whether malicious instructions persist in memory and propagate through artifacts. Appendix E extends this evaluation to security-critical goals, but does not appear to measure whether agents actually execute those malicious objectives. Have the authors observed examples of such downstream malicious behavior? I would expect the models' safety training to provide an additional layer of protection when a malicious instruction is translated into an actual action, even if it has already been stored in memory. It would be interesting to understand how often infection translates into successful execution of the adversarial goal.
3 comments
Thank you for the review [2/2]
Q2. Infection versus execution of malicious behavior
Appendix E shows that security-critical goals propagate, but not whether agents act on them. How often does infection translate into executing the adversarial goal, given the models' safety training?
We thank the reviewer for this question. Storing an instruction in memory is not the same as acting on it, so we have measured execution directly, in a new appendix (Downstream Impact of Injected Memories).
Setup. The security-critical goals we use in the paper are agent actions from Hidden in Memory (Pulipaka et al., 2026) [1]: operational sabotage, system hijacking, running attacker-chosen software, data exfiltration, and propagation to third parties. The paper also provides environments to test downstream impact, we use 100 environments from them. For each of the four main subject models, we place poisoned memory alongside benign memories and place the agent in the environment and evaluate. The queries given to the agent are benign user-queries which are slightly adjacent to the adversarial goal. The agent completes the task in a simulated OpenClaw environment, and an LLM judge reads the execution log to decide whether the agent performed the attacker-intended action.
| Target model | Adversarial usage rate (%) |
|---|---|
| GPT-5.6 Luna | 76.0 ± 7.0 |
| Kimi-K2.6 | 82.0 ± 7.5 |
| GPT-OSS-120B | 72.0 ± 9.0 |
| DeepSeek-V4-Pro | 80.0 ± 7.5 |
Rate at which the agent performs the attacker-intended action, given that the poisoned memory is retrieved into context. ± is the 95% bootstrap confidence-interval half-width.
Infection usually translates into execution. Once the poisoned memory is in context, every model carries out the attacker-intended action in 72–82% of queries. Like the reviewer, we expected safety training to act as a second barrier at this point, but in this setting the evaluated models usually believe the injected memory. We attribute this to how the instruction reaches the agent: it is retrieved as one of the user's own stored memories, next to benign ones, rather than arriving as an untrusted request, so the agent treats the action as a routine part of the task.
We describe this in Appendix G.
[1] Sidharth Pulipaka, Stanislau Hlebik, Leonidas Raghav, Sahar Abdelnabi, Vyas Raina, Ivaxi Sheth, and Mario Fritz. Hidden in Memory: Sleeper Memory Poisoning in LLM Agents. arXiv:2605.15338, 2026. https://arxiv.org/abs/2605.15338
We thank the reviewer again for these two questions, which prompted the endpoint-free evaluation and the downstream-impact study. We hope the new results address the concerns raised, and we are glad to provide further detail.
This is interesting. Thanks for the additional experiments!
Thank you for the review [1/2]
We thank the reviewer for their strong positive assessment of the paper’s clarity, informativeness, and soundness. We appreciate their recognition of (1) the novelty and practical significance of artifact-mediated attack propagation and (2) the systematic evaluation of multi-hop propagation across models and held-out scenarios.
Summary of our response
We thank the reviewer for two questions that both led to new experiments. We now evaluate endpoint-free propagation on all four main models (Q1): three of the four propagate to a third assistant in 31–39% of runs, while GPT-5.6 Luna resists the attack. We also measure whether stored adversarial memories are acted on (Q2): once the poisoned memory is retrieved, agents carry out the attacker-intended action in 72–82% of goal-adjacent queries. Details follow below.
Q1. Endpoint-free propagation on the main models
We thank the reviewer for this question. We have now evaluated endpoint-free propagation on all four main models, on the same 36 held-out test universes and with a single seed artifact per run.
An attack optimized for the prompt-only setting. To test an attack designed to propagate without any endpoint, we ran the same automated search for each model in the prompt-only setting, with the same budget of 250 evaluation jobs as an equal-budget endpoint-assisted search. We then froze each template and evaluated it on the 36 test universes.
(Observed S(h) for both attack variants; templates from the same 250-job search. Endpoint-assisted: payload restoration, prompt-only: no endpoint. 36 held-out universes, two runs each.)
Endpoint-free propagation is stronger on three of the four main models than on DeepSeek-V4-Flash. With a prompt-only template, GPT-OSS-120B, Kimi-K2.6 and DeepSeek-V4-Pro reach a third assistant in 31–39% of runs, and chains reach hop 4 on all three. Propagation is weaker than with endpoint assistance, but the artifact–memory–artifact mechanism sustains several successive transmissions with no external service restoring the payload. GPT-5.6 Luna, the most robust model in the main experiments, resists the prompt-only attack: the seed infects a first assistant in 20% of runs, but no chain reaches a second hop.
We have added these results to Appendix E, which previously covered only DeepSeek-V4-Flash.