acceptodds
Under review as a conference paper at ICLR 2027

Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections

15 views

Abstract

Self-evolving agents can carry information across sessions by updating their memories. While this capability can improve task execution, it also introduces a critical attack surface: once compromised, an agent can inadvertently carry adversarial information forward through its own state-update processes. In this work, we formulate persistent control as a distinct attack objective for self-evolving agents and introduce the Zombie Agent. Unlike conventional prompt injection, whose effects are typically bounded by the current session, we seeks to establish adversarial influence that persists across future sessions. Our core mechanism is self-reinforcement: following an initial exposure to adversarial content, the compromised agent is induced to repeatedly rewrite and propagate the payload into subsequent states, allowing the attack to survive memory evolution and be activated during later, unrelated sessions. The victim thereby becomes a latent “zombie” that actively preserves the attacker’s influence over time. We systematically evaluate Zombie Agent across two memory-level form of self-evolution, i.e., sliding-window and retrieval-augmented generation (RAG) memory, demonstrating sustained attack effectiveness and persistence over extended interactions. A configured OpenClaw case study also illustrates file-backed persistence. Our findings demonstrate that persistent control poses a security threat to self-evolving agents.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Self-Evolving Agent, AI Safety, Prompt Injection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_3FnqOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Zombie, which extends memory poisoning in LLM agents by getting agents to repeatedly refresh malicious instructions picked up from webpages. This can keep the instructions active across sessions and trigger unauthorized actions during later tasks.

Critical strengths

• Under a realistic threat model. The attacker controls external content without direct access to the agent’s memory, model parameters, or tools, which is more common in real environment.

• Poisoned behavior persistence. The paper measures repeated unauthorized actions during later benign tasks, focusing on the long-time control instead of traditional prompt injection.

Critical weaknesses

• Limited novelty or contribution over existing memory poisoning. The self-evolving setting in consists of memory updates while the model, workflow, and tools remain fixed. There are works already evaluate persistence under multiple update policies. The paper should compare the differences with such works, rather than present memory evolution or persistence alone as a new security problem.

Questions for authors

• Limited coverage of memory update mechanisms. Figure 4 compares raw history, reflection, and experience refinement, but the ASR drops sharply under the latter two conditions instead of showing generalization. The authors should compare more memory update policies and try to improve the effectiveness rather than only effective in the simplest raw history.

• User-side utility-preservation is not established. Table 4 reports substantial utility losses, including a drop from 76.5% to 47.6% for DeepSeek. Thus, I think it is easier to be noticed by the user with the low utility performance.

• What’s the persistant performance without renewed exposure. The paper describes a loop that revisits attacker-controlled content and writes it back into memory. The paper does not measure how long the stored injection remains effective after the attack source becomes unavailable or returns clean content. I think it heavily relies continued access to that source.

Reviewer_ZoeTOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates a novel security threat to LLM agents equipped with long-term memory and self-evolving capabilities: a cross-session, persistent-control attack dubbed the "Zombie Agent." Unlike traditional indirect prompt injection—which is typically confined to a single session—this attack exploits the agent’s own state-update mechanism. Through a "self-reinforcement" strategy, the attack embeds a malicious payload into the agent’s long-term memory; this payload is then repeatedly rewritten and reactivated across subsequent, unrelated, and otherwise benign sessions, enabling covert, long-lasting data exfiltration or unauthorized operations. The authors conduct extensive experiments across various memory architectures and LLM backbones, and—using the "OpenClaw" medical assistant as a case study—demonstrate the severe threat this poses to real-world deployments.

Critical strengths
  1. The paper astutely identifies new attack surfaces introduced by self-evolving agents. By extending the scope of attacks from single-session prompt injection to persistent, cross-session control, the work aligns perfectly with the current trajectory of agent systems, offering significant practical security implications and forward-looking insights.
  2. The authors have devised highly targeted persistence strategies for two prevalent memory architectures: they propose "Recursive Self-Replication" for systems utilizing sliding-window (FIFO) truncation, and "Semantic Aliasing via Broad Carrier Text"—complemented by a K-Center greedy selection algorithm—for systems based on RAG with sparse retrieval.
  3. Beyond systematically evaluating ASR, persistence, and the failure modes of various defense mechanisms, the paper presents exhaustive ablation studies and parameter sensitivity analyses. Furthermore, the inclusion of a real-world case study involving OpenClaw significantly enhances the paper’s persuasiveness, readability, and practical utility for engineering applications.
Critical weaknesses
  1. The paper assumes that an agent will automatically write observations from external web pages into its long-term memory via an evolution function. In practical agent systems, however, writing to long-term memory typically requires an assessment of importance or a check for contradictions. If a payload is deemed of low importance or conflicts with existing knowledge, the write operation will fail. The paper lacks a quantitative analysis of this write bottleneck, leading to an overestimation of the initial infection rate in real-world systems.
  2. To maintain persistent control, the Zombie Agent requires the agent to frequently execute specific tool calls during subsequent, unrelated sessions. Such high-frequency, patterned, and anomalous network requests or tool invocations would easily trigger statistical anomaly alerts within an enterprise's audit logs or EDR systems. The paper evaluates defenses only at the LLM layer, completely overlooking the impact of system-level monitoring on the attack's survival rate.
  3. In a RAG setup, the authors rely on proxy datasets and the K-Center greedy algorithm to generate carrier text that covers the semantic space. However, the paper fails to discuss how an attacker would obtain prior knowledge regarding the target agent's task distribution. Furthermore, the computational cost and feasibility of generating large volumes of high-quality carrier text are not addressed.
  4. Modern agents with long-term memory typically incorporate mechanisms such as memory decay, importance-based weighting, and memory deduplication or consolidation. The Zombie Agent relies on recursive self-replication and the injection of large amounts of repetitive data—approaches highly likely to trigger system deduplication mechanisms or result in the data being discarded by decay functions. The paper does not evaluate the survival rate of the attack when subjected to these advanced memory management mechanisms.
Questions for authors
  1. To what extent would the persistence strategy of a Zombie Agent (relies on high-frequency, repetitive injection) be undermined by the implementation of advanced memory management mechanisms, such as memory deduplication, temporal decay, and pre-write importance scoring?
  2. In the context of enterprise-grade system monitoring (e.g., network traffic auditing or EDR), how can a Zombie Agent—characterized by high-frequency and pattern-rigid anomalous tool invocations—evade statistical baseline alerts to maintain its stealth and survival?
  3. In a black-box scenario—where there is no prior knowledge regarding the task distribution of the target agent within its specific domain. how can an attacker efficiently construct carrier text to ensure that a malicious payload is successfully retrieved via semantic search, even when triggered by seemingly unrelated queries?
Reviewer_mSnTOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores an attack against self-evolving agents, where an outside attacker crafts a prompt injection which persists in the agent's memory. The authors propose self-reinforcement mechanisms, which allow the attack to persists as memory gets updated. To this end, they propose two techniques for improving persistency: Instructing the agent to self-replicate the attack, and adding carrier text that makes it more likely to be sampled in RAG systems. The authors then conducted experiments on 4 LLMs, where they evaluate the effectiveness, persistence, robustness to defence, and practical impact of their method.

Critical strengths
  1. The writing is quite good and easy to read. The problem and methods are well motivated.
  2. The experiments are quite interesting. They demonstrate that their attack is both more persistent and effective compared to existing attacks. I also like the experiments on an OpenClaw agent, though a more detailed study would have been interesting.
Critical weaknesses
  1. The actual contributions of this paper should be stated more explicitly and concretely. The paper claims of durable attacks, which has been done before, even under the same threat setting [1]. The main contributions seem to be the self-replication and the optimisation for RAG, i.e. a focus on attacking evolving architectures. The actual contributions of this paper should be stated more explicitly, and the method should be compared to more of the existing work.
  2. My main issue is that the method assumes two specific memory architectures, while in practice, many more exist. A more general approach, which doesn't rely on specific architectures, would be more interesting. It is not clear if the current approach would generalise to more complex memory architectures, which are more common in practice, such as compaction-based memory or agentic RAG.

[1] https://arxiv.org/pdf/2503.03704

Questions for authors
  1. Is it possible to extend the current method to more general assumptions about the memory architecture, i.e. an attacker with no knowledge about the underlying architecture?
  2. How do the payloads optimised using your method generalise to new architectures. Especially interesting could be to optimize them for the two considered ones and measure the effectiveness against held-out architectures.
  3. Can you clarify which memory architecture is used in the experiments , e.g. Table 1?
  4. Can you evaluate the experiments on OpenClaw in more scenarios? Do you observe similar results, or is it dependent on specific scenarios?
Reviewer_RTwvOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies persistent indirect prompt injection against agents whose memories evolve across sessions. Zombie Agent combines recursive payload refresh for sliding-window memory with broad carrier text for RAG retrieval, aiming to induce repeated unauthorized tool calls during later benign tasks. The evaluation covers four models, two memory architectures, several defenses, and a configured OpenClaw case study. A further experiment tests whether an initial compromise can support dynamically updated attacker commands.

Critical strengths
  • A meaningful behavioral objective. The paper evaluates repeated unauthorized actions on later benign tasks, rather than treating the presence of malicious text in memory as sufficient evidence of compromise. The dynamic-target experiment in Section 4.3 is particularly relevant: Gemini invokes an average of 16 distinct mock-tool commands across 20 scheduled targets, providing evidence of continued attacker-directed behavior, although the other models are substantially less reliable.
  • Evidence across memory interfaces and a concrete framework. Figures 3a and 5a show sustained attack behavior and payload retention in the evaluated sliding-window setting while ordinary injections decay. The OpenClaw case traces external content into MEMORY.md and then to unauthorized calls in subsequent interactions. Together, these results make the persistence risk concrete. The reported reductions under reflection and experience refinement also provide useful evidence about the attack's limits.
Critical weaknesses
  • The reported results do not reconcile across experiments. Tables 1 and 5 report standard RAG ASRs of 72.4% for command execution and 73.1% for exfiltration. Table 2 reports no-defense averages of 88.8% and 87.5% across both memory settings, while Section 4.4 states that the settings are unchanged. Equal weighting would require sliding-window ASRs above 100%, and Figure 3a instead ends around 83%. The paper does not explain a change in configuration or aggregation. This matters because every defense in Table 2 is measured against this no-defense row. Unless these numbers are reconciled, one cannot tell how much each defense actually helps.

  • The central self-reinforcement claim is not isolated in RAG. The main RAG experiment uses 300 exposure rounds followed by only 20 benign trigger rounds, and most of the storage growth in Figure 5b occurs during exposure. Table 7 does include a cumulative ablation, but adding recursive renewal changes command ASR from 51.7% to 55.0% and exfiltration ASR from 48.3% to 51.7%, on one model. These results show that the combined attack works, but do not establish how much persistence comes from autonomous refresh rather than initial pollution and semantic coverage. A comparison starting from the same infected memory and changing only renewal is needed to substantiate the paper's central mechanism.

  • The adapted baselines do not isolate the mechanism advantage. Section 4.2 attributes the weak InjecMEM and MemoryGraft results partly to model-transfer and objective/actionability mismatches. The paper does not sufficiently describe the adaptations, matched payload and exposure budgets, or tuning opportunities. Consequently, the large ASR gap in Table 1 may reflect how well the payloads target the evaluated tool actions, as well as differences in persistence.Prior work already shows memory poisoning can persist, so the novelty here is the refresh mechanism. The baseline comparison cannot show that refresh is the key to make the difference as the adapted baselines may simply have weaker payloads.

Questions for authors
  • Result consistency. Can you reconcile Table 2's no-defense row with Tables 1 and 5 and Figure 3, reporting the per-model, per-memory results, sample counts, and aggregation rule for each experiment?
  • Refresh attribution. Starting from identical infected RAG memory, how do per-round ASR and newly written malicious records differ with renewal enabled versus disabled during subsequent benign tasks? Please also distinguish memory-only persistence from refresh through revisiting the attacker-controlled page.
  • Baseline alignment. What exactly was changed when adapting MemoryGraft and InjecMEM? Were the action objectives, exposure and payload budgets, and tuning opportunities matched, and can the adapted payloads and protocols be provided?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.