acceptodds
Under review as a conference paper at ICLR 2027

EvoAttack: Adaptive Strategy Learning for Agent Attacks with Evolving Memory

Abstract

Memory allows LLM agents to carry information across interactions, making their behavior increasingly shaped by an evolving internal state. This same capability makes attacking memory based agents fundamentally nonstationary, since the effectiveness of an attack strategy can change with both the target memory state and subsequent interactions. Existing memory attacks primarily focus on injecting or sustaining malicious information, while adaptive attack methods refine attacks from local feedback, but neither explicitly learns how strategy utility evolves with the target. We propose , which formulates attacks on memory based agents as adaptive strategy learning against an evolving target under black box access. EvoAttack infers the target state from interaction feedback and assigns temporal credit using attack outcomes observed over multiple horizons. It then maintains an attacker memory that links target states, strategies, and realized rewards, turning previously isolated attack outcomes into reusable utility knowledge. Retrieved memories serve as a nonparametric critic to estimate state conditioned strategy advantages and directly recalibrate the LLM attack policy without parameter training. Extensive experiments across diverse agent domains, model backbones, initial memories, and defensive settings show that EvoAttack consistently improves attack effectiveness across both target states and evaluation horizons. These results suggest that effective attacks on memory-based agents require learning how strategy utility changes with the evolving target, rather than repeatedly optimizing against a fixed target condition.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agents, Persistent Attacks, Adaptive Strategy Learning, Attack Memory

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_E5QtOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper studies attacks on memory-based agents with evolving memory. It proposes EvoAttack, which uses interaction feedback and attacker memory to learn state-conditioned strategy utilities and adjust the attack policy. Experiments show improved attack effectiveness across target states and persistence horizons.

Critical strengths
  1. The experiments are comprehensive. The paper evaluates 10 agents across four domains, different model backbones, initial memory settings, and defense mechanisms. It also includes ablation studies and evaluates attack persistence over multiple horizons.

  2. The paper is well organized and easy to follow. The motivation and problem formulation are clearly presented. The methodology is also explained step by step.

Critical weaknesses
  1. The novelty seems limited. The target state is constructed from simple feedback statistics and attack history. The method then retrieves historical rewards to adjust strategy preferences. These components largely build on existing memory retrieval and reward-based strategy selection techniques. The methodological contribution is therefore not sufficiently clear.
  2. The paper lacks comparisons with closely related methods. AutoDAN-Turbo (ICLR 2025) and AutoRedTeamer (NeurIPS 2025) also use historical attack experience to guide subsequent attacks. Without these comparisons, it is difficult to assess the actual benefit of the proposed attacker memory over existing methods.
Questions for authors
  1. Why is AutoDAN-Turbo (ICLR 2025) not included in the comparison? It is a more recent SOTA method and is more closely related to EvoAttack.
  2. The improvement from single to multi-attack is smaller for EvoAttack than for PAIR and AutoDAN. Why do the authors choose only three attack rounds? Would its advantage persist with more attack rounds?
  3. AutoDAN-Turbo (ICLR 2025) and AutoRedTeamer (NeurIPS 2025) also use historical attack experience to guide strategy selection. What is fundamentally new in EvoAttack beyond state-conditioned retrieval and multi-horizon rewards?
6 comments

W1. Methodological Novelty and Theoretical Analysis

We thank the reviewer for this insightful comment and apologize for the insufficiently clear presentation of our methodological contributions.

The key novelty of EvoAttack is not the individual operators, but a principled formulation of persistent memory attacks as test-time, gradient-free reinforcement learning against partially observable and nonstationary targets. Unlike conventional adaptive attacks that optimize prompts based on immediate feedback, EvoAttack addresses a dynamic learning problem in which strategy effectiveness depends on an evolving and unobservable target memory, and successful attacks may lose their effectiveness through subsequent interactions.

To address this challenge, EvoAttack establishes a reinforcement learning framework without gradient updates or parameter training. Specifically, the interaction-inferred target state provides an observable representation of the partially observable environment, the selected attack strategy serves as the action, and multi-horizon persistence outcomes provide delayed rewards. Historical attack experiences are maintained as a nonparametric critic that estimates state-conditioned strategy values and advantages. Rather than merely retrieving successful attacks, EvoAttack uses these advantage estimates to directly improve the LLM strategy policy through logit recalibration. This forms a closed reinforcement learning loop in which attack experience continuously updates the policy while the target memory evolves.

Importantly, this formulation is supported by theoretical analysis rather than relying solely on a heuristic combination of existing components. We have added a new Theoretical Analysis section in Appendix A, establishing three results:

(1) Exact policy optimization. We prove that EvoAttack's logit update is the unique closed-form solution to a KL-regularized policy optimization problem: πr∗(s)∝π0,r(s)exp⁡(λA^r(s)).\pi_r^*(s)\propto\pi_{0,r}(s)\exp(\lambda\widehat A_r(s)). This yields exactly zr′=zr+λA^rz'_r=z_r+\lambda\widehat A_r, demonstrating that our gradient-free policy recalibration implements an explicit reinforcement learning optimization objective rather than an arbitrary adjustment of strategy preferences.

(2) Reliability of the nonparametric critic. We derive a finite-sample estimation bound that decomposes critic error into state mismatch, target drift, hidden-state ambiguity, and sampling uncertainty. This characterizes when the lightweight interaction-derived state representation supports reliable strategy evaluation and explains how state-conditioned experience retrieval can reduce estimation bias under partial observability and nonstationarity.

(3) Policy tracking under nonstationarity. When the critic estimation error is bounded by εr\varepsilon_r, we prove:

\leq 2\lambda\varepsilon_r, \qquad F_r(\pi_r^{\mathrm{oracle}})-F_r(\widehat\pi_r) \leq 2\varepsilon_r.$$ These bounds establish that accurate state-conditioned value estimation enables the gradient-free policy to remain close to the current target-specific optimum, even as the target condition changes. The experiments under A-MemGuard in Section 4.5 further support the practical effectiveness of this adaptive mechanism under memory-based defenses. Together, these results establish EvoAttack as more than historical reward retrieval and strategy preference adjustment. Its methodological contribution lies in realizing a theoretically grounded, gradient-free reinforcement learning framework that connects partial state inference, delayed reward assignment, nonparametric value estimation, and direct policy improvement. This enables persistent strategy learning against evolving memory-based agents without updating model parameters.

W2 & Q1. Comparisons with Lifelong Adaptive Attack Methods

We thank the reviewer for this valuable suggestion. Our original baseline selection focused on specialized memory attacks and representative adaptive attack optimization methods. AutoDAN-Turbo primarily addresses lifelong jailbreak strategy discovery, while AutoRedTeamer focuses on autonomous red teaming through continuous discovery and integration of attack methods. Their original objectives and evaluation settings differ from our memory persistence benchmark, which motivated our initial baseline selection.

Nevertheless, we agree that comparisons with more recent lifelong adaptive methods are important. We have therefore extended the main performance comparison in Section 4.2 (Tables 1 and 2) to include AutoDAN-Turbo and Evo-Attacker, another recent memory-augmented adaptive attack method. We evaluate them using the same target agents, attack budgets, and persistence evaluation protocol.

The results averaged across four domains are summarized below.

MethodSingle P@0Single P@3Single P@5Multi P@0Multi P@3Multi P@5
AutoDAN-Turbo78.0454.7139.5986.3871.5861.65
Evo-Attacker75.5451.4035.4884.3868.5658.04
EvoAttack78.4656.1744.5888.9674.8865.75

EvoAttack outperforms both baselines across all persistence horizons. Compared with AutoDAN-Turbo, it improves P@5 by 4.99 percentage points in the single attack setting and 4.10 points in the multi attack setting. Notably, the single attack advantage increases from 0.42 points at P@0 to 4.99 points at P@5.

These results suggest that EvoAttack's advantage lies particularly in maintaining attack effectiveness after subsequent benign interactions. While lifelong strategy exploration improves attack success, EvoAttack additionally learns the state-dependent long-term utility of strategies, enabling it to favor attacks with stronger persistence rather than optimizing primarily for immediate success.

Q2. Effectiveness Under More Attack Rounds

We thank the reviewer for this important question. We originally evaluate one and three attack rounds to assess attack effectiveness under limited interaction budgets, where efficient strategy selection is particularly important. The 3 round setting allows feedback-driven adaptation while avoiding excessive repeated attempts that may obscure differences in strategy learning efficiency. To examine whether our findings depend on this choice, we further extend the evaluation to five attack rounds while keeping the target agents and evaluation protocol consistent.

The P@5 results, weighted across the four domains according to their numbers of agents, are summarized below.

Method1 Attack3 Attacks5 AttacksΔ 1→3Δ 3→5
PAIR29.5554.1064.35+24.55+10.25
LLM-Fuzzer23.5545.5054.65+21.95+9.15
AutoDAN32.9557.9068.30+24.95+10.40
AutoDAN-Turbo39.2060.7070.15+21.50+9.45
Evo-Attacker35.0557.0065.85+21.95+8.85
EvoAttack44.2564.8575.05+20.60+10.20

First, the smaller absolute improvement from one to three rounds does not indicate weaker adaptation. EvoAttack already achieves substantially higher persistence with a single attack, reaching 44.25% compared with 29.55% for PAIR and 32.95% for AutoDAN, leaving less room for further improvement. More importantly, when extending from three to five rounds, EvoAttack gains another 10.20 percentage points, comparable to PAIR at 10.25 points and AutoDAN at 10.40 points. This demonstrates that EvoAttack continues to benefit from additional interactions rather than exhibiting weaker adaptation as the attack budget increases.

Second, EvoAttack consistently maintains its advantage across different attack budgets. At 5 rounds, EvoAttack achieves 75.05% P@5, outperforming the strongest baseline, AutoDAN-Turbo, by 4.90 percentage points. This margin is slightly larger than its 4.15 point advantage at three rounds. These results demonstrate that EvoAttack continues to improve with additional attack opportunities and that its superiority does not depend on the specific choice of three attack rounds.

Finally, stronger performance with fewer attack rounds is itself an important advantage. As the attack budget increases, repeated attempts provide all methods with additional opportunities to improve, potentially narrowing performance differences as success rates approach higher levels. We therefore emphasize one and three rounds in the main evaluation to better distinguish the effectiveness of strategy learning under limited interaction budgets. The additional 5 round results further confirm that EvoAttack achieves stronger persistence with fewer attempts while maintaining its leading performance as more attack rounds become available. Together, these findings demonstrate both the interaction efficiency and robustness of EvoAttack across different attack budgets.

Reviewer_8DUxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper introduces EvoAttack, a black-box adaptive strategy-learning framework for attacking LLM agents with evolving long-term memory. Because agent memory shifts across interactions, attack effectiveness is both state- and time-dependent.

  • Contributions: EvoAttack infers a proxy target state from interaction feedback and assigns temporal credit across multiple persistence horizons. It maintains an attacker memory of (state, strategy, reward) tuples as a non-parametric critic to estimate strategy advantages and directly recalibrate the attacker LLM's logits without parameter training.

  • Validation: Evaluated across 10 agents in 4 domains, multiple LLM backbones, initial memory conditions, and defenses, showing consistent attack gains across target states and evaluation horizons.

Critical strengths
  • Dynamic Persistence Evaluation Across States and Horizons: Moving beyond immediate, static attack success, the paper evaluates how attack utility varies across preexisting memory states and subsequent benign interactions (P@0,P@3,P@5P@0, P@3, P@5). This is consequential because it captures the real-world non-stationarity of long-running memory agents and reveals how specific semantic framing strategies survive memory dilution over time.

  • Training-Free Policy Recalibration with Reusable Strategy Memory: Using a structured (state, strategy, reward) memory as a non-parametric critic to directly adjust LLM strategy logits converts delayed, multi-horizon feedback into immediate policy updates without parameter training. This is consequential because it provides a lightweight mechanism to accumulate reusable attack knowledge that transfers effectively across tasks and tools.

Critical weaknesses
  • Degenerate Target State Conditioning in Single-Attack Experiments: Because target susceptibility is reset to v0=0v_0 = 0 and history $\mathcal{L}1 = [] $ at the start of every episode, every single-attack decision (r=1r=1) occurs under an identical initial state (∣vi−v1∣=0\vert{}v_i - v_1\vert{} = 0 and dedit([],[])=0d_{\text{edit}}([], []) = 0). This is consequential because the proposed target state proxy xr=(vr,Lr)x_r = (v_r, \mathcal{L}_r) is mathematically inactive across the majority of the paper's experiments (all single-attack tables), reducing the method to a static task/tool lookup and contradicting the single-attack state-conditioning ablation.

  • Unrealistic Black-Box Threat Model and Evaluation Protocol: Training relies on resetting the victim's memory 10 times per task, issuing read-only oracle probes (Br,P@kB_r, P@k) that bypass the agent's memory-update mechanism (Mt+1=Mt⊕mt\mathcal{M}_{t+1} = \mathcal{M}_t \oplus m_t), and stripping normal tool observations during observation-stage injection. This is consequential because a real black-box attacker cannot rewind a persistent victim's memory or probe it without altering Mt\mathcal{M}_t, meaning the reported gains depend on oracle privileges unavailable in deployment.

Questions for authors
  • State Conditioning in the Single-Attack Setting: Because v0=0v_0 = 0 and L1=[]L_1 = [] are reset at the start of every episode, every single-attack decision (r=1r=1) has an identical state (∣vi−v1∣=0\vert{}v_i - v_1\vert{} = 0 and dedit([],[])=0d_{\text{edit}}([], []) = 0 for all entries). How does (vr,Lr)(v_r, \mathcal{L}_r) differentiate strategies in the single-attack setting, and what exact condition was removed in the "w/o State Conditioning" single-attack ablation (Figure 4) to cause a drop in P@0P@0?

  • Continual Non-Resetting Evaluation & Probe Side-Effects: How does EvoAttack perform in a realistic continual stream where the target memory is not reset across the 10 episodes, and where probes (Br,P@kB_r, P@k) are standard black-box queries that write to Mt\mathcal{M}_t via Eq. (2) rather than read-only oracle checks?

  • Observation-Stage Threat Model & Baseline Parity: Appendix C.2 notes that observation payloads are appended to target memory after removing ordinary tool observations. How is this achieved under a black-box interface, and were adaptive baselines (PAIR, LLM-Fuzzer, AutoDAN) given this same observation-injection capability?

  • Missing Equations & Hyperparameters: What is the exact formula for the optimistic uncertainty bonus for unvisited strategies (Sec. 3.4), what value of λ\lambda is used in Eq. (11), and why is k=200k=200 in Table 6 when 10 unshared episodes produce fewer than 200 tuples per agent?

  • Missing Baselines & Model Identity: Why were lifelong adaptive attack methods cited in Section 5 (AutoDAN-Turbo, Evo-Attacker) omitted from the baselines, and what exact model checkpoint corresponds to GPT-5.6-Luna?

Minor Corrections / Future Work:

  • Fix the typo in Section 5 (line 475) referring to Yan et al. (2026) as EvoAttack instead of Evo-Attacker, and deduplicate the six repeated bibliography entries.
  • Evaluating beyond Mem0 (e.g., A-MEM or MemGPT) would help verify that the state proxy generalizes across memory architectures.
8 comments

W1 & Q1. Clarification of State Conditioning in Single Attack.

We thank the reviewer for this careful observation and apologize for the misunderstanding caused by our insufficiently clear description of the ablation setting.

Although the inferred target state is initialized as x1=(0,[])x_1=(0,[]) in each single attack episode, this does not make attack memory retrieval or strategy learning inactive. As described in Appendix B.4, attacker memory persists across episodes and accumulates experiences from different tasks and target tools within each agent. Therefore, even under an identical initial target state, EvoAttack can continuously improve strategy selection using historical attack outcomes.

Importantly, our memory retrieval conditions on both attack context and inferred target state, as defined in Eq. (8):

\land |v_i-v_r|\leq\epsilon.$$ In the single attack setting, the fixed initial state represents a valid target condition before the current attack. While the state variables provide no additional discrimination when all historical records share this initial state, task and tool matching still identifies relevant experiences for estimating strategy effectiveness. Crucially, the *w/o State Conditioning* ablation in Section 4.4 estimates strategy advantages over the entire attacker memory instead of the contextually matched memory neighborhood. This mixes historical rewards from unrelated tasks and target tools, introducing irrelevant experience into strategy evaluation. Thus, single attack experiments evaluate contextual strategy learning under a controlled initial target condition, whereas multi attack experiments additionally evaluate adaptation to dynamically evolving target states. These two settings assess complementary aspects of EvoAttack rather than requiring state variation within every attack episode. To avoid ambiguity, we have renamed the ablation from "w/o State Conditioning" to "w/o Context and State Conditioning" in Section 4.4 and Figure 4, more accurately reflecting the removal of both context matching and target state conditioning.

W2 & Q2. Black Box Threat Model and Evaluation Protocol.

(1) Rationale for Resetting Target Memory and Continual Memory Evaluation

The original episodic reset protocol is designed to ensure a fair and interpretable comparison of attack strategies. In our test time reinforcement learning framework, attacker memory plays a role similar to trainable model parameters, allowing strategies to improve through accumulated experience. If target memory is preserved across episodes, previous attacks may continue to influence the target, keeping attack success rates high even when subsequent strategies become less effective. We therefore reset target memory while preserving attacker memory to distinguish improvements from strategy learning from the cumulative effects of previous attacks. This is an evaluation control, not a capability granted to the attacker.

Experiment 1: Continual Target Memory. To evaluate a more realistic deployment setting, we also conduct experiments where both target and attacker memories persist across episodes. The single attack P@5 results over ten episodes are shown below.

Method / Episode12345678910
EvoAttack39.3547.2550.8553.9555.3556.3557.2057.9058.3058.85
PAIR22.3531.0534.8038.1540.3040.9540.8040.5540.3040.15
LLM-Fuzzer19.9027.7031.3533.8035.6536.1036.0535.7035.5535.50
AutoDAN25.4033.7037.5041.0543.2544.0543.8543.7543.3043.05

We observe that all methods initially benefit from preserving target memory, confirming that accumulated attack influence can contribute to higher success rates independently of strategy improvement. However, while the baselines plateau or slightly decline after Episodes 6–7, EvoAttack continues to improve, reaching 58.85% P@5 and exceeding the strongest baseline by 15.80 percentage points. This suggests that continued strategy learning provides additional benefits beyond memory accumulation and confirms that EvoAttack remains effective without episodic target resets.

(2) Continual Memory with Standard Black Box Queries

The original read only probes are also part of the controlled evaluation, preventing the measurement process from introducing additional memory changes. However, ordinary queries to deployed agents may update their memories. We therefore evaluate whether EvoAttack remains effective without this restriction.

Experiment 2: Standard Queries with Memory Updates. We retain target and attacker memories across episodes and replace read only probes with standard agent queries that follow the normal memory update process. Each episode contains three attack rounds, with probes providing feedback for subsequent strategy selection. All methods use the same interaction protocol and attack budget. The multi attack P@5 results at Episode 10 are shown below.

MethodRead Only ProbesStandard QueriesChange (pp)
PAIR66.2563.45−2.80
LLM-Fuzzer59.1055.90−3.20
AutoDAN70.8068.60−2.20
AgentPoison58.2056.50−1.70
ZombieAgents69.6070.10+0.50
MINJA71.4571.00−0.45
EvoAttack79.2583.10+3.85
Improvement over Best Baseline+7.80+12.10+4.30

We observe that EvoAttack improves from 79.25% to 83.10% when probes are allowed to update target memory, while most baselines experience slight declines. Its advantage over the strongest baseline increases from 7.80 to 12.10 percentage points. This is consistent with EvoAttack's ability to adjust strategy preferences using feedback from evolving target conditions, even when the probes themselves introduce additional memory changes.

These two experiments clarify the distinction between evaluation fairness and deployment applicability. The original protocol controls memory accumulation and probe effects to measure strategy quality reliably, while the additional experiments verify EvoAttack's effectiveness when target memories persist and ordinary interactions update them. Thus, neither episodic resets nor read only probes are necessary for EvoAttack to remain effective in the evaluated black box setting.

Q3. Observation Stage Threat Model and Baseline Parity.

We thank the reviewer for this question and apologize for the ambiguity in our description of observation attacks.

Observation attacks do not grant the attacker direct access to or control over the target's memory. Instead, they simulate a realistic scenario where the target agent receives untrusted information from an external tool or data source. The attacker controls the content of this external observation, not the target's memory storage. As defined in Eq. (2), the target agent processes the observation through its normal execution and memory update procedure, which may subsequently incorporate the malicious information into persistent memory. Thus, the attack influences memory indirectly through external interactions, consistent with our black box threat model. The wording in Appendix D.2 describes the construction of the injected observation, not an operation that directly edits the target's stored memory. We have clarified this distinction to avoid confusion.

Regarding baseline fairness, as described in Section 4.1 and Appendix B.3, all adaptive baselines are evaluated using the same 14 attack strategies, covering both query and observation stages, with identical target interaction budgets and feedback. Their original optimization mechanisms are preserved, ensuring that performance differences reflect strategy learning rather than unequal attack opportunities.

Q4. Missing Equations and Hyperparameters.

We thank the reviewer for pointing out the need for clearer implementation details. We have revised Section 3.4 to explicitly specify the uncertainty bonus and logit scaling coefficient. For strategies without retrieved experience, we assign an optimistic value:

\frac{\eta}{\sqrt{|\mathcal N_r|+1}}, \qquad |\mathcal N_r(s)|=0,$$ where $\eta=1.0$ and $\widehat V_r=0$ when the retrieved memory is empty. This encourages exploration of unvisited strategies, with the bonus decreasing as relevant experiences accumulate. We also clarify that $\lambda=0.5$ is fixed across all experiments and controls the strength of logit recalibration. Regarding the retrieval budget, $k=200$ specifies the maximum number of retrieved experiences, not the total size of attack memory. As described in Appendix B.4, attacker memory accumulates across episodes, tasks, and target tools within each agent, rather than being limited to ten episodes of a single task. When fewer than 200 eligible records are available after context and state filtering, all are retained. These clarifications make the policy update procedure and its implementation settings explicit.
Reviewer_dxQ6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper formulates attacks on memory-based LLM agents as adaptive strategy learning against an evolving target. The key observation is that attack effectiveness is both state-dependent (varying with the target agent's memory condition) and time-dependent (changing as subsequent interactions reshape the memory). The proposed method, EvoAttack, maintains an attacker-side memory that stores tuples of (inferred target state, chosen strategy, temporally-assigned reward). A target state representation is constructed from interaction feedback (susceptibility score and attack history) without accessing the agent's internal memory. Temporal credit assignment distributes episode-level persistence rewards to individual attack rounds. Retrieved attack memories serve as a nonparametric critic to estimate state-conditioned strategy advantages, which are injected as logit biases into the attacker LLM's strategy-selection distribution. Experiments on Agent Security Bench (10 agents across 4 domains with 40 attack tools) show improvements over both specialized memory attacks (MP, AgentPoison, ZombieAgents, MINJA) and adaptive strategy optimization baselines (PAIR, LLM-Fuzzer, AutoDAN) on P@0P@0, P@3P@3, and P@5P@5 metrics. Additional evaluations demonstrate generalization across target LLM backbones (Qwen3-32B, Llama3.3-70B, GPT-5.6-Luna), robustness to different initial memory conditions, and maintained effectiveness under the A-MemGuard defense.

Critical strengths
  1. Well-motivated problem framing: The explicit separation of state dependence and temporal dependence in attack effectiveness is a genuine conceptual contribution. The paper convincingly argues (with supporting evidence in Tables 1–2) that no single fixed strategy dominates across all memory states and persistence horizons, which motivates the adaptive approach. This shifts the conversation from "which attack strategy is best" to "how should an attacker select strategies as the target evolves."

  2. Comprehensive evaluation with meaningful ablations: The evaluation spans 10 agents across 4 domains, 3 target LLM backbones, multiple initial memory conditions, a defense (A-MemGuard), and cross-task generalization (Table 9 in the appendix). The ablation study (Figure 4) cleanly isolates the contribution of state conditioning, reward-based learning, and logit recalibration, showing that removing reward causes the largest degradation—validating that the method learns from realized outcomes rather than relying solely on semantic plausibility.

Critical weaknesses
  1. Incremental methodology over existing components: Each individual component of EvoAttack—memory retrieval conditioned on context, advantage estimation from retrieved examples, logit biasing for policy adjustment—has clear precedents in the LLM agent and bandit literatures. The paper combines them in a reasonable way, but the combination itself is straightforward: retrieve similar past experiences, compute an advantage, add it to logits. There is no theoretical analysis of why this combination should work (e.g., convergence properties of the nonparametric critic, conditions under which the susceptibility proxy is informative). This makes the contribution feel more like a well-engineered system than a principled advance.

  2. Evaluation protocol makes reproducibility and interpretation difficult: The P@kP@k metric depends on a specific interleaved protocol (A→B→A→B→…A \to B \to A \to B \to \ldots with probes at fixed positions), where benign interactions DiD_i are generated by running the same task without the attack tool. Small changes in protocol—probe timing, number of benign interactions, memory retrieval parameters—could substantially affect measured persistence. The paper does not study sensitivity to these protocol choices. Additionally, the reported improvements, while consistent, are often within a few percentage points of the best baseline on individual domain/horizon combinations, and no confidence intervals or significance tests are provided for the main results (Tables 1–2).

Questions for authors
  1. How sensitive are the results to the susceptibility-score hyperparameters (α\alpha, cc, ϵ\epsilon) and the retrieval budget kk? The current settings are fixed across all experiments—would per-domain tuning substantially change the comparison with baselines?

  2. The target memory is reset each episode. Have you evaluated EvoAttack when the target memory persists across episodes (i.e., a truly long-running agent)? This seems like the most natural deployment scenario for the problem you motivate.

  3. Table 9 shows cross-task generalization. When attack experience from the current task is excluded, EvoAttack preserves performance while adaptive baselines degrade. Can you clarify what information in the transferred memories is actually useful—is it the strategy-advantage estimates, or primarily the state-representation similarity matching?

  4. The logit recalibration applies a uniform scaling λ\lambda to all strategy advantages. Have you considered state-dependent scaling, or does the uniform approach already capture sufficient variation through the advantage estimates themselves?

6 comments

W1. Methodological Novelty With Theoretical Analysis

We thank the reviewer for this insightful comment and apologize for the misunderstanding caused by our insufficiently clear presentation. The key novelty of EvoAttack is not the individual operators, but a principled formulation of persistent memory attacks as test-time, gradient-free reinforcement learning framework against partially observable and nonstationary targets. Unlike conventional attacks that repeatedly optimize prompts against a fixed target, EvoAttack must adapt to target memories and defenses that continuously evolve through interaction. We address this by jointly modeling interaction-inferred target states, multi-horizon reward assignment, and state-conditioned policy improvement. This enables the attacker to learn from delayed outcomes and adapt its strategy without parameter updates.

To clarify the theoretical foundations, we have added a new Theoretical Analysis Section to the Appendix, establishing three results:

(1) Exact policy optimization. We prove that EvoAttack's logit update is the unique closed-form solution to a KL-regularized policy optimization problem: πr∗(s)∝π0,r(s)exp⁡(λA^r(s)).\pi_r^*(s)\propto\pi_{0,r}(s)\exp(\lambda\widehat A_r(s)). This yields exactly zr′=zr+λA^rz'_r=z_r+\lambda\widehat A_r, showing that strategy recalibration is a principled optimization step rather than heuristic logit biasing.

(2) Reliability of the nonparametric critic. We derive a finite-sample estimation bound that decomposes critic error into state mismatch, target drift, hidden-state ambiguity, and sampling uncertainty. This characterizes when the interaction-derived susceptibility proxy is informative and explains how state-conditioned retrieval can reuse historical experience under sparse feedback. We further show that ignoring target states introduces systematic estimation bias when strategy effectiveness differs across memory conditions.

(3) Policy tracking under nonstationarity. When the critic estimation error is bounded by εr\varepsilon_r, we prove:

\leq 2\lambda\varepsilon_r, \qquad F_r(\pi_r^{\mathrm{oracle}})-F_r(\widehat\pi_r) \leq 2\varepsilon_r.$$ Here, $\pi_r^{\mathrm{oracle}}$ is the KL-regularized optimal policy under the current target condition, and $F_r$ is its regularized objective. These bounds establish that accurate state-conditioned value estimation enables EvoAttack to track the evolving target-specific optimum without gradient updates. Furthermore, **the results under A-MemGuard in Section 4.5 empirically support this adaptive capability**, demonstrating that EvoAttack can effectively recalibrate its strategy preferences and maintain superior attack performance when the target's memory defense changes. Together, these results establish EvoAttack as more than a combination of existing components: it turns persistent memory attacks from repeated prompt optimization into state-conditioned, nonparametric policy improvement over an evolving adversarial environment, with a closed-form update and explicit tracking guarantees under target nonstationarity.

W2. Evaluation Reliability

We thank the reviewer and apologize for the insufficiently clear presentation of our evaluation protocol. Our evaluation follows a standardized protocol with identical attack budgets, benign interactions, and probe schedules across methods, ensuring fair, reproducible, and interpretable comparisons. To further clarify its reliability, we conduct a series of additional experiments in the multi attack setting, addressing each concern as follows.

Concern 1: Is P@k overly dependent on rigid interaction protocols?

Experiment 1: Persistence Protocol Sensitivity. We evaluate EvoAttack under extended persistence horizons and different benign interaction patterns, while keeping the attack budget unchanged. The results are shown below.

Benign InteractionHorizonEvoAttackBest Original BaselineImprovement
Same TaskP@564.8558.05+6.80
Same TaskP@1053.7545.50+8.25
Different TaskP@567.4059.95+7.45
Mixed TaskP@565.5557.85+7.70

Horizon Generalization: Although P@10 is not included in the training reward, EvoAttack still outperforms the strongest baseline by 8.25 percentage points, exceeding its 6.80 point advantage at P@5. This demonstrates that the learned strategies generalize beyond the evaluated reward horizons.

Interaction Robustness: EvoAttack consistently outperforms the strongest baseline across same task, different task, and mixed task interactions, with improvements ranging from 6.80 to 7.70 percentage points at P@5. These results indicate that its effectiveness does not depend on a specific benign interaction pattern.

Concern 2: Does probe timing affect the evaluation?

All probes are read only and do not modify the target's persistent memory. P@k is determined by the number of intervening benign memory updates, rather than the timing or placement of evaluation probes. Therefore, inserting evaluation probes does not alter memory persistence. Moreover, all methods follow the same probe schedule and receive identical feedback opportunities, ensuring fair and consistent comparisons.

Concern 3: Are the results sensitive to target memory retrieval settings?

Experiment 2: Memory Retrieval Sensitivity. We vary the target agent's memory retrieval budget across Top-5, Top-10, and Top-20 while keeping all other settings unchanged. The P@5 results are shown below.

Target RetrievalEvoAttack P@5Best Original Baseline P@5Improvement
Top-555.2049.15+6.05
Top-1064.8558.05+6.80
Top-2067.3556.20+11.15

EvoAttack consistently outperforms the strongest baseline across all retrieval budgets, with improvements ranging from 6.05 to 11.15 percentage points. While limited retrieval reduces attack effectiveness for both methods, expanding retrieval to Top-20 further benefits EvoAttack but slightly degrades the baseline. This suggests that EvoAttack better maintains attack effectiveness as more contextual memories become available, demonstrating that its advantage does not depend on a specific retrieval budget.

Concern 4: Are the reported improvements statistically reliable?

Experiment 3: Statistical Significance Analysis. We conduct three independent runs and report the mean and standard deviation of the main results in Tables 10、11.

SettingHorizonBest Original BaselineImprovementPaired 95% CIp-value
SingleP@0AutoDAN+3.10[2.67, 3.53]0.0010
SingleP@3ZombieAgents+3.68[1.74, 5.62]0.0147
SingleP@5ZombieAgents+3.20[1.77, 4.63]0.0107
MultiP@0AutoDAN+3.22[2.25, 4.18]0.0048
MultiP@3ZombieAgents+3.40[0.27, 6.53]0.0430
MultiP@5MINJA+5.88[3.70, 8.06]0.0073

All six comparisons achieve statistical significance (p < 0.05), with 95% confidence intervals consistently excluding zero. In particular, EvoAttack improves P@5 by 3.20 and 5.88 percentage points in the single and multi attack settings, respectively. These results provide statistical evidence that EvoAttack's improvements are consistent across repeated runs rather than attributable to random variation. Detailed results for all four domains are provided in the revised Appendix C.3.

Q1. Hyperparameter Sensitivity

We thank the reviewer for this suggestion. We evaluate the sensitivity of EvoAttack to four key hyperparameters: the target state update coefficient α\alpha, event magnitude cc, state retrieval tolerance ϵ\epsilon, and attack memory retrieval budget kk. We vary one parameter at a time while keeping the others fixed. Results are reported as P@5 (%) under the multi attack setting, weighted across the four domains according to their numbers of agents.

ParameterLower SettingDefaultHigher Setting
α\alpha0.2 (61.00)0.5 (64.85)0.8 (59.85)
cc2.5 (61.25)5 (64.85)10 (60.40)
ϵ\epsilon1 (59.45)2 (64.85)4 (61.45)
kk100 (64.30)200 (64.85)400 (65.80)

The results show that α\alpha, cc, and ϵ\epsilon have a greater impact on performance than kk. An inappropriate α\alpha can make state updates too slow or overly sensitive to recent feedback. Meanwhile, cc and ϵ\epsilon affect which historical experiences are considered relevant to the current target state. In contrast, performance remains stable when varying kk, suggesting that the quality of retrieved attack memory matters more than their quantity.

Importantly, cc and ϵ\epsilon are coupled rather than independent. The state retrieval condition depends on the normalized distance ϵ/c\epsilon / c, while the contribution of the state transition reward depends on βc\beta c. We therefore conduct an additional experiment that jointly adjusts these parameters while preserving both quantities.

ccϵ\epsilonβ\betaϵ/c\epsilon/cβc\beta cP@5P@5
2.510.20.40.564.60
520.10.40.564.85
1040.050.40.565.80

Despite substantial changes in the individual parameters, P@5 remains between 64.60% and 65.80%. This is consistent with the scale invariance of our state representation and retrieval mechanism. Together with the sensitivity results, it suggests that performance depends more on maintaining appropriate state similarity and reward scaling than on the absolute parameter values.

Regarding domain specific tuning, we use the same configuration across all four domains without individual optimization. Thus, our reported improvements do not depend on selecting different parameters for different domains. Although further tuning may improve performance, its effect on relative baseline performance has not been established.

Comments on the paper

2 comments
AuthorsOpenReview ↗

A Gentle Follow Up and Request for Feedback

We sincerely thank the reviewer again for your thoughtful comments and valuable suggestions. We have provided detailed responses to all your concerns, along with additional experiments and theoretical analyses.

As the discussion period is approaching its deadline, we would be very grateful if you could take another look at our responses when convenient. Any further feedback or suggestions would be greatly appreciated and would help us improve our work.

Thank you again for your time and consideration.

AuthorsOpenReview ↗

Summary of Our Rebuttal and Request for AC Consideration

We sincerely thank the Area Chair and reviewers for their time and constructive feedback. We have carefully addressed the concerns raised during the discussion through new theoretical analysis, additional experiments, and clarifications. We would like to highlight four key points.

  1. EvoAttack introduces a new gradient free reinforcement learning framework for persistent attacks against evolving memory agents, rather than simply combining existing attack techniques. It uses persistent attacker memory as a nonparametric critic to estimate strategy advantages and directly optimize the attack policy without updating model parameters. We further provide theoretical analysis of policy optimization, critic estimation error, and policy tracking under changing target conditions.

  2. We conducted additional controlled ablations that separate dynamic state conditioning from static task and tool matching. Removing both dynamic state components reduces P@5 by 4.90 percentage points, demonstrating their independent contribution to strategy learning.

  3. We extended the experiments to continual target memory, standard queries that update memory, longer persistence horizons, and different retrieval settings. We also provided confidence intervals and statistical significance tests. These results address concerns about evaluation realism, robustness, and reliability.

  4. We added AutoDAN-Turbo and Evo-Attacker as strong baselines and extended the evaluation to five attack rounds. EvoAttack continues to outperform these methods, particularly at longer persistence horizons, even when the baselines receive the same delayed feedback.

We believe these results address the reviewers' main concerns and strengthen the evidence for the novelty and soundness of EvoAttack. Its central contribution is a new gradient free reinforcement learning framework that uses a persistent nonparametric critic to learn and optimize attack strategies against evolving targets.

As the discussion period approaches its conclusion, we respectfully ask the Area Chair to carefully consider the methodological contributions and the additional evidence provided during the rebuttal when making the final assessment.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.