- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
This paper studies attacks on memory-based agents with evolving memory. It proposes EvoAttack, which uses interaction feedback and attacker memory to learn state-conditioned strategy utilities and adjust the attack policy. Experiments show improved attack effectiveness across target states and persistence horizons.
Critical strengths
-
The experiments are comprehensive. The paper evaluates 10 agents across four domains, different model backbones, initial memory settings, and defense mechanisms. It also includes ablation studies and evaluates attack persistence over multiple horizons.
-
The paper is well organized and easy to follow. The motivation and problem formulation are clearly presented. The methodology is also explained step by step.
Critical weaknesses
- The novelty seems limited. The target state is constructed from simple feedback statistics and attack history. The method then retrieves historical rewards to adjust strategy preferences. These components largely build on existing memory retrieval and reward-based strategy selection techniques. The methodological contribution is therefore not sufficiently clear.
- The paper lacks comparisons with closely related methods. AutoDAN-Turbo (ICLR 2025) and AutoRedTeamer (NeurIPS 2025) also use historical attack experience to guide subsequent attacks. Without these comparisons, it is difficult to assess the actual benefit of the proposed attacker memory over existing methods.
Questions for authors
- Why is AutoDAN-Turbo (ICLR 2025) not included in the comparison? It is a more recent SOTA method and is more closely related to EvoAttack.
- The improvement from single to multi-attack is smaller for EvoAttack than for PAIR and AutoDAN. Why do the authors choose only three attack rounds? Would its advantage persist with more attack rounds?
- AutoDAN-Turbo (ICLR 2025) and AutoRedTeamer (NeurIPS 2025) also use historical attack experience to guide strategy selection. What is fundamentally new in EvoAttack beyond state-conditioned retrieval and multi-horizon rewards?
6 comments
W2 & Q1. Comparisons with Lifelong Adaptive Attack Methods
We thank the reviewer for this valuable suggestion. Our original baseline selection focused on specialized memory attacks and representative adaptive attack optimization methods. AutoDAN-Turbo primarily addresses lifelong jailbreak strategy discovery, while AutoRedTeamer focuses on autonomous red teaming through continuous discovery and integration of attack methods. Their original objectives and evaluation settings differ from our memory persistence benchmark, which motivated our initial baseline selection.
Nevertheless, we agree that comparisons with more recent lifelong adaptive methods are important. We have therefore extended the main performance comparison in Section 4.2 (Tables 1 and 2) to include AutoDAN-Turbo and Evo-Attacker, another recent memory-augmented adaptive attack method. We evaluate them using the same target agents, attack budgets, and persistence evaluation protocol.
The results averaged across four domains are summarized below.
| Method | Single P@0 | Single P@3 | Single P@5 | Multi P@0 | Multi P@3 | Multi P@5 |
|---|---|---|---|---|---|---|
| AutoDAN-Turbo | 78.04 | 54.71 | 39.59 | 86.38 | 71.58 | 61.65 |
| Evo-Attacker | 75.54 | 51.40 | 35.48 | 84.38 | 68.56 | 58.04 |
| EvoAttack | 78.46 | 56.17 | 44.58 | 88.96 | 74.88 | 65.75 |
EvoAttack outperforms both baselines across all persistence horizons. Compared with AutoDAN-Turbo, it improves P@5 by 4.99 percentage points in the single attack setting and 4.10 points in the multi attack setting. Notably, the single attack advantage increases from 0.42 points at P@0 to 4.99 points at P@5.
These results suggest that EvoAttack's advantage lies particularly in maintaining attack effectiveness after subsequent benign interactions. While lifelong strategy exploration improves attack success, EvoAttack additionally learns the state-dependent long-term utility of strategies, enabling it to favor attacks with stronger persistence rather than optimizing primarily for immediate success.
Q2. Effectiveness Under More Attack Rounds
We thank the reviewer for this important question. We originally evaluate one and three attack rounds to assess attack effectiveness under limited interaction budgets, where efficient strategy selection is particularly important. The 3 round setting allows feedback-driven adaptation while avoiding excessive repeated attempts that may obscure differences in strategy learning efficiency. To examine whether our findings depend on this choice, we further extend the evaluation to five attack rounds while keeping the target agents and evaluation protocol consistent.
The P@5 results, weighted across the four domains according to their numbers of agents, are summarized below.
| Method | 1 Attack | 3 Attacks | 5 Attacks | Δ 1→3 | Δ 3→5 |
|---|---|---|---|---|---|
| PAIR | 29.55 | 54.10 | 64.35 | +24.55 | +10.25 |
| LLM-Fuzzer | 23.55 | 45.50 | 54.65 | +21.95 | +9.15 |
| AutoDAN | 32.95 | 57.90 | 68.30 | +24.95 | +10.40 |
| AutoDAN-Turbo | 39.20 | 60.70 | 70.15 | +21.50 | +9.45 |
| Evo-Attacker | 35.05 | 57.00 | 65.85 | +21.95 | +8.85 |
| EvoAttack | 44.25 | 64.85 | 75.05 | +20.60 | +10.20 |
First, the smaller absolute improvement from one to three rounds does not indicate weaker adaptation. EvoAttack already achieves substantially higher persistence with a single attack, reaching 44.25% compared with 29.55% for PAIR and 32.95% for AutoDAN, leaving less room for further improvement. More importantly, when extending from three to five rounds, EvoAttack gains another 10.20 percentage points, comparable to PAIR at 10.25 points and AutoDAN at 10.40 points. This demonstrates that EvoAttack continues to benefit from additional interactions rather than exhibiting weaker adaptation as the attack budget increases.
Second, EvoAttack consistently maintains its advantage across different attack budgets. At 5 rounds, EvoAttack achieves 75.05% P@5, outperforming the strongest baseline, AutoDAN-Turbo, by 4.90 percentage points. This margin is slightly larger than its 4.15 point advantage at three rounds. These results demonstrate that EvoAttack continues to improve with additional attack opportunities and that its superiority does not depend on the specific choice of three attack rounds.
Finally, stronger performance with fewer attack rounds is itself an important advantage. As the attack budget increases, repeated attempts provide all methods with additional opportunities to improve, potentially narrowing performance differences as success rates approach higher levels. We therefore emphasize one and three rounds in the main evaluation to better distinguish the effectiveness of strategy learning under limited interaction budgets. The additional 5 round results further confirm that EvoAttack achieves stronger persistence with fewer attempts while maintaining its leading performance as more attack rounds become available. Together, these findings demonstrate both the interaction efficiency and robustness of EvoAttack across different attack budgets.
W1. Methodological Novelty and Theoretical Analysis
We thank the reviewer for this insightful comment and apologize for the insufficiently clear presentation of our methodological contributions.
The key novelty of EvoAttack is not the individual operators, but a principled formulation of persistent memory attacks as test-time, gradient-free reinforcement learning against partially observable and nonstationary targets. Unlike conventional adaptive attacks that optimize prompts based on immediate feedback, EvoAttack addresses a dynamic learning problem in which strategy effectiveness depends on an evolving and unobservable target memory, and successful attacks may lose their effectiveness through subsequent interactions.
To address this challenge, EvoAttack establishes a reinforcement learning framework without gradient updates or parameter training. Specifically, the interaction-inferred target state provides an observable representation of the partially observable environment, the selected attack strategy serves as the action, and multi-horizon persistence outcomes provide delayed rewards. Historical attack experiences are maintained as a nonparametric critic that estimates state-conditioned strategy values and advantages. Rather than merely retrieving successful attacks, EvoAttack uses these advantage estimates to directly improve the LLM strategy policy through logit recalibration. This forms a closed reinforcement learning loop in which attack experience continuously updates the policy while the target memory evolves.
Importantly, this formulation is supported by theoretical analysis rather than relying solely on a heuristic combination of existing components. We have added a new Theoretical Analysis section in Appendix A, establishing three results:
(1) Exact policy optimization. We prove that EvoAttack's logit update is the unique closed-form solution to a KL-regularized policy optimization problem: πr∗(s)∝π0,r(s)exp(λAr(s)). This yields exactly zr′=zr+λAr, demonstrating that our gradient-free policy recalibration implements an explicit reinforcement learning optimization objective rather than an arbitrary adjustment of strategy preferences.
(2) Reliability of the nonparametric critic. We derive a finite-sample estimation bound that decomposes critic error into state mismatch, target drift, hidden-state ambiguity, and sampling uncertainty. This characterizes when the lightweight interaction-derived state representation supports reliable strategy evaluation and explains how state-conditioned experience retrieval can reduce estimation bias under partial observability and nonstationarity.
(3) Policy tracking under nonstationarity. When the critic estimation error is bounded by εr, we prove:
\leq 2\lambda\varepsilon_r, \qquad F_r(\pi_r^{\mathrm{oracle}})-F_r(\widehat\pi_r) \leq 2\varepsilon_r.$$ These bounds establish that accurate state-conditioned value estimation enables the gradient-free policy to remain close to the current target-specific optimum, even as the target condition changes. The experiments under A-MemGuard in Section 4.5 further support the practical effectiveness of this adaptive mechanism under memory-based defenses. Together, these results establish EvoAttack as more than historical reward retrieval and strategy preference adjustment. Its methodological contribution lies in realizing a theoretically grounded, gradient-free reinforcement learning framework that connects partial state inference, delayed reward assignment, nonparametric value estimation, and direct policy improvement. This enables persistent strategy learning against evolving memory-based agents without updating model parameters.