acceptodds
Withdrawn from ICLR 2027

RedOpus: Multi-turn Red-teaming LLM Agents with Reflective Memories

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The development of Large Language Model (LLM) Agents that can invoke tools and act upon external data and entities is built upon LLMs' instruction-following and reasoning capabilities. The insufficient robustness in these capabilities, however, raises security risks and facilitates red teaming research aiming to uncover agent vulnerabilities under complex, noisy and sometimes adversarial conditions. While existing works focus on one-shot attacks and defenses, real application scenarios usually involve multi-round interaction where LLM agents face continuous, strategic threats. To bridge this gap, we propose RedOpus, a multi-turn red-teaming framework that leverages reflective memory to guide iterative attack refinement. We also propose RedOpusArena, a multi-turn interactive sandbox featuring diverse simulated scenarios and automatic generation of massive tasks. We test our red-teaming framework on agents with multiple LLM backbones and reveal a general vulnerability of existing LLMs to automatic iterative attacks.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agent, Agent Security, Red-teaming, Multi-agent game

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_uJHCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes RedOpus, an attacker agent that keeps notes as it goes: a short-term memory of what has happened in the current attempt and how the target tends to respond, and a long-term memory of what worked in earlier attempts. It also uses a planning step and a set of predefined attack "skills". The attacker cannot touch the environment directly. It can only change what the target agent sees, through user messages, retrieved data, or tool descriptions, and has to trick the agent into doing the harmful action itself. The paper frames this setting as a partially observable game and argue that an attacker who tracks the target's behavior can do no worse than one who uses a fixed strategy.

By extending RedOpus to RedOpusArena, a sandbox with 32 generated tasks across eight risk categories, where success is checked by rules on the final state rather than by an LLM judge. Using a small Qwen3-4B model as the attacker, and test five recent models. The main findings are that attacks through retrieved data work best, that models which refuse at first often give in later.

Critical strengths
  • The sandbox verifies success through executed tool trajectories and changes to the environment state, rather than relying on an LLM judge to decide whether a response appears malicious. This gives a more reliable judge results.
  • The problem that the paper tries to study is a timely problem under agent safety/security settings.
  • The memory architecture separates meaningful sources of information, in which RedOpus distinguishes between episode memory, which summarizes task progress and obstacles; opponent memory, which tracks observed refusal patterns and tool-use preferences; and long-term memory, which stores selected transitions for reuse across episodes.
Critical weaknesses
  • The major issue of this paper is there is not baseline comparison with static injection, a non-adaptive multi-turn retry, or existing memory-based attackers (AutoDAN-Turbo, RedAgent, X-Teaming, Crescendo/GOAT). The claim that static benchmarks “severely underestimate” vulnerability is unsupported.
  • My main concern is that the paper claims to be a multi-turn attack, but it never clearly explains how the injection actually happens across turns. The formulation says the attacker adds a payload to the agent's context at every turn, but this is only described at an abstract level. For the message channel I can guess how it works, since the attacker acts as the user. For the data and tool channels it is not clear at all. Does the attacker rewrite the poisoned email or tool description after every step of the agent? When does the agent see the new version, and who is sending the user messages during this time? The paper also does not define what one "turn" is, and there is no algorithm, example trajectory, or example payload, even in the appendix. Without this, it is hard to judge whether the setting is realistic, because a real indirect injection usually gets one chance and no feedback. I would like the authors to show one full attack example for each channel and report results when the attacker is allowed only a single injection.
  • Another concern is the missing benign task evaluation. For direct prompt injection this is fine, because the user is the attacker and there is no benign task to complete. But for indirect prompt injection and tool poisoning, the attacker is not the user. The user gives a normal task, and the harmful instruction is hidden in the data or in the tool description. In this setting, the benign task completion rate is an important metric. If the agent drops the user's task, the user will notice quickly, so the attack is not really stealthy, even though the paper describes these channels as stealthy. It is also unclear what a "refusal" means here, since an agent that ignores the injection and finishes the user's task is very different from an agent that refuses to do anything. The paper mentions this only as a limitation, but every task already has a benign instruction, so I think the completion rate with and without the attack should be reported for these two channels.
  • RedOpusArena is listed as a main contribution, but Section 6 is only a few short paragraphs and most details are in the appendix. From Appendix A, the environment seems to be a set of mock Python data models and tool functions, not real applications or systems. The paper does not say how many environments and tools there are, how the orchestrator runs the two agents, or what makes the sandbox “high-fidelity”. There is also no comparison with existing sandboxes. For example, RedTeamCUA uses a real OS and real web platforms, which seems more realistic than a mocked environment. For tool-calling agents, AgentDojo, Agent Security Bench, and SHADE-Arena already exist, and the code examples in Appendix A.2 look very similar to AgentDojo’s workspace environment. The paper should clarify clearly whether RedOpusArena is built on an existing sandbox, and add a table comparing it with prior ones so that the new part is clear.
  • The effective refusal rates in Figure 5 do not seem to match the definition in Section 7.1. ERR is defined as the fraction of initially refusing episodes that stay uncompromised until the end, so its denominator is the number of initial refusals. But several cells in Figure 5 show a refusal rate of 0% together with a positive ERR. For example, Gemini on the message channel for property damage has RR 0% and ERR 100%, DeepSeek on the message channel for data loss has RR 0% and ERR 25%, and GPT-5.4 on the data channel for ethics/morality has RR 0% and ERR 25%. If no episode starts with a refusal, ERR should be undefined, not a positive number.
  • All quantitative results are in the appendix, with no numbers or confidence intervals in the main text. Section 7.1 says eight risk categories, but Appendix A.3 lists nine (Computer Security has no results). This make the paper really hard to follow.
  • The ablation study is not enough to show that each component of RedOpus is needed. Appendix D only removes two parts, the long-term memory and the skill injection. The opponent memory, the episode memory, the reflection stage, and the planning module are never removed, even though the opponent memory is the part that connects to the theory in Section 4. The two ablations that are reported also do not give clear evidence. Removing skills changes the success rate by less than 1% overall, and removing long-term memory lowers it by only about 6%, with the data channel going in the opposite direction. In addition, these ablations use qwen3-max as the target instead of the five models in the main experiments, and they cover only three tasks with wide confidence intervals. I suggest the paper ablate every component on the same targets and tasks as the main experiments, and move the key numbers into the main text.
  • The paper is hard to follow, and after reading the method section I am still not sure how RedOpus actually works. The method is described only at a high level. There is no algorithm or pseudocode, no prompts, and no example of an attack in the paper. Important details are missing, such as how the memory summaries are produced, how retrieval works, whether the memory is shared across tasks and target models, and how many skills there are and what they contain. The organization also makes it harder. The theory takes about two pages, while the sandbox is described in a few short paragraphs and all the results are in the appendix, so the main text has no numbers at all.
Questions for authors
  1. Can you add some baselines to compare RedOpus with some static baselines? (AutoDAN-Turbo, X-Teaming or MUZZLE [1])
  2. For the data and tool channels, what exactly does the attacker change at each turn, and when does the target agent see the new version?
  3. What is the benign task completion rate with and without the attack for the data and tool channels?
  4. Is RedOpusArena built on top of AgentDojo or another existing sandbox, and if so, which parts are new?
  5. How is ERR computed for the cells in Figure 5 where RR is 0%, such as Gemini on the message channel for property damage?
  6. How does the success rate change when the opponent memory is removed, using the same target models and tasks as the main experiments?
  7. Can you provide an algorithm or one full example episode that shows how planning, memory retrieval, and skills work together step by step?

[1] MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attacks

Reviewer_VDSHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

RedOpus studies multi-turn attacks against LLM agents with tool-use capabilities. The attacker adapts its strategy based on the target’s responses, using within-episode memory, opponent profiling, and cross-task reflective memory. The paper also introduces RedOpusArena, a simulated enterprise environment that determines attack success through actual tool execution and environment state changes.

Critical strengths
  1. The paper examines agent security under sustained interaction, beyond single-turn harmful-response evaluation, showing great practicality.
  2. Experiments cover message, external-data, and tool-schema injection, supporting controlled security evaluations.
Critical weaknesses
  1. The comparison section missed strong attack baselines. The main experiments compare target models without establishing whether RedOpus outperforms ordinary multi-turn attackers or existing adaptive attacks under matched budgets.
  2. The core contributions are insufficiently validated. Opponent memory is not directly ablated, while the memory and skill ablations are too limited to support strong mechanistic conclusions.
  3. The assumptions of Lemma 1 do not establish strict informational superiority. The remaining non-strict advantage does not guarantee improvements for the implemented system.
  4. Differences from existing adaptive attacks and strategy-library methods are insufficiently established, which corrupts the novelty. Some claims about security policies and comprehensive threat coverage exceed the conclusion.
Questions for authors

The main text reports eight risk categories, while the appendix lists nine. why?

Reviewer_HT4uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RedOpus attacks tool-using LLM agents over multiple turns. It keeps summaries of the conversation and the target’s behavior, retrieves experience from earlier episodes, and uses planning and attack skills to choose subsequent prompts. RedOpusArena tests these attacks through dialogue, external data, and tool schemas, checking whether the target performs the attacker’s intended actions. The paper formulates the interaction as a partially observable stochastic game and evaluates five target models.

Critical strengths
  • Attack success is checked against executed actions and changes to the simulated environment. An unsafe-sounding response alone is therefore insufficient to count as a successful attack.
  • The experiments cover dialogue, data, and tool-schema attacks against five target models. Using the same environment allows the authors to compare these attack channels without changing the underlying tasks.
  • The ablations report cases where the added components do not help. Skills do not consistently improve ASR, and removing long-term memory improves the data-channel result. The paper acknowledges these mixed results rather than claiming that every component always helps.
Critical weaknesses
  • The paper does not show whether opponent memory improves attacks. Section 7 compares target models but includes no competing attack method. The LTM ablation reports a 5.1-percentage-point reduction in overall ASR when retrieval is removed, but does not separately test the opponent profile (Figure 7). This distinction matters because reflection and reusable experience are already central to Reflexion and AutoDAN-Turbo [1, 2], while Crescendo and AgentVigil already adapt attacks using target feedback [3, 4]. A comparison using the same attacker model, conversation history, and query budget is needed to show what RedOpus adds.
  • Figures 5 and 6 report different refusal rates for the same result. For GPT-5.4 on message-channel illegal activities, both figures show ASR = 35%, but Figure 5 reports RR = 45% and Figure 6 places the point at approximately 80% RR. Figure 5 also reports RR = 0.0% and ERR = 100.0% for Gemini on message-channel property damage. Section 7.1 defines ERR using initially refusing episodes as its denominator. At the stated sample size of 200 episodes per cell, RR = 0.0% means that denominator is zero. The authors need to explain these calculations before the figures can support claims about refusal persistence.
  • Lemma 1’s stated condition does not ensure strict Blackwell dominance. It requires two opponent policies to differ at some state. Suppose they behave identically in every context the attacker can reach and differ only in an unreachable context. The condition holds, but the observed histories contain no information that distinguishes the two types. Full and garbled histories are then equally informative. The proof also needs to establish that the particular mapping gg loses information about the opponent; deleting feedback records alone does not establish this. These assumptions should be stated explicitly. The weaker inequalities in Equations 8–9 are unaffected.
  • The results do not distinguish learning a repeated task from handling a new task. The main evaluation repeats each of 32 generated tasks, while memories can carry information from earlier episodes. The paper does not clearly specify which memories are reset between tasks, channels, and target models. It also omits legitimate-task completion, so low ASR could partly reflect an agent’s inability to execute the required tools. A small test on unseen tasks from a public benchmark such as AgentDojo [5], using a newer model available to the authors, would help establish whether the findings carry over. An open-weight model would be sufficient.

[1] Noah Shinn et al. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. NeurIPS 2023.

[2] Xiaogeng Liu et al. 2025. AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs. ICLR 2025.

[3] Mark Russinovich, Ahmed Salem, and Ronen Eldan. 2025. Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack. USENIX Security 2025.

[4] Zhun Wang et al. 2025. AGENTVIGIL: Automatic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents. Findings of EMNLP 2025.

[5] Edoardo Debenedetti et al. 2024. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. NeurIPS 2024, Datasets and Benchmarks Track.

Questions for authors
  1. Which refusal calculation was used in each figure? Please provide the numerators, denominators, aggregation rules, and treatment of zero initial refusals. Explain both the GPT-5.4 discrepancy between Figures 5 and 6 and the Gemini RR = 0%, ERR = 100% result.
  2. Does opponent memory help when conversation history and attack budget are held fixed? Please compare against an attacker that keeps the full history without an explicit opponent profile, and one relevant publicly available attack method.
  3. What assumptions make Lemma 1 strict? Please address policy differences at unreachable contexts and mappings gg that preserve all information about the opponent. Is information loss assumed or derived?
  4. Does memory learned from earlier tasks help on unseen tasks? Please specify what is retained or reset across episodes, tasks, channels, and target models. If feasible, test a small public benchmark subset with a newer accessible model and report both attack success and legitimate-task completion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.