- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes RedOpus, an attacker agent that keeps notes as it goes: a short-term memory of what has happened in the current attempt and how the target tends to respond, and a long-term memory of what worked in earlier attempts. It also uses a planning step and a set of predefined attack "skills". The attacker cannot touch the environment directly. It can only change what the target agent sees, through user messages, retrieved data, or tool descriptions, and has to trick the agent into doing the harmful action itself. The paper frames this setting as a partially observable game and argue that an attacker who tracks the target's behavior can do no worse than one who uses a fixed strategy.
By extending RedOpus to RedOpusArena, a sandbox with 32 generated tasks across eight risk categories, where success is checked by rules on the final state rather than by an LLM judge. Using a small Qwen3-4B model as the attacker, and test five recent models. The main findings are that attacks through retrieved data work best, that models which refuse at first often give in later.
Critical strengths
- The sandbox verifies success through executed tool trajectories and changes to the environment state, rather than relying on an LLM judge to decide whether a response appears malicious. This gives a more reliable judge results.
- The problem that the paper tries to study is a timely problem under agent safety/security settings.
- The memory architecture separates meaningful sources of information, in which RedOpus distinguishes between episode memory, which summarizes task progress and obstacles; opponent memory, which tracks observed refusal patterns and tool-use preferences; and long-term memory, which stores selected transitions for reuse across episodes.
Critical weaknesses
- The major issue of this paper is there is not baseline comparison with static injection, a non-adaptive multi-turn retry, or existing memory-based attackers (AutoDAN-Turbo, RedAgent, X-Teaming, Crescendo/GOAT). The claim that static benchmarks “severely underestimate” vulnerability is unsupported.
- My main concern is that the paper claims to be a multi-turn attack, but it never clearly explains how the injection actually happens across turns. The formulation says the attacker adds a payload to the agent's context at every turn, but this is only described at an abstract level. For the message channel I can guess how it works, since the attacker acts as the user. For the data and tool channels it is not clear at all. Does the attacker rewrite the poisoned email or tool description after every step of the agent? When does the agent see the new version, and who is sending the user messages during this time? The paper also does not define what one "turn" is, and there is no algorithm, example trajectory, or example payload, even in the appendix. Without this, it is hard to judge whether the setting is realistic, because a real indirect injection usually gets one chance and no feedback. I would like the authors to show one full attack example for each channel and report results when the attacker is allowed only a single injection.
- Another concern is the missing benign task evaluation. For direct prompt injection this is fine, because the user is the attacker and there is no benign task to complete. But for indirect prompt injection and tool poisoning, the attacker is not the user. The user gives a normal task, and the harmful instruction is hidden in the data or in the tool description. In this setting, the benign task completion rate is an important metric. If the agent drops the user's task, the user will notice quickly, so the attack is not really stealthy, even though the paper describes these channels as stealthy. It is also unclear what a "refusal" means here, since an agent that ignores the injection and finishes the user's task is very different from an agent that refuses to do anything. The paper mentions this only as a limitation, but every task already has a benign instruction, so I think the completion rate with and without the attack should be reported for these two channels.
- RedOpusArena is listed as a main contribution, but Section 6 is only a few short paragraphs and most details are in the appendix. From Appendix A, the environment seems to be a set of mock Python data models and tool functions, not real applications or systems. The paper does not say how many environments and tools there are, how the orchestrator runs the two agents, or what makes the sandbox “high-fidelity”. There is also no comparison with existing sandboxes. For example, RedTeamCUA uses a real OS and real web platforms, which seems more realistic than a mocked environment. For tool-calling agents, AgentDojo, Agent Security Bench, and SHADE-Arena already exist, and the code examples in Appendix A.2 look very similar to AgentDojo’s workspace environment. The paper should clarify clearly whether RedOpusArena is built on an existing sandbox, and add a table comparing it with prior ones so that the new part is clear.
- The effective refusal rates in Figure 5 do not seem to match the definition in Section 7.1. ERR is defined as the fraction of initially refusing episodes that stay uncompromised until the end, so its denominator is the number of initial refusals. But several cells in Figure 5 show a refusal rate of 0% together with a positive ERR. For example, Gemini on the message channel for property damage has RR 0% and ERR 100%, DeepSeek on the message channel for data loss has RR 0% and ERR 25%, and GPT-5.4 on the data channel for ethics/morality has RR 0% and ERR 25%. If no episode starts with a refusal, ERR should be undefined, not a positive number.
- All quantitative results are in the appendix, with no numbers or confidence intervals in the main text. Section 7.1 says eight risk categories, but Appendix A.3 lists nine (Computer Security has no results). This make the paper really hard to follow.
- The ablation study is not enough to show that each component of RedOpus is needed. Appendix D only removes two parts, the long-term memory and the skill injection. The opponent memory, the episode memory, the reflection stage, and the planning module are never removed, even though the opponent memory is the part that connects to the theory in Section 4. The two ablations that are reported also do not give clear evidence. Removing skills changes the success rate by less than 1% overall, and removing long-term memory lowers it by only about 6%, with the data channel going in the opposite direction. In addition, these ablations use qwen3-max as the target instead of the five models in the main experiments, and they cover only three tasks with wide confidence intervals. I suggest the paper ablate every component on the same targets and tasks as the main experiments, and move the key numbers into the main text.
- The paper is hard to follow, and after reading the method section I am still not sure how RedOpus actually works. The method is described only at a high level. There is no algorithm or pseudocode, no prompts, and no example of an attack in the paper. Important details are missing, such as how the memory summaries are produced, how retrieval works, whether the memory is shared across tasks and target models, and how many skills there are and what they contain. The organization also makes it harder. The theory takes about two pages, while the sandbox is described in a few short paragraphs and all the results are in the appendix, so the main text has no numbers at all.
Questions for authors
- Can you add some baselines to compare RedOpus with some static baselines? (AutoDAN-Turbo, X-Teaming or MUZZLE [1])
- For the data and tool channels, what exactly does the attacker change at each turn, and when does the target agent see the new version?
- What is the benign task completion rate with and without the attack for the data and tool channels?
- Is RedOpusArena built on top of AgentDojo or another existing sandbox, and if so, which parts are new?
- How is ERR computed for the cells in Figure 5 where RR is 0%, such as Gemini on the message channel for property damage?
- How does the success rate change when the opponent memory is removed, using the same target models and tasks as the main experiments?
- Can you provide an algorithm or one full example episode that shows how planning, memory retrieval, and skills work together step by step?
[1] MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attacks