acceptodds
Under review as a conference paper at ICLR 2027

Autonomous Red Teaming for Utility-Preserving Indirect Prompt Injection via Fine-Grained Diagnosis

Abstract

Indirect prompt injection (IPI) embeds malicious instructions in external content to redirect tool-using language agents toward unauthorized actions. Effective red teaming must therefore identify payloads that induce the target behavior without disrupting the legitimate user task. However, existing adaptive attacks typically rely on scalar end-to-end outcomes or coarse holistic feedback, offering limited insight into why an attempt fails or how it should be improved. We introduce EPOA, an adaptive black-box framework that provides fine-grained, evidence-grounded guidance for iterative IPI optimization. At its core, the EPOA Judge decomposes observable attack progress into four ordered diagnostic blocks: Exposure, Provenance Promotion, Objective Adoption, and Action Binding. For each evaluated payload, it determines which blocks are supported by observable execution evidence; an optimizer then uses this structured diagnosis, together with the accumulated trajectory history, to generate the next candidate. This diagnose-and-refine process explores multiple carrier trajectories within a fixed query budget. We evaluate EPOA on AgentDojo across four victim models and three defense settings. EPOA achieves competitive utility-preserving attack performance across all 12 model-defense combinations, obtaining an average utility-preserving attack success rate of 33.5%, compared with 31.6% for IterInject and 28.9% for PAIR. It also achieves a 55.1% attack success rate and 74.3% task utility. These results support the effectiveness of fine-grained, evidence-grounded diagnosis for adaptive IPI optimization across the evaluated models and defenses.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Indirect prompt injection, Tool-using language agents, agent security, red teaming, adversarial attacks, black-box optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mRG5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EPOA, an adaptive black-box method for indirect prompt injection on tool-using agents. An LLM judge labels each attack attempt on four stages—Exposure, Provenance Promotion, Objective Adoption, Action Binding—as Positive, Negative, or Unknown, citing execution evidence; an LLM optimizer uses these labels, the evidence, and the trajectory history to write the next payload. K trajectories of T candidates each are run under a fixed query budget, and the best evaluated payload is reported, prioritizing joint attack-and-utility success. On AgentDojo (449 cases) with four victims and three defense settings, EPOA averages 33.5% utility-preserving attack success (UPA) against 31.6% for IterInject and 28.9% for an adapted PAIR. Ablations on one victim without defenses mask subsets of the four stages, compare against no feedback and coarse feedback, remove the evidence text from the judge output, and vary the budget up to 12 queries.

Critical strengths
  1. The four-stage decomposition is a sensible way to describe where an injection fails, and the paper tests it rather than just proposing it. The evidence ablation (Figure 3) is the one result with a clear effect: removing the supporting evidence from the judge output drops UPA from 43.0% to 35.9% and ASR from 71.9% to 56.8% on Qwen3.5, which suggests that what helps the optimizer is the quoted trace, not the stage label.

  2. The evaluation reports attack success and task utility separately and in combination, under three defenses and four victims, with all methods sharing initialization and query budget. The appendix case studies show full payloads, tool calls, and observations, which makes it possible to see what the optimized attacks actually look like.

Critical weaknesses
  1. The methodological advance over IterInject appears concentrated in the diagnostic representation: four stage-specific labels with supporting execution evidence. Table 3 provides evidence that the complete diagnosis helps in the evaluated setting, with single-stage removals reducing overall UPA by 2.7–6.9 pp. However, these ablations do not isolate the benefit of the stage structure from the information supplied in the feedback. Likewise, Figure 3 establishes the value of adding evidence to stage labels, but does not establish whether stage labels add value beyond evidence alone. A matched comparison using the same execution excerpts, optimizer, history, initialization, and query budget, but without stage labels, would clarify the distinctive contribution. In other words, what separates EPOA from IterInject is the format of the judge’s output, not the search procedure.

  2. The gains are too small to separate from noise. Averaged over 12 settings, EPOA leads IterInject by 1.9 pp UPA. Per setting it is behind IterInject on Gemini/Instructional Prevention (50.6 vs 50.8), behind PAIR on DeepSeek/Instructional Prevention (5.8 vs 6.5), and tied on Gemini/Repeat User Prompt (49.4), so it is best in 9 of 12. With 449 cases and one run, a 2 pp gap is within normal variation, and the paper gives no intervals, no repeats, and no victim temperature. Some cells are identical across methods to the decimal (Gemini/No Defense Banking for IterInject and EPOA; Gemini/No Defense Slack for all three), which suggests the shared starting templates decide those suites and the optimizer changes nothing.

  3. The judge is never checked, and the obvious baseline is missing. There is no human agreement score, no label accuracy, and no breakdown of which stage fails first, so we cannot tell whether the labels are right or whether the rewrites target the diagnosed stage. The case-study payloads are rewrites of AgentDojo’s “Important Instructions” template (“first complete the user’s original request... then also do this follow-up action... do not mention this note”), but that template is not run as a baseline, so we cannot tell how much of the 55.1% ASR comes from the optimizer and how much from the template.

  4. “Utility-preserving” is a metric, not something the method does, and parts of the protocol are not disclosed. None of the four stages or the optimizer prompt mentions the user task; utility only enters at final selection, and EPOA has lower utility than both baselines on several suites (Travel on Qwen3.5: 45.0 vs 52.0 and 64.0). The reproducibility statement mentions “reused cases,” “exclusions,” and “Final-validation executions” that are not in the reported numbers; none of this appears in the paper, and neither does how 100 of 140 Travel and 100 of 240 Workspace cases were chosen.

Questions for authors
  1. What are K and T, and what temperature do the victims use? Can you add paired bootstrap intervals for EPOA vs IterInject in each of the 12 settings, and at least one repeated run?

  2. Which cases were excluded or reused, what were the “Final-validation executions,” and why are the reported metrics based on search records instead of those runs? How were the Travel and Workspace subsets chosen?

  3. Can you add the static “Important Instructions” template as a baseline at the same injection point, and run IterInject with the same quoted trace excerpts but no stage labels? Together these would show what the four-stage structure adds.

Reviewer_FJFtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This auto red-teaming paper uses previous attacks to provide fine-grained feedback by decomposing observable attack progress into four ordered blocks: Exposure, Provenance Promotion, Objective Adoption, and Action Binding, and this feedback will be used to guide the attacker LLM to search for better strategies. Instead of binary feedback, these fine-grained feedbacks enable the attacker model to propose stronger attacking strategies. On AgentDojo, this method demonstrates improved indirect prompt injection ASR and also maintains the utility

Critical strengths

the ideas of fine-grained Diagnosis is simple and useful

the paper's is clear and easy to follow

Critical weaknesses

this paper is very similar to IterInject's idea, but this paper's utility-preserving attack success rate is 33.5%, and this is a very small difference compared with 31.6% for IterInject.

the 4 diagnostic blocks are a bit artificial and less scalable.

Questions for authors

"he EPOA Judge decomposes observable attack progress into four ordered diagnostic blocks: Exposure, Provenance Promotion, Objective Adoption, and Action Binding.": I am curious why only decomposing them into those 4 categories? are there auto process to incldue more categories?

Reviewer_mDYpOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EPOA, a black-box framework for iterative indirect prompt injection against tool-using language agents. It then uses an LLM judge to diagnose execution traces along four dimensions: Exposure, Provenance Promotion, Objective Adoption, and Action Binding. An LLM optimizer then uses this feedback and the search history to refine payloads. On AgentDojo, the reported average utility-preserving attack success rate (UPA) is 33.5%, compared with 31.6% for IterInject and 28.9% for PAIR.

Critical strengths
  1. The problem is relevant. Jointly evaluating attack success and legitimate-task completion provides useful information beyond attack success alone.
  2. The diagnostic process is interpretable, and separating it into distinct stages may help clarify how an attack progresses and where it fails.
  3. The evaluation covers four victim models and three defense settings, with ablations of diagnostic dimensions, feedback content, and query budget.
Critical weaknesses
  1. Limited novelty and insufficient isolation of the contribution.

The rationale for decomposing the attack process into these particular four stages is unclear. While the decomposition is intuitive, the paper does not articulate a general design principle that motivates these stages or explains why they are preferable to alternative formulations. As presented, the framework appears to be an engineering heuristic whose broader methodological significance remains insufficiently established.

The paper identifies IterInject as the closest related work and briefly contrasts its outcome-level diagnosis with EPOA’s stage-level, evidence-grounded feedback (Section 2). However, this comparison remains high-level. The generate–execute–diagnose–refine pipeline is already established, and Section 4.3 passes the diagnosis to a general-purpose LLM optimizer without introducing a stage-specific update mechanism or a new search algorithm.

The main addition therefore appears to be a manually specified feedback structure. The current ablations do not convincingly isolate its distinctive value, since removing diagnostic dimensions or supporting evidence also changes the information available to the optimizer. Although the coarse-feedback comparison is relevant, its relationship to IterInject’s original feedback remains unclear. Replacing only the diagnostic module while holding the remaining components fixed, and comparing against free-form diagnosis based on the same execution information, would better establish the contribution of the proposed decomposition.

  1. The diagnostic judge is not independently validated.

The judge is central to the method, but its diagnostic reliability is not evaluated. The paper does not report whether its labels agree with human annotations or how often it outputs Unknown. It is also unclear whether observable traces provide enough evidence to distinguish Provenance Promotion (treating injected content as an instruction) from Objective Adoption (pursuing the injected goal). More fundamentally, the first stage without a Positive label does not necessarily identify where the attack failed: the relevant behavior may have occurred without being visible in the trace. The paper should explain how the optimizer distinguishes an actual attack failure from insufficient evidence, since these situations may require different revisions.

  1. The empirical advantage is modest and its stability is unclear.

The average UPA improvement over IterInject is approximately 1.9 percentage points, with several small gains and some tied or worse results in Table 2. Since the optimizer is stochastic, repeated independent searches and uncertainty estimates are needed to assess the stability of these gains. Moreover, most diagnostic ablations are restricted to Qwen3.5-35B-A3B without defenses, leaving it unclear whether the proposed diagnosis consistently explains the improvements across settings. Extending these ablations to models of different sizes and families, as well as defended settings, would strengthen the evidence for the method’s generality.

  1. Important protocol details remain unclear.

The main experiments do not clearly specify K and T, budget allocation, complete prompts, or feedback construction. Equal victim-query counts also need not imply equal total cost. In addition, Section 4.1 describes a hidden user task, while several appendix payloads reproduce the user request. The authors should explain when and how this information becomes available and whether all baselines receive equivalent information.

Questions for authors
  1. What are the component-level differences from the evaluated IterInject implementation? Does the advantage persist when only the diagnostic module is replaced, with the remaining components and information budget fixed?
  2. How reliable are the diagnostic labels, and how are Unknown labels and answer-only objectives handled?
  3. Are the gains stable across repeated searches and fresh executions of selected payloads? Please also clarify the K/T configuration and attacker-visible information.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.