acceptodds
Withdrawn from ICLR 2027

When Conversation Turns Hostile: Conversation-Observing Indirect Prompt Injection for LLM Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

As LLMs become more capable, agents can perform more complex tasks with tools and maintain longer interactions with users. These benign conversations contain evidence on task results, user concerns, and operational constraints. The risk is that attackers who can observe benign conversations (evidence) may construct stronger indirect prompt injections without increasing the attackers' control over prior turns or the later injection. However, we still lack a clear study of how effectively attackers can select and use this evidence. To address this gap, we introduce EvoAmbush, a red-teaming framework in which an attacker silently observes an existing benign conversation, extracts evidence, and constructs an injection. The challenges are twofold: (a) selecting useful evidence and (b) presenting it as reasons for an unauthorized action, without changing the prior benign conversation. To efficiently exploit (a) the effect of evidence on attack success, our framework selects evidence across 2 dimensions: User/System Preferences with 3 selection policies. And to enhance (b) the process of turning the selected information into the malicious request, we introduce an Urgency/Priority matrix with 4 rewriting strategies. Across the evaluated conditions, we reach 67.4% ASR on AgentDojo and 87.3% on InjecAgent. These results show that attackers can use benign interaction evidence in later attacks without first poisoning the benign conversation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Indirect Prompt Injection, LLM Agent, Red-Teaming, Self-Evolution, LLM Security

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_vPiDOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies an indirect prompt injection (IPI) threat model in which an attacker has read-only access to a prior benign user–agent conversation and can later place exactly one malicious payload in an untrusted channel, without modifying earlier turns. The proposed framework, EvoAmbush, uses an LLM to extract “source-verifiable” evidence items from the benign prefix (six categories; three selection policies), rewrites the attack goal into one injection under one of four urgency/priority conditions (Low-Impact reference, Strategic, Time-Sensitive, Critical), and annotates victim refusals post hoc (“reflection”).

Critical strengths
  1. The threat model is clean and under-studied. Separating read-only observation of benign context from the ability to poison it is a meaningful distinction for memory- and retrieval-augmented agents, and the single-injection, no-feedback constraint makes the setting strict.

  2. The evaluation is broad and includes the right controls in principle: paired gain/loss counts with 95% intervals for the rewriting comparisons (Table 4), a 2×2 evidence × rewriting factorial (Figure 8), five defenses with utility reported alongside ASR, and an unusually candid limitations paragraph.

Critical weaknesses
  1. The main quantitative claims are not reliable as reported, and the comparison table cannot be verified. (a) The abstract and introduction state 67.4% ASR on AgentDojo. In Table 1 and Table 3, 67.4 is the UA value for Qwen3.5-27B (cells are ordered ASR/UA/JSR); the corresponding ASR is 33.8. The headline number appears to be a misreported metric. (b) Table 1 does not state which victim it reports (it matches the Qwen3.5-27B row of Table 3), mixes decimal precisions across methods (e.g., 19.66/1.59/36.07 for Muzzle vs. one-decimal elsewhere), and Section 6 says “literature rows remain source-labeled and are never presented as local reproductions” while no row in Table 1 is labeled as such. Similar mismatch appears elsewhere, which makes me very confused. (c) Six victims are claimed; four appear in the method comparison, Qwen3.6-35B-A3B only in the policy heatmaps, and Qwen3-235B-A22B appears nowhere. (d) Section 6 itself states that campaigns use different cohorts, providers, checkpoints, and denominators, so cross-table comparisons are limited. Taken together, I cannot determine whether the central claim—that EvoAmbush beats existing attacks on matched victims—is supported.

  2. The main idea of the paper is already very well-established. The concept that indirect prompt injection (IPI) attacks become significantly more effective when tailored to context or presented with logical justifications is well-explored in existing literature. (e.g., https://arxiv.org/abs/2302.12173) While the authors reframe this as "observing benign conversations without prior poisoning," the underlying idea is a well-recognized vulnerability in context-aware LLM agents.

  3. The evidence-specific contribution is small, confounded with urgency framing, and the method is under-specified. The paper’s own factorial (Figure 8) gives Default 24.6, evidence-only 29.5 (+4.9), rewriting-only 38.0 (+13.4), both 44.6. Most of the gain comes from the urgency/consequence rewriting, which is a well-known IPI tactic and is already the mechanism behind AgentDojo’s “important instructions” attack.

Questions for authors
  1. Please confirm whether 67.4% in the abstract is ASR or UA.

  2. How is the synthesized five-message prefix inserted into the victim’s context, and do the Benchmark attack and all baselines see the same prefix?

  3. Please define the three Evidence Extracting Policies operationally, name the attacker LLM, and report what the reflection stage produced (e.g., distribution of refusal reasons by evidence/relation/action). Where are the Qwen3-235B-A22B results?

  4. Why are AgentVigil’s ASR/JSR identical to Benchmark on three suites?

Reviewer_ZJU2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents EvoAmbush, a red-teaming framework for indirect prompt injection attacks against LLM agents. The framework considers an attacker who can observe previous benign user-agent conversations without modifying them and subsequently inject a single malicious message through an untrusted channel. EvoAmbush extracts and selects evidence from benign conversations using three selection policies and incorporates the selected evidence into attack messages using different rewriting strategies based on urgency and priority. The paper also introduces a reflection stage to analyze victim responses and rejection reasons. Experiments on AgentDojo and InjecAgent report that EvoAmbush generally achieves higher attack success rates than the evaluated baselines, with additional analyses of evidence selection, rewriting strategies, and defenses.

Critical strengths
  1. The paper raises a threat in which an attacker observes prior benign conversations, without modifying them, and reuses their content to justify unauthorized actions. This is a meaningful threat model given that agents increasingly rely on long-term conversation history and memory.

  2. The paper analyzes evidence selection and rewriting separately and evaluates them on two benchmarks, multiple victim models, and several defenses. In particular, the ablation that separates the presence of evidence and rewriting lets the reader examine each component’s contribution. It is informative even where its results diverge from the authors’ claims.

Critical weaknesses
  1. The central claim, that evidence extracted from benign conversations provides attack leverage, is not sufficiently supported by the experimental results. The ablation separating evidence and rewriting shows some contribution from evidence. However, its additional effect is smaller than that of rewriting and is not consistent across defense conditions. In addition, Figure 6, which reports gains by rewriting policy, uses a reference condition without evidence, so the effects of evidence and rewriting are conflated. Some conclusions rely on small samples or single results, and uncertainty is not adequately reported. It is therefore difficult to accept the strong claim that evidence reuse is the key attack mechanism. A substantial part of the observed effect may stem from well-known urgency framing.

  2. Insufficient specification of the experimental setup, together with reporting errors, makes the results difficult to interpret. The source datasets of the benign conversations (ABCD and Taskmaster) barely overlap with the evaluation domains of AgentDojo and InjecAgent. The paper does not explain how the two are combined or whether the victim agent actually sees the benign conversation, so it is unclear how the threat model is implemented in the experiments. The headline AgentDojo ASR in the Abstract and Introduction matches the UA value in Table 1, and no condition in the paper attains that ASR. The InjecAgent Time-Sensitive figure reported in the Introduction cannot be found in any table or figure. The two panels of Figure 4 are identical. Table 1 and Figure 8 do not specify the victim model or case set, so it is unclear what scope the numbers presented as general results actually cover, or whether they are comparable to each other. Finally, the main text provides no quantitative results supporting the contributions on reflection analysis and attack chains.

Questions for authors
  1. ABCD consists of customer-service dialogues about accounts and services, and Taskmaster covers pizza ordering, ride booking, and movie ticketing, whereas AgentDojo consists of banking, Slack, workspace, and travel environments. How is each synthesized benign conversation paired with an evaluation task? Is the conversation provided to the victim agent as its own prior interaction, and if so, in what form (system prompt, conversation history, memory, etc.)? If the victim does not see the conversation, the cited evidence is hard to distinguish from generic fabricated context.

  2. How is rewriting implemented in the "Without Evidence" condition? All rewriting policies are defined in terms of the selected evidence.

  3. Is the evidence ablation in Figure 8 controlled for the same cases, victim model, and rewriting policy? Can you report case counts and confidence intervals?

  4. Which victim model and case set do Table 1 and Figure 8 each correspond to?

  5. Were the baselines in Table 1 run with the same victim models and the same benign-conversation context as EvoAmbush?

  6. What is the source of the InjecAgent Time-Sensitive figure reported in the Introduction?

  7. Can you provide quantitative results for the reflection analysis?

Reviewer_1UXFOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces EvoAmbush, an indirect prompt injection framework that exploits evidence extracted from prior benign user/agent interactions to construct persuasive malicious attacks. The framework combines goal-aware evidence selection to make unauthorized actions appear as legitimate continuations or necessary resolutions of previous tasks. Experiments on several benchmarks like AgentDojo and InjecAgent show attack success rates of 33.8% and 87.3%, respectively. This demonstrates the potential risks of reusing benign conversational context in subsequent agent interactions.

Critical strengths
  • The paper shows a security risk in which attackers exploit information from existing benign conversations without modifying prior interactions. This read-only observation setup is somewhat interesting and relevant to LLM agents that maintain conversational histories or persistent memory.

  • The framework examines different evidence categories, extraction policies, and urgency/priority-based rewriting strategies, providing empirical insights into how contextual information and attack framing interact to influence injection success.

Critical weaknesses
  • At first, I found the novelty of the paper is somewhat limited when I checked several related works on indirect prompt injection. ([1] Chang et al., ICLR 2026 and [2] Anonymous et al., LogicInception: Discourse-Level Indirect Prompt Injection via Causal Chain Construction , ARR May Submission) Although the paper cited [1], the paper did not cite LogicInception [2] and also did not discuss the main difference for some point in [1]. EvoAmbush seems to be an incremental extension of prior persuasion-based and context-aware attacks. ChatInject already explores persuasive dialogue construction, while LogicInception investigates how causal dependencies can make unauthorized actions appear necessary for legitimate task completion. EvoAmbush primarily differs by extracting contextual evidence from prior benign conversations rather than constructing persuasive contexts synthetically. However, it remains unclear whether this constitutes a fundamentally new attack mechanism.

  • The paper did not adequately distinguish the effect of genuine evidence extracted from prior conversations from that of plausible but fabricated contextual information. In particular, the ablation results show that rewriting without evidence already provides substantial improvements, while adding evidence slightly increases ASR. Without comparisons against fabricated, mismatched, or synthetic evidence under controlled rewriting conditions, the benefit of observing authentic conversations remains unclear and I hope this to be updated.

  • EvoAmbush improves overall ASR over ChatInject by only around 1.5 percentage points. While the performance gap itself is not critical, if the paper cannot distinguish the key difference between ChatInject, this marginal difference can be important. Moreover, the paper acknowledges differences in experimental cohorts, models, and execution protocols across campaigns. The absence of matched comparisons with conversation-conditioned versions of existing attacks makes it difficult to establish a meaningful performance advantage.

  • The proposed extraction policies and urgency/priority-based rewriting strategies seem heuristic, without a clear theoretical or algorithmic justification. Their effectiveness also varies substantially across victim models and strategy combinations, making it difficult to derive generalizable insights about which evidence types or rewriting strategies are consistently effective.

[1] Chang et al., ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents, ICLR 2026 [2] Anonymous et al., LogicInception: Discourse-Level Indirect Prompt Injection via Causal Chain Construction, ARR May Submission

Questions for authors
  • What is the fundamental methodological distinction between EvoAmbush and existing persuasion-based or causal-dependency-based prompt injection methods, such as ChatInject and LogicInception as stated in W1? In particular, would conversation-conditioned variants of these existing approaches achieve comparable performance when given access to the same benign interaction history?

  • How can you establish that the observed improvements specifically result from using authentic evidence from prior conversations rather than from persuasive rewriting or plausible contextual claims? Could you compare authentic, fabricated, and mismatched conversational evidence while controlling for payload length, rewriting strategy, and attack objective?

1 comment
AuthorsOpenReview ↗

Novelty Concerns and related works

We accept the reviewer's criticism that the paper did not present the relationship to prior work clearly enough, and we will revise accordingly: LogicInception will be cited and discussed, and a dedicated related-work paragraph will map EvoAmbush against both ChatInject and LogicInception. However, we respectfully but firmly disagree with the conclusion that the contribution itself lacks novelty. The distinction is methodological, not presentational, and it is threefold:

  1. Attacker capability. ChatInject fabricates multi-turn conversations and abuses chat templates to impersonate roles; LogicInception constructs synthetic causal chains to make an action appear necessary. EvoAmbush's attacker has a strictly weaker capability: read-only observation of an existing, frozen benign conversation, with no ability to author or modify any prior turn. The evidence it cites is source-verifiable — a verbatim span of a conversation that genuinely occurred — rather than fabricated context.
  2. What is being measured. Because the evidence is real and traceable, the victim (and any provenance-checking defense) cannot dismiss the cited premise as fabricated. This is precisely the security gap our defense analysis targets: provenance establishes origin but not permission. A synthetic-causal-chain attack does not create this phenomenon, because its premises are attacker-authored. This is a new security phenomenon, not a new persuasion recipe.
  3. Conversation-conditioned baselines. The reviewer asks whether conversation-conditioned variants of existing attacks would match EvoAmbush. This is an excellent suggestion; we will add a control in which ChatInject-style construction is given access to the same observed benign conversation, so the comparison is matched on context exposure.、

In short: to our knowledge, EvoAmbush is the first framework to formulate one-shot IPI as context observation under a fixed goal, with a formal contract (frozen prefix τ_b, one injection point t*, N_payload = 1) that none of the cited works imposes. The reflection stage, which links victim-stated refusal reasons back to evidence items and asserted relations, is also absent from ChatInject and LogicInception. The revision will make this case explicitly rather than leaving it implicit.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.