acceptodds
Under review as a conference paper at ICLR 2027

Who Is Your Agent Serving? Provider-Side Indirect Prompt Injection in Proactive Agents

Abstract

Proactive personal agents increasingly decide what to recommend, how to personalize advice, and what follow-up assistance to offer, creating a new user-decision attack surface for provider-side indirect prompt injection. We show that an external provider need not access private user context, compromise the agent, or gain additional permissions: by controlling only content associated with its own target, it can redirect an otherwise benign agent to advance that target, recruit legitimately available user context to justify it, and proactively reduce the friction of adoption. We characterize this failure mode through Target Control, Private Binding, and Prospective Support, which respectively steer what the agent advances, how it connects the target to the user, and what target-specific assistance it offers next. Across three proactive-agent environments and six simulated user models, the full attack increases target authorization in all tested environment-user-model combinations, with a macro gain of up to 77.4 percentage points. Controlled replay shows that correct user-target binding is more consequential than additional proposal detail alone, while a multi-turn extension reveals that provider objectives can remain influential even without final authorization by reshaping how the agent responds to user constraints and resistance. These findings expose a broader trust boundary: capabilities designed to serve the user can be redirected toward objectives originating outside the user–agent relationship.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
indirect prompt injection, LLM agents, proactive agents, agent security, human-AI decision making

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_2UniOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This work studies provider-side indirect prompt injection in proactive agents, in which malicious providers might advance their own target by changing only the context relevant to that target, without accessing the private user context or gaining additional permissions.

The core of the attack design is target control, private binding, and prospective support.

Using 3 proactive-agent environments and 6 different models as user simulators, the authors show that the attack increases the target authorization rate by 77.4 percentage points.

Critical strengths
  • The studied problem seems interesting and timely to me, as we are about to see more and more proactive agents in the future.
Critical weaknesses
  • There is no effort to demonstrate the results transfer to real users. We already knew that LLMs are vulnerable to prompt injection attacks, so the results with simulated users provide limited information here.
  • The proposed approach is poorly documented in the main paper. Without a clear definition of proactive agents and their environment, I cannot understand how the proposed approach is implemented at all.
Questions for authors
  • What is the biggest challenge for prompt injection targeting proactive agents, compared to general agents?
Reviewer_5vufOpenReview ↗
Rating
2(+1)
Confidence
3(−1)
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies provider-side indirect prompt injection against proactive personal agents. A provider controls only agent-readable content associated with its own target, yet can steer the user's assistant toward recommending that target. The authors decompose the attack into target control, private binding, and prospective support, and provide representative injection constructions that instantiate them. Experiments on three proactive-agent environments use 144 task–profile–difficulty blocks drawn from 24 test task families (36 families total; 12 held out for development) and six simulated users, with GPT-5 as the primary assistant. The full attack reports macro-average gains of up to 77.4 percentage points in added target authorization. A separate 25-task study examines whether the effect persists across multi-turn interaction.

Critical strengths

1.The paper formalizes a realistic setting: a provider changes only content associated with its own target. It measures the user's final decision, not just the agent's output.

2.The main results are easy to read.

Critical weaknesses
  1. Authorization means one model agreeing with another, not a real person deciding. The authors admit this in Section 4.6. This limits how far the main claims generalize.

  2. The main episodes allow only one proposal and one decision, which is close to a single-turn setting. The only multi-turn evaluation is the 25-task PARE study, which remains simulated. There is no reactive baseline, so the paper cannot isolate whether the effect depends on proactivity.

  3. The paper relies on an LLM judge (gpt5.6-sol) with unspecified human checks, and reports no agreement statistics. There are no reruns, confidence intervals, or significance tests. The robustness of small differences is therefore unclear.

  4. T is inherited from prior work, and B and S are essentially natural-language instructions. The method is a simple combination of existing prompt-injection patterns and standard recommendation steps. The main contribution is an evaluation framework, not a new technique. Your contribution is very limited.

  5. The main candidate setting is synthetic and effectively single-turn; the long-horizon PARE study is multi-turn but still simulated. The paper does not evaluate defense layers such as content filtering, instruction hierarchy, or action approval. It is unclear whether the findings transfer to deployed agents. The paper therefore offers little guidance for real deployment or defense.

Questions for authors

1.Can you add a standard reactive recommendation-agent baseline? In the single-turn protocol, what is the essential difference between the proactive frameworks and a reactive baseline? If the two are close, how does the claim of a new attack surface hold?

2 Can you add B-only, S-only, and B+S conditions, together with a length-matched but content-irrelevant injection control, to separate main effects, interactions, and text length?

  1. What fraction of the outputs was human-checked, and what inter-annotator agreement, such as Cohen's κ, was obtained?

  2. Do T, B, and S remain effective against real agents that include anti-injection system prompts or action-approval mechanisms?

  3. The threat model is not limited to personal consumption decisions. Coding agents also read provider-controlled READMEs, dependency documentation, and plugin descriptions. Do the authors expect T, B, and S to hold there? If yes, why do the experiments cover only recommendation tasks? If not, what is the key difference?

1 comment

The authors provided no rebuttal. My concerns remain unaddressed, so I maintain my original score.

Reviewer_GL57OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies provider-side indirect prompt injection against proactive personal agents. An external provider embeds instructions in its own product's content, steering a benign agent to recommend that product and justify it with the user's private context. The authors evaluate this attack on a synthetic benchmark across three proactive-agent environments, with six LLMs as simulated users, and find that it increases target authorization in all tested settings.

Critical strengths
  • The topic is important and the framing is clear. The attack difficulty level is clearly defined and it can provide a more nuanced understanding of how each contributes to the attack's effect.
  • I also appreciate that the authors do not stop at agent-side behavior but also evaluate downstream user decisions, albeit with simulated users, and that they attempt to study long-horizon steering across multi-turn dialogue.
Critical weaknesses

My main concern is the validity of the LLM-simulated users. The authors acknowledge that the results do not reflect real users' reactions, yet the paper's main claims and findings concern how users react. Although several models are used as simulated users, the paper provides no evidence or discussion of whether LLMs are valid proxies for users in this setting. Since how the user reacts to the agent is one of the most important parts of this study, this gap limits what the results can support.

Questions for authors
  1. In sec 4.1, the authors state that "All users are simulated, so authorization rates characterize benchmark behavior rather than effects on real users." However, the conclusions claim that an external provider can influence users' decisions through the agent. These two statements are inconsistent: if the results do not represent effects on real users, they only show that the agent's proposals influence the decisions of another LLM agent. I think the authors could still discuss to what extent a simulated user's acceptance of a recommendation reflects a real user's decision. For example, could the authors conduct a validation study with human participants on a subset of the data and compare their decisions with those of the LLM-simulated users?

  2. How are the simulated users built? The paper does not describe them or show their prompts. For example, what information does a simulated user see when making its decision?

  3. Each task family has two user profiles, but the appendix shows only one. How do the two profiles differ, and does this difference affect the results?

  4. Was the difficulty design validated?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.