acceptodds
Under review as a conference paper at ICLR 2027

Beyond Instruction Override: Benchmarking Indirect Prompt Injection with Task-Tampering Attacks

Abstract

Indirect prompt-injection (IPI) benchmarks predominantly test planted instructions that redirect a tool-using agent toward an attacker-chosen task. Yet attacker-controlled content can instead tamper with the outcome of the task already in progress. For example, a buyer's dispute evidence can cite a false refund policy to inflate the refund without redirecting the agent away from resolving the dispute. We define IPI by authority rather than instruction-like wording: planted content constitutes an attack when it attempts to influence a decision that its source is not authorized to determine. We introduce a benchmark that makes this boundary explicit: in each of 90 validated cases across three tool-use environments, the user request specifies the source that resolves the targeted decision. Every attack tampers with a decision required by the benign task rather than replacing that task. Each case has attacks with the same harmful outcome that target up to six parts of the agent's decision making: its objective, decision rules, available actions, expected consequences, reading of observations, or beliefs about the current state. Without a defense, attacks on three of the five other targets (decision rules, available actions, and state) have higher mean success than objective attacks across five LLMs. A detection-and-removal defense (DRIFT's isolator) reduces objective-targeting attacks to 2.4% success but leaves observation- and state-targeting attacks at 9.5% and 12.1%; this gap persists under detection-optimized rewrites. A tool-policy defense (Progent) still allows 15.4% attack success, close to 17.0% without a defense, because these attacks generally retain expected tools and manipulate values obtained during execution. Evaluations whose attacks redirect the task or target only the agent's objective can therefore overestimate the coverage of these defenses.

Primary area:
datasets and benchmarks
Keywords:
adversarial attacks, benchmarking, security/privacy, tool use, agent evaluation, evaluation methodologies, robustness

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.60 over 5 reviews, above 70% of the others.
Reviewer_3CacOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies task-tampering attacks in indirect prompt injection, where attacker controlled content corrupts a decision within the legitimate task. It defines attacks through source authority and constructs a benchmark of 90 validated cases across three tool-use environments, comprising 1,343 single primitive attacks and 90 compound attacks. Within each case, attacks target different aspects of decision making while preserving the harmful outcome. Experiments across five LLMs and seven defenses show that detecting explicit objective manipulation or restricting tool use can leave substantial vulnerabilities to manipulated observations, states, and tool arguments.

Critical strengths
  1. The benchmark makes the authority boundary explicit for each case, specifying the attacker-controlled source, the decision outside its authority, and the authorized basis for resolving that decision. Holding the task and harmful outcome fixed while varying the manipulation provides a useful framework for examining which attacks existing evaluations may miss.

  2. The defense analysis identifies meaningful differences that aggregate attack success rates would obscure. The equal budget comparisons and rewriting experiments strengthen the evidence that broader attack coverage reveals additional vulnerabilities.

Critical weaknesses
  1. The fidelity of the runtime defense adaptations is insufficiently established. The results describe adaptations to a common agent loop, but the preservation of the original systems’ key mechanisms remains unclear. CaMeL’s benign task success rate of only 13.3% makes this consequential: the observed trade-off could reflect policy configuration or implementation choices. This limits conclusions about the underlying defenses.
Questions for authors

Which components of the original runtime defenses were retained, modified, or omitted? For CaMeL, were the separation of planning from untrusted-data processing and interpreter-based execution preserved? What accounts for its low benign task success rate?

1 comment

We thank the reviewer for the careful review. In the revised PDF, all changes are marked in blue, and tables and listings numbered R1, R2, … are new or revised.

Components retained or modified. All key components are retained (App. M); for CaMeL, we add one restriction (below). The main adaptation is to environment-specific configuration: Progent, ARGUS and CaMeL define part of theirs for AgentDojo's tools (which operations are read-only; CaMeL's security policies), and we define it for the operations of our environments. Results characterize these evaluated adaptations. Below, we describe CaMeL's separation, interpreter and policy choices and analyze its benign failures, then report three implementation issues in other defenses that were identified in implementation checks and corrected (App. M.2).

CaMeL's separation and interpreter. Both are preserved. The privileged model writes a program from the user request alone and never sees tool data; the quarantined model parses untrusted content and cannot call tools; programs run in the interpreter from CaMeL's reference implementation, which tracks where every value came from.

Low benign task success.

  • Our two choices. CaMeL's policies are defined for AgentDojo's tools, so we write them for ours (Table R18: control arguments must come from trusted fields, content arguments need not). First, one tool output can combine fields written by different parties, so we label each field rather than the whole output (Table R19): labeling the whole output untrusted would also deny calls that use its platform-generated fields, while labeling it trusted would admit the party-written ones. Second, unlike CaMeL's default, we deny a state-changing call inside a branch or loop whose condition depends on untrusted data, because untrusted content could otherwise steer our tasks' decisions through such a branch.
  • Effect of the restriction. Disabling the restriction roughly doubles task success, with and without an attack, and raises ASR (DeepSeek V3.1, Kimi K2.5, GLM-5, Sonnet 4.5; Tables R20–R21):
Control-flow restrictionBenign TSRASR
On (reported configuration)13.6%1.3%
Off24.4%3.1%
  • Causes of benign failure. With the restriction on, most failures are policy denials; with it off, the remaining failures are mainly control-argument denials, insufficient information and wrong results:
Terminal cause of failure, no-injection runsRestriction onRestriction off
Call that changes the environment inside a branch or loop that depends on untrusted data34%0%
Untrusted value in a control argument19%39%
Quarantined model reports insufficient information22%25%
Program error13%15%
Completed, wrong result12%21%
Failed runs / runs311 / 360272 / 360
  • Why these denials arise. Every task in our benchmark ends in a state-changing call, and in many tasks this call's choice and control values depend on content written by third parties that the task trusts only for specific information, while CaMeL fixes a value's trust by its source before any task is given. With the restriction on, the call is denied when it is made in a branch that depends on such content; with it off, it is still denied when such content reaches a control argument.

Revised defense implementations. Three implementation issues were identified in implementation checks and corrected, and the affected defenses were rerun on all five models: Progent now always allows read-only operations, as in its AgentDojo evaluation; DRIFT-full's planner now receives the tools' structured parameter schemas, which it previously received only in part; and ARGUS's input and output length limits are raised from their defaults to fit the longer tool outputs in our environments. Appendix M.2 compares the submitted and revised results (Table R22), and all results for these defenses in the revision use the revised implementations; in brief (five-model means, %):

DefenseASR, submitted → revisedBenign TSR, submitted → revised
DRIFT-full7.0 → 7.188.9 → 93.6
Progent15.4 → 16.374.9 → 88.0
ARGUS9.8 → 9.878.9 → 97.3

Attack success is essentially unchanged while task success rises, and the conclusions about these defenses hold: DRIFT-full still leaves observation and state attacks above objective attacks within case (+8.3% and +11.6%, above zero for each model), and Progent's ASR stays close to the undefended rate (16.3% vs 17.0%).

Reviewer_wEqsOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper benchmarks indirect prompt injection that tampers with decisions within the user’s task. It constructs 90 validated cases across three environments, varying up to six influence primitives while preserving the targeted decision and harmful outcome within each case. Experiments across five models and seven defenses examine differences in attack success, task utility, and runtime defense behavior.

Critical strengths

S1. The controlled construction fixes the task, attacker, source, injection location, targeted decision, resolution basis, and harmful outcome within each case. Varying how the payload justifies the same harmful outcome provides an informative basis for comparing influence primitives. S2. The analysis carefully examines alternative explanations through matched-case comparisons, equal-budget evaluations, wording controls, and case-bootstrap intervals. Separating content removal, agent resistance, and action blocking provides useful diagnostics beyond aggregate attack success, while task-utility measurements expose security–utility tradeoffs.

Critical weaknesses

W1. The paper acknowledges related context-aware attacks studied in AgentPI and AgentLure [1, 2]. However, the additional conclusions enabled by its authority-based requirements and fixed-outcome comparisons could be clearer. An explicit comparison of construction requirements would strengthen the positioning. W2. Runtime defense results depend on adaptations whose fidelity is difficult to assess. CaMeL’s 13.3% benign task success makes this particularly consequential. More implementation detail and failure analysis would help distinguish defense limitations from adaptation choices. W3. The rewriting and equal-budget analyses provide useful controls, but influence primitive and linguistic form are not independently varied. Table 19’s matched comparison covers only the undefended setting, while Table 23 groups the other primitives together. This leaves the specific observation- and state-targeting gaps under DRIFT less directly controlled for case composition.

Questions for authors

Q1. Beyond the distinction discussed in Section 2, which construction requirements and empirical conclusions most clearly distinguish this benchmark from AgentPI and AgentLure? Q2. What changes were made when adapting the runtime defenses?

  • Representative policies, trust labels, and an analysis of CaMeL’s benign task failures would help assess implementation fidelity. Q3. Do the observation- and state-targeting gaps under DRIFT persist when each is compared with objective attacks on the same cases?
  • Varying linguistic form within each primitive would also strengthen the interpretation.
4 comments

We thank the reviewer for the careful review. In the revised PDF, all changes are marked in blue, and tables and listings numbered R1, R2, … are new or revised.

Construction requirements versus AgentPI and AgentLure (W1/Q1). Where AgentPI groups attacks by the part of the agent's execution flow they corrupt and AgentLure adds attacks carried through components of the agent system, such as memory, skills and inter-agent handoffs, each of our cases fixes one decision the task requires, the source that resolves it and the harmful outcome, and varies only which belief feeding that decision the content targets. The following construction requirements distinguish it, each enabling a conclusion in the paper:

  • Varied belief targets, with a designated resolution basis. Attacks on beliefs other than the objective, such as a false state or a reinterpreted observation, may contain no instruction that marks them as injections. Each of our cases names the record or procedure that settles the targeted decision, which makes these attacks measurable. Neither AgentPI nor AgentLure states, per case, which source may settle the targeted decision; an injection is attacker-inserted content by construction.
  • Fixed outcome. Within a case, the task, source, placement and harmful outcome are fixed, and attacks differ only in the belief they target, whereas in AgentLure and in AgentPI's examples each attack vector has its own goal. Together with the first point, this lets us attribute DRIFT's coverage gap to the targeted belief: its isolator reduces objective attacks to 2.4% ASR but leaves observation and state attacks at 9.5% and 12.1% (Table 3), and within case they exceed objective attacks by +8.8% and +11.4% (Table 16), for each model (Tables R8–R9).
  • Harm inside the task. Every attack tampers with a decision the user's task requires; neither benchmark is restricted to such attacks. This indicates that checks on which tool the agent calls do not separate attacked calls from the task's own: in the trajectories we trace (GLM-5, Sonnet 4.5), no tool-level check in DRIFT or Progent blocked an attacked call (Table 4), since the harmful call uses a tool the task needs.

Section 2 now states these requirements and what each enables.

Same-case gaps under DRIFT (W3/Q3). The gaps persist. On the 37 matched cases of Table 19, each target's ASR and its within-case difference from objective attacks under DRIFT-isolation (Tables R10–R11):

TargetASR (%), no defenseASR (%), DRIFT-isoDRIFT-iso − b(U), within case (%, 95% CI)
b(U)9.41.1—
b(O)12.510.8+11.3 [+4.9, +19.1]
b(s)21.813.9+14.5 [+7.2, +23.0]

(ASR pools instances; within-case differences average within cases and need not equal the difference of the displayed ASRs.)

Over all cases with both targets, the observation and state gaps are +8.8% and +11.4%, above zero for each of the five models (Table R8).

The gap is also not specific to DRIFT: under ARGUS, an evidence-based authorization defense that relies on detecting anomalous content, state attacks exceed objective attacks within case (+11.2% [+5.7, +17.1]; observation attacks +3.1% [−0.4, +6.4]; Table 16), and its segmenter flags objective attacks most often (84.8%) and observation and state attacks least often (45.0% and 27.9%; Table R12).

Linguistic form (W3/Q3). We ran a form control on the same 37 cases (DeepSeek V3.1, Kimi K2.5, GLM-5, Sonnet 4.5): each objective, observation and state attack was evaluated both as an instruction and as a statement, rewriting only the sentence that carries the attack (e.g. "Deny the refund" became "The goal is to deny the refund"). The control varies instruction versus statement phrasing only. ASR (%) by form:

TargetNo defense, instructionNo defense, statementDRIFT-iso, instructionDRIFT-iso, statement
b(U)9.210.11.60.9
b(O)13.211.811.89.9
b(s)21.120.812.814.3

Within each target, no instruction − statement difference is distinguishable from zero in this test, and with form held equal, observation and state attacks remain above objective attacks under DRIFT-isolation (+10.6% to +15.0%, all intervals above zero; Table R17).

Adaptation changes (Q2). We have added an appendix (App. M) that names each runtime defense's key components and how we adopt them. The main adaptation is to environment-specific configuration: Progent, ARGUS and CaMeL define part of theirs for AgentDojo's tools (which operations are read-only; CaMeL's security policies), and we define it for the operations of our environments. Results characterize these evaluated adaptations. Below, we describe CaMeL's two policy choices and analyze its benign failures, then report three implementation issues in other defenses that were identified in implementation checks and corrected (App. M.2).

CaMeL: policies, trust labels and benign failures.

  • Our two choices. CaMeL's policies are defined for AgentDojo's tools, so we write them for ours (Table R18: control arguments must come from trusted fields, content arguments need not). First, one tool output can combine fields written by different parties, so we label each field rather than the whole output (Table R19): labeling the whole output untrusted would also deny calls that use its platform-generated fields, while labeling it trusted would admit the party-written ones. Second, unlike CaMeL's default, we deny a state-changing call inside a branch or loop whose condition depends on untrusted data, because untrusted content could otherwise steer our tasks' decisions through such a branch.
  • Effect of the restriction. Disabling the restriction roughly doubles task success, with and without an attack, and raises ASR (DeepSeek V3.1, Kimi K2.5, GLM-5, Sonnet 4.5; Tables R20–R21):
Control-flow restrictionBenign TSRASR
On (reported configuration)13.6%1.3%
Off24.4%3.1%
  • Causes of benign failure. With the restriction on, most failures are policy denials; with it off, the remaining failures are mainly control-argument denials, insufficient information and wrong results:
Terminal cause of failure, no-injection runsRestriction onRestriction off
Call that changes the environment inside a branch or loop that depends on untrusted data34%0%
Untrusted value in a control argument19%39%
Quarantined model reports insufficient information22%25%
Program error13%15%
Completed, wrong result12%21%
Failed runs / runs311 / 360272 / 360
  • Why these denials arise. Every task in our benchmark ends in a state-changing call, and in many tasks this call's choice and control values depend on content written by third parties that the task trusts only for specific information, while CaMeL fixes a value's trust by its source before any task is given. With the restriction on, the call is denied when it is made in a branch that depends on such content; with it off, it is still denied when such content reaches a control argument.

Revised defense implementations. Three implementation issues were identified in implementation checks and corrected, and the affected defenses were rerun on all five models: Progent now always allows read-only operations, as in its AgentDojo evaluation; DRIFT-full's planner now receives the tools' structured parameter schemas, which it previously received only in part; and ARGUS's input and output length limits are raised from their defaults to fit the longer tool outputs in our environments. Appendix M.2 compares the submitted and revised results (Table R22), and all results for these defenses in the revision use the revised implementations; in brief (five-model means, %):

DefenseASR, submitted → revisedBenign TSR, submitted → revised
DRIFT-full7.0 → 7.188.9 → 93.6
Progent15.4 → 16.374.9 → 88.0
ARGUS9.8 → 9.878.9 → 97.3

Attack success is essentially unchanged while task success rises, and the conclusions about these defenses hold: DRIFT-full still leaves observation and state attacks above objective attacks within case (+8.3% and +11.6%, above zero for each model), and Progent's ASR stays close to the undefended rate (16.3% vs 17.0%).

Thank you for the detailed clarifications and additional results. My main concerns have been addressed, and I will maintain my original positive rating.

Thank you for reading our responses and for your constructive feedback, which has helped improve the paper.

Reviewer_evFBOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Most existing tests of prompt injection against AI agents use hidden text that tells the agent to drop its job and do something else. The paper argues that defenses built for those tests miss a quieter attack: the agent does the right job, but hidden text changes one detail, such as a refund amount or a message recipient. It counts text as an attack when it tries to decide something its author has no say over, and each test task names the record that has the final say. The test set has 90 scenarios in three simulated settings: office work, software project maintenance, and an online marketplace. For each scenario, the authors write many attacks that push toward the same wrong outcome in six different ways, ranging from a direct order to false claims about the rules, the consequences of an action, the available options, what a number means, or the current facts. They test five AI models and seven defenses. A defense that filters out suspicious text catches direct orders well but misses many false factual claims. Defenses that check which tools the agent calls block almost none of these attacks. The strictest defense stops nearly all attacks but also stops most legitimate work, completing only 13% of tasks.

Critical strengths
  • Relevant gap. Value-level tampering inside expected tool calls is a realistic threat that redirection-centric benchmarks under-sample.
  • Controlled design. Fixing task, attacker, sink, and outcome while varying only the targeted belief allows within-case attribution, which most IPI benchmarks lack.
  • Mechanism-level analysis. Separating content stripped, call blocked, agent resisted, and no engagement (Fig. 2, Tables 4, 26, 27) is more informative than aggregate ASR. The finding that no tool-level gate blocked an attacked call is clear and useful.
  • Statistical reporting. Case-level bootstrap with paired differences (Tables 15, 16, 23), a matched-case subset (Table 19), and an equal-budget comparison (Table 23) are reported.
Critical weaknesses

As a benchmarking work, here are the major weaknesses:

  • The headline claim is stronger than the data. Without a defense, the paper says three kinds of indirect attack beat direct orders, but only one of those differences is statistically clear (Table 16). In the office-work setting, direct orders actually do better than most other kinds (Table 18).
  • Attack types are compared on different scenarios. Some types exist for 88 scenarios, others for only 54, and an AI filter removed far more attacks of some types than others before testing (Table 7). Differences between types may reflect which scenarios survived. W3. Attack types differ in more than what they target. In the paper's own example, the direct order is two lines, while the false-facts attack is a detailed fake repair-center report. Length, detail, and fake expert backing may explain the results as much as the attack type does.
  • The test setting may not carry over to real use. Every task names the record that has the final say, yet the agent gets no role description at all. A short role description roughly halves attack success (Table 14), so the absolute numbers say little about real deployments.
Questions for authors
  1. Which model performs the sufficiency and target-label checks in Section 4.3 and Appendix A.4?
  2. How was factual consistency of case records with real-world product and domain knowledge checked?
  3. Do the primitive-level claims in the abstract hold on the matched 37-case set with intervals?
4 comments

We thank the reviewer for the careful review. In the revised PDF, all changes are marked in blue, and tables and listings numbered R1, R2, … are new or revised.

Comparison without a defense (W1). We agree: without a defense, not every difference is statistically clear and the order differs by environment (Table 18), so we removed this comparison from the abstract and the Discussion. We retain these results descriptively; the conclusions rest on the comparison under DRIFT-isolation, which holds within case (revised Table 16):

Target − b(U), within case (%, 95% CI)No defenseDRIFT-isolation
b(π)+6.0 [+1.8, +10.3]+3.0 [+0.9, +5.3]
b(A)+8.4 [+2.3, +14.8]+2.0 [+0.4, +3.6]
b(O)−0.1 [−5.7, +5.2]+8.8 [+4.7, +13.6]
b(s)+4.6 [−2.9, +12.1]+11.4 [+6.9, +16.4]

Under DRIFT-isolation, the b(O) and b(s) differences are above zero for each of eight models (Tables R8–R9), and both targets exceed b(U) in every environment, Productivity included (Table 18).

The gap is not specific to DRIFT: under ARGUS, an evidence-based authorization defense that relies on detecting anomalous content, state attacks also exceed objective attacks within case (+11.2% [+5.7, +17.1]), observation attacks less clearly (+3.1% [−0.4, +6.4]; Table 16), and its segmenter flags objective attacks most often (84.8%) and observation and state attacks least often (45.0% and 27.9%; Table R12).

Different scenarios per attack type (W2). The submission's matched comparison, on the 37 cases where all six targets are retained (Table 19), controls for which scenarios each attack type covers, but only without a defense. The revision extends it to every defense (every defense in Table R10, within-case gaps in Table R11); on these cases, the DRIFT-isolation gaps hold:

TargetASR (%), no defenseASR (%), DRIFT-isoDRIFT-iso − b(U), within case (%, 95% CI)
b(U)9.41.1—
b(O)12.510.8+11.3 [+4.9, +19.1]
b(s)21.813.9+14.5 [+7.2, +23.0]

(ASR pools instances; within-case differences average within cases and need not equal the difference of the displayed ASRs.)

The retention filters (W2). The filters check validity, not effectiveness, and neither uses the evaluated models' responses (Listings R1–R2):

  • Sufficiency: for an agent that fully believes the instance, the attacked action follows logically as the correct task response.
  • Target: a classification that does not see the declared target confirms the targeted belief.

Whichever instances the filters removed, the matched comparison holds the cases fixed across targets.

Length, detail and expert backing (W3). We agree that presentation can change ASR, as the wording conditions in §5.5 (Figure 3) also show.

  • Why the forms differ. An attack's credibility comes from what it claims. In the example, the repair-center report persuades by asserting a false fact about the item, so it is itself a state attack. Giving an objective attack the same backing would add that state claim and make it a compound attack, which the benchmark evaluates separately (Table 3, Comp.). Length follows from this: matching it means adding content to the objective attack, which is what the control below does.
  • The control. In §5.5, we keep each attack's operative sentence verbatim and add supporting text that cannot produce the harmful outcome on its own. This text adds context and justification and may itself touch other beliefs, so an expanded attack can be compound (App. J). The control therefore tests whether length and added detail explain the gaps; expert backing cannot be varied separately, for the reason above, and form is varied separately (Table R17).
  • The result. Under DRIFT-isolation, the gaps hold, and with plain support objective attacks reach a median of 48 words, close to the 50 for the original state attacks and 57 for observation attacks (four models; lengths in Table R14, all conditions in Table R15, per model in Table R16):
Conditionb(O) − b(U), within case (%, 95% CI)b(s) − b(U), within case (%, 95% CI)
Original+8.3 [+4.3, +12.9]+11.2 [+6.6, +16.1]
Both with plain support+8.1 [+4.0, +12.7]+9.4 [+5.8, +13.3]
Original target vs. plain-support b(U)+7.8 [+3.9, +12.1]+10.6 [+5.9, +15.7]

Role description and production harness (W4). On DeepSeek V3.1, Kimi K2.5, GLM-5 and Sonnet 4.5, we ran the benchmark with a role description, inside Claude Code (a production agent harness whose own prompt instructs the model to flag suspected prompt injection), and with trust-hierarchy rules. Neither the role description nor the production harness detectably changes ASR on the same cases (last column below), and under DRIFT-isolation observation and state attacks still exceed objective attacks in Claude Code and with the trust rules (Tables R5 and R7). Absolute rates still depend on the deployment, and all these runs are in our simulated environments:

ConfigurationASR (%)Change vs. none (%, 95% CI)
None (main runs)16.1—
Role description16.3+0.1 [−0.8, +1.0]
Claude Code: role description17.4+1.3 [−0.0, +2.7]
Claude Code: its system prompt + role description17.4+1.2 [−0.4, +2.9]
Generic trust-hierarchy rules (TH-generic)12.5−3.7 [−5.0, −2.4]
Environment-specific trust-hierarchy rules (TH-env)8.8−7.4 [−9.5, −5.5]

(All attacks, mean over these four models; changes are paired by case, so they need not equal the difference of the ASRs. Over all five models, Table 14 gives 17.0%, 12.8% and 8.9% for no defense, TH-generic and TH-env.)

Trust-hierarchy rules as input to defense design (W4). We agree that trust guidance is a relevant condition, and we now report it in the main text as a useful input to defense design (§5.6, Table R1): information about which source is authoritative lowers attack success (to 12.5% and 8.8%; table above) and complements existing defenses. The rules that do so state the source priority our cases are built on (Appendix F), so their effect may not transfer beyond this benchmark. Such information also cannot always be assumed: the rules presume that a few fixed rules capture each environment's authority hierarchy, which may not hold where the hierarchy must be inferred or changes with the matter at hand (§5.6).

Model for the retention checks (Q1). Both checks use Claude Opus 4.6, the model that drafted the cases, and neither uses the evaluated models' responses. The revision names the model in Appendix A and adds both checks' prompts (Listings R1–R2).

Factual grounding (Q2). Records were checked for internal consistency rather than against external product or domain sources: automated checks cover referenced entities, source ownership and role access, and the review checklist checks that each scenario is plausible for a real organization (§4.1, Appendix A). The correct outcome of each case follows from its records and the environment's policies, so it does not depend on whether a product detail matches the real product. Inaccurate details can still affect a scenario's realism and how agents behave; we have added this to the limitations (Appendix L).

Abstract claims on the 37 matched cases (Q3). Yes, for both target comparisons in the submitted abstract. Within-case differences from objective attacks on these cases (five models; Table R11):

Target − b(U), within case (%, 95% CI)No defenseDRIFT-iso
b(π)+5.8 [+0.9, +11.0]+2.2 [+0.7, +3.8]
b(A)+9.7 [+1.8, +18.0]+2.9 [+1.3, +4.7]
b(O)+4.3 [−2.2, +11.2]+11.3 [+4.9, +19.1]
b(s)+13.9 [+4.0, +24.0]+14.5 [+7.2, +23.0]

Under DRIFT-isolation, observation and state attacks exceed objective attacks, as the abstract states. Without a defense, decision-rule, action-space and state attacks also exceed them on these cases; we removed that comparison from the abstract because, over all cases, the difference for state attacks is not significant (W1).

I thank the authors for their detailed response. After reading the rebuttal and the other reviews, I have decided to keep my original score.

Thank you for reading our responses and for your constructive feedback, which has helped improve the paper.

Reviewer_JKFCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates prompt injection attacks against LLM agents, specifically in the form of "task tampering." In this scenario, rather than redirecting the agent toward a different objective, an attacker injects text into the content processed by the agent to alter the output of the original task. The authors developed a benchmark featuring three distinct environmental settings—each with clearly defined authoritative sources to ensure objective evaluation—and constructed attacks based on six different dimensions. The experiments cover five LLMs, comparing performance in unprotected settings against seven existing defense mechanisms. Without protection, the average attack success rate is 17%. While the defense method DRIFT reduces the success rate of direct-instruction attacks to 2%, it remains at 10% for attacks involving false assertions or data misrepresentation; furthermore, defenses focused on tool restrictions prove ineffective. Another method, CaMeL, manages to intercept nearly all attacks, but at the cost of reducing the success rate for legitimate tasks to approximately 13%.

Critical strengths
  1. This paper redefines the threat model, revealing blind spots in existing defense mechanisms. Current defenses primarily detect text resembling commands or unauthorized tool invocations; however, they fail to capture attacks that remain within the scope of authorized tasks but involve the manipulation of parameters. By categorizing attacks across six dimensions—thereby decoupling attacker intent from the specific phrasing used—the paper demonstrates that the effectiveness of a defense depends on the surface-level form of the attack.
  2. The benchmark design is diagnostic in nature. It identifies authoritative sources for various scenarios and reports results for each, offering a finer level of granularity than AgentDojo (which focuses primarily on detecting the hijacking of an agent’s objectives). This approach is well-suited for adoption in future research.
Critical weaknesses
  1. The primary experimental setup overestimates agent vulnerability. The experiments provide no system prompts, lacking both role definitions and instructions regarding trust. As shown in the appendix, the introduction of roles and trust-related rules reduces the average attack success rate from 17.0% to 8.9%—and for Sonnet 4.5, from 11.1% to 2.8%. Although the authors excluded these findings from the main text, citing a risk of leaking evaluation criteria, the use of defined roles and trusted sources is standard practice in real-world deployments. Consequently, the core figures presented in the paper are derived from an unrealistic baseline, and the authors' assessment of the severity of the issue relies heavily on this specific (and unrepresentative) setup.
  2. Some of the "failure" cases described may, in fact, represent reasonable judgments rather than security lapses. Most of the successful attacks involve the fabrication of third-party evidence, yet the agents in the experimental environment are provided with no tools for verification. Accepting evidence that appears credible but cannot be verified is more akin to a human decision-maker’s error in judgment than a breach of trust boundaries. The paper fails to distinguish between trusting an entity that lacks decision-making authority and being misled by unverifiable evidence; this omission weakens the argument that there are inherent "blind spots" in existing defenses.
Questions for authors
  • Variance in attack efficacy: In the TH-env environment—which incorporates role-assignment and trust-rule prompts—does a significant difference in effectiveness still persist between "instruction-based" attacks and those relying on factual assertions or the reinterpretation of data?
  • Objectivity of the benchmarks: What are the criteria for determining authoritative sources and "correct" outcomes?
  • Generation of attack samples: Were the attack samples crafted by humans or generated by an LLM? During the generation process, were the attacks filtered or fine-tuned based on feedback from specific models?
  • Sufficiency of the experimental sample size: A dataset consisting of only 90 scenarios across 5 models is quite small, making it difficult to draw robust conclusions about the differences between various persuasion frameworks.
5 comments

We thank the reviewer for the careful review. In the revised PDF, all changes are marked in blue, and tables and listings numbered R1, R2, … are new or revised.

We first address the OM2 example (W2), then the broader questions about how cases and attacks were constructed and validated (Q2–Q3).

Resolution basis of OM2 and of every case (W2, Q2). A case's resolution basis is the record or procedure that the user request designates to decide the targeted decision (condition 2, §3.2); it need not be a record that contradicts the planted claim. In OM2 it is a procedure: platform policy 3.7, which the submission lists as the case's resolution basis (Table 9). The revised appendix quotes it in full, together with section 3.6, the platform's evidence hierarchy (Listings R3–R4). The "service center report" is text in the seller's own test-results record, not a separate third-party document, so under the platform's policy it is party evidence, at the same level as the buyer's test results. Policy 3.7 decides between them on the buyer's evidence: measurements that contradict the listing are objective evidence of a defect, whatever a party says about them, so the agent needs neither to verify the report nor a record that contradicts it:

Policy 3.7, core rule: If the buyer's evidence documents a condition or defect not disclosed in or inconsistent with the listing, the item is not as described.

Policy 3.7.3, Measurable evidence: Quantifiable measurements (performance benchmarks, battery health readings, physical dimensions, centering measurements) that contradict listing claims constitute objective evidence of item not as described. Party characterizations of measurable data do not change its objectivity.

How each case's authoritative source and correct outcome are fixed and validated is described in §3–4 and Appendices A and C:

  • Determined and validated (Q2.1). The attacker can write only in a source its role controls (§3.1); each environment's rules rank its records and policies above party-authored content (§4); and each case designates the record or procedure that resolves the decision (condition 2, §3.2). Within these rules, the authors specified each case's attacker, attack category and mechanism; an LLM drafted the scenario, including its resolution basis, records and checks; and the authors reviewed and revised each stage (§4, App. A).
  • Annotators and agreement (Q2.2). Two annotators independently checked each of the 111 designed cases against the three conditions: one author, and one annotator who is not an author and was not involved in designing the cases (the revision adds who they were, App. C). In this initial round they agreed on 84.7% of cases for condition 2, whether the user request designates the source that resolves the targeted decision.
  • Debatable sources (Q2.3). Debatable sources did arise during design: disagreements and failed conditions were inspected jointly, and cases were revised and rechecked, or discarded, leaving 90 of the 111.

Attack generation (Q3). The construction is described in §4.3 and Appendix A:

  • How written (Q3.1). By an LLM, Claude Opus 4.6, following written procedures: three instances per case and targeted belief are written independently. The revision names the model (App. A).
  • Filtering and tuning (Q3.2). Neither check uses the evaluated models' responses. An original instance is retained only if it passes two checks: an LLM judge (Claude Opus 4.6) confirms that, for an agent that fully believes it, the attacked action follows logically as the correct task response (sufficiency, not persuasion), and a classification blind to the declared target confirms the targeted belief. These checks dropped 148 of 1,491 instances. The revision adds both checks' prompts (Listings R1–R2). The detection-optimized rewrites of §5.5 are a separate condition, optimized against a carrier-only detection probe rather than the evaluated models (App. J).
  • Bias toward particular models (Q3.3). The retention checks use no model responses, and every evaluated model runs the same retained instances (Table 6). Drafting and both checks use a Claude model, and two evaluated agents are Claude models, so we cannot rule out that construction affects them differently (Limitations).

Role description and system prompt (W1). In the configurations we tested, a role description or a production system prompt does not lower ASR. We ran the benchmark with each configuration below on DeepSeek V3.1, Kimi K2.5, GLM-5 and Sonnet 4.5. Claude Code is a production agent harness whose own prompt instructs the model to flag suspected prompt injection; it changes the harness as well as the prompt:

ConfigurationASR (%)Change vs. none (%, 95% CI)
None (main runs)16.1—
Role description16.3+0.1 [−0.8, +1.0]
Claude Code: role description17.4+1.3 [−0.0, +2.7]
Claude Code: its system prompt + role description17.4+1.2 [−0.4, +2.9]
Generic trust-hierarchy rules (TH-generic)12.5−3.7 [−5.0, −2.4]
Environment-specific trust-hierarchy rules (TH-env)8.8−7.4 [−9.5, −5.5]

(All attacks, mean over these four models; changes are paired by case, so they need not equal the difference of the ASRs. Over all five models, Table 14 gives 17.0%, 12.8% and 8.9% for no defense, TH-generic and TH-env.)

Trust-hierarchy rules (W1). We agree that trust guidance is a relevant condition, and we now report it in the main text as a useful input to defense design (§5.6, Table R1): information about which source is authoritative lowers attack success across targets and adds to DRIFT-isolation (below). The rules that do so state the source priority our cases are built on (Appendix F), so their effect may not transfer beyond this benchmark. Such information also cannot always be assumed: the rules presume that a few fixed rules capture each environment's authority hierarchy, which may not hold where the hierarchy must be inferred or changes with the matter at hand (§5.6).

The gap under trust-hierarchy rules (Q1). The gap persists with DRIFT-isolation; under TH-env alone, the difference for state attacks is above zero and the one for observation attacks is not. Within case, on the same four models, including a new run that stacks TH-env on DRIFT-isolation:

Configurationb(U)b(O)b(s)b(O) − b(U)b(s) − b(U)
No defense14.411.718.2−0.5 [−5.8, +4.6]+4.7 [−2.5, +11.9]
TH-env6.95.910.9+0.9 [−2.4, +4.0]+5.2 [+0.6, +10.2]
DRIFT-isolation2.29.211.8+8.3 [+4.3, +12.9]+11.2 [+6.6, +16.1]
TH-env + DRIFT-isolation0.34.57.4+4.9 [+2.7, +7.4]+7.8 [+4.7, +11.5]

(Four models; ASR and within-case differences in %, all attacks, with 95% case-bootstrap intervals; the differences need not equal the difference of the displayed ASRs.) On their own, the rules lower all three targets by similar amounts; with DRIFT-isolation, both gaps persist with or without them. Table R6 gives these gaps per model, including the three additional models.

Scale and intervals (Q4). The differences between attack types that our conclusions rest on are statistically clear at the benchmark's scale, for each of eight models:

  • Intervals. The submission already reports 95% case-bootstrap intervals for each target's difference from objective attacks (Table 16); the revision computes these differences within case and per model.
  • More models. The revision adds GPT-5.4, gpt-oss-120b and Qwen3-VL-235B-A22B, run with no defense and every defense (Tables R2–R3).
  • Result. Under DRIFT-isolation, observation and state attacks exceed objective attacks with intervals above zero for every model (revised Table 16, Tables R8–R9):
ModelDRIFT-iso b(O) − b(U), within case (%, 95% CI)DRIFT-iso b(s) − b(U), within case (%, 95% CI)
Five evaluated models (mean)+8.8 [+4.7, +13.6]+11.4 [+6.9, +16.4]
DeepSeek V3.1+9.6 [+4.6, +15.1]+11.6 [+5.4, +18.2]
Kimi K2.5+8.0 [+2.7, +14.2]+10.9 [+5.4, +17.1]
Sonnet 4+10.7 [+5.0, +17.1]+12.2 [+6.2, +18.8]
Sonnet 4.5+9.1 [+3.7, +15.1]+7.9 [+2.5, +13.8]
GLM-5+6.6 [+1.8, +11.9]+14.3 [+8.1, +20.9]
GPT-5.4+5.0 [+1.4, +10.0]+11.0 [+5.4, +17.2]
gpt-oss-120b+5.7 [+0.5, +11.4]+10.9 [+6.4, +15.9]
Qwen3-VL-235B-A22B+16.2 [+9.6, +23.5]+18.4 [+10.7, +26.4]

Without a defense, the point estimates for decision-rule, action-space and state attacks are also above objective attacks (+6.0%, +8.4% and +4.6% within case, Table 16), but the difference for state attacks is not significant, so the revision reports these differences without ranking them.

The gap is also not specific to DRIFT: under ARGUS, an evidence-based authorization defense that relies on detecting anomalous content, state attacks exceed objective attacks within case (+11.2% [+5.7, +17.1]; observation attacks +3.1% [−0.4, +6.4]; Table 16), and its segmenter flags objective attacks most often (84.8%) and observation and state attacks least often (45.0% and 27.9%; Table R12).

Thank you again for the detailed review. As the discussion is closing, a brief summary of our responses above:

  • W2 (case validity): OM2's "service center inspection report" sits in the seller's own test-results record, so it is party evidence under the platform's policy. Policy 3.7 resolves the case using the buyer's measurements without requiring verification of the claimed report (Listings R3–R4).
  • Q2/Q3 (construction and validation): every designed case was checked by two annotators, one not an author (App. C). The retention checks use no evaluated model's responses (Listings R1–R2).
  • W1/Q1 (system prompts): a role description and the Claude Code harness do not detectably lower ASR, while trust-hierarchy rules do (§5.6). On whether the gap then disappears: with these rules added to DRIFT-isolation, observation and state attacks still exceed objective attacks (+4.9% and +7.8%, intervals above zero, Table R5).
  • Q4 (scale): the DRIFT-isolation gaps hold for each of eight models (the five evaluated and three added), with 95% intervals above zero (Tables R8–R9). Under ARGUS, another defense that relies on detection, state attacks also exceed objective attacks (Table 16).
Reviewer_K5qCOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies an indirect prompt injection attack method that tampers with the agent at authority level. The manipulation is done at a specific decision point and authors add a benign task that ensures that the agent reaches the considered decision point. The authors build a benchmark of validated cases in three simulated business settings (productivity, DevOps, marketplace disputes). Attacks for the same harmful outcome are written to target six parts of the agent’s decision making: its objective, rules, expected consequences, available actions, reading of observations, and beliefs about state. Authors also propose and analyze different defense mechanisms.

Critical strengths
  • The paper proposes a benchmark targeting authority base prompt injection attacks
  • The paper provide a detailed analysis of different defense mechanisms
  • The paper provides additional behavioral analysis under the considered attack
Critical weaknesses
  • Considering prompt injection attacks with authority level tampering has been previously studied and lacks novelty
  • Authors only provide analysis on 5 LLMs. It is unclear whether the results generalize across different model families
  • Authors do not consider the effect of different agentic scaffoldings
Questions for authors
  • Can actors provide results and analysis on a few more LLMs to study whether the claims would hold
  • Can authors provide analysis on how the effect would generalize additional agentic scaffoldings other than ReAct.
4 comments

We thank the reviewer for the review. In the revised PDF, all changes are marked in blue, and tables and listings numbered R1, R2, … are new or revised.

Novelty (W1). In the paper, authority is not the attack technique: it defines when planted content is an injection, namely when it attempts to influence a decision its source is not authorized to determine (§3.2). This differs from claimed authority as an attack technique, and from authority to instruct (e.g., Siu et al.; Wallace et al.), which leaves out content that steers a decision without instructing. Defining injection by authority over the decision brings such content inside the boundary, provided each case makes clear which source may decide. The paper's contributions (§1) build on this:

  • C1: An authority-based IPI boundary, operationalized. We extend IPI's authority boundary from commands to any influence on a decision, and specify per case the attacker-controlled source, the decision outside its authority, and an agent-visible resolution basis. Each case thus gives the agent a visible basis for resolving the conflict, also when the attack contains no instruction (condition 2, §3.2). In AgentPI and AgentLure (Wang et al., 2026a; Weng et al., 2026), an injection is attacker-inserted content by construction; neither states, per case, which source may settle the targeted decision.
  • C2: A controlled task-tampering benchmark. On this basis, we co-design each case's benign task, attacker role, controlled source and targeted decision, and use influence primitives to vary how the planted content justifies the same harmful outcome, so that differences between attacks under a defense can be attributed to the targeted belief; in AgentLure and in AgentPI's examples, each attack vector has its own goal.
  • C3: A mechanism-level defense analysis. Tracing where attacks are stopped shows gaps in existing defenses at both ends: detectors miss observation and state attacks, whose content reads as ordinary task data, more often than objective attacks, and action-level policies block few task-tampering calls, since the harmful call uses a tool the task needs.

Section 2 now states these requirements and what each enables.

More models (W2/Q1). The paper's two defense findings hold for three additional models from two further model families, which we ran on the full benchmark with no defense and every defense: the tool-policy defense Progent stays close to the undefended rate, and under DRIFT-isolation, observation and state attacks succeed more often than objective attacks on the same cases, with every within-case 95% interval above zero:

ModelASR (%), no defenseASR (%), ProgentDRIFT-iso b(O) − b(U), within case (%, 95% CI)DRIFT-iso b(s) − b(U), within case (%, 95% CI)
Five evaluated models (mean)17.016.3+8.8 [+4.7, +13.6]+11.4 [+6.9, +16.4]
GPT-5.410.28.7+5.0 [+1.4, +10.0]+11.0 [+5.4, +17.2]
gpt-oss-120b11.410.0+5.7 [+0.5, +11.4]+10.9 [+6.4, +15.9]
Qwen3-VL-235B-A22B26.923.0+16.2 [+9.6, +23.5]+18.4 [+10.7, +26.4]

(The first row is from Tables 3 and 16; the others are from Tables R2–R3, which give every defense and target, and Table R9.)

Beyond models, the gap is also not specific to one defense: under ARGUS, an evidence-based authorization defense that relies on detecting anomalous content, state attacks exceed objective attacks within case (+11.2%; Table 16), and its segmenter flags them least often (Table R12).

Agent scaffolding (W3/Q2). We ran all cases and attacks in Claude Code, a production agent harness, using its own system prompt and agent loop followed by each environment's role description, without a defense and with DRIFT-isolation. These runs use DeepSeek V3.1, Kimi K2.5, GLM-5 and Sonnet 4.5. Attack success does not change detectably (17.4% in Claude Code vs. 16.1% in our harness, +1.2% [−0.4, +2.9]), and the DRIFT-isolation gap replicates (Table R7):

ConfigurationASR (%), b(U)ASR (%), b(O)ASR (%), b(s)b(O) − b(U), within case (%, 95% CI)b(s) − b(U), within case (%, 95% CI)
Our harness, no defense14.411.718.2−0.5 [−5.8, +4.6]+4.7 [−2.5, +11.9]
Claude Code, no defense15.013.120.0+0.1 [−5.3, +5.3]+6.0 [−0.1, +12.1]
Our harness, DRIFT-isolation2.29.211.8+8.3 [+4.3, +12.9]+11.2 [+6.6, +16.1]
Claude Code, DRIFT-isolation1.710.211.4+10.0 [+5.8, +14.7]+11.2 [+6.7, +16.1]

(ASR pools instances; within-case differences average within cases and need not equal the difference of the displayed ASRs.)

Beyond the harness, the defenses in the paper already vary the agent's structure: DRIFT-isolation, DRIFT-full, Progent and ARGUS add components around the ReAct-style loop (content screening, an up-front plan, policy checks, call audits), and CaMeL replaces the loop with a plan-then-execute program (Table 3). Other agent architectures remain to be evaluated (App. L).

Thank you again for your review. As the discussion is closing, a brief summary of our responses above:

  • W1 (novelty): we use authority not as an attack technique but to define injection: planted content is an injection when its source lacks authority over the targeted decision, even when it gives no instruction. Each case makes this measurable by naming the source that settles the decision (§3.2).
  • W2/Q1 (models): on GPT-5.4, gpt-oss-120b and Qwen3-VL-235B-A22B, Progent stays near the undefended rate and the DRIFT-isolation gaps hold (Tables R2–R3, R9). Under ARGUS, another defense that relies on detection, state attacks also exceed objective attacks (Table 16).
  • W3/Q2 (scaffolding): Claude Code, a production harness, does not detectably change ASR, and the DRIFT-isolation gap persists under it (Tables R1, R7).
Reviewer_K5qCOpenReview ↗

Novelty of the work is marginal

Novelty of the work remains low and I will keep my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.