- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies task-tampering attacks in indirect prompt injection, where attacker controlled content corrupts a decision within the legitimate task. It defines attacks through source authority and constructs a benchmark of 90 validated cases across three tool-use environments, comprising 1,343 single primitive attacks and 90 compound attacks. Within each case, attacks target different aspects of decision making while preserving the harmful outcome. Experiments across five LLMs and seven defenses show that detecting explicit objective manipulation or restricting tool use can leave substantial vulnerabilities to manipulated observations, states, and tool arguments.
Critical strengths
-
The benchmark makes the authority boundary explicit for each case, specifying the attacker-controlled source, the decision outside its authority, and the authorized basis for resolving that decision. Holding the task and harmful outcome fixed while varying the manipulation provides a useful framework for examining which attacks existing evaluations may miss.
-
The defense analysis identifies meaningful differences that aggregate attack success rates would obscure. The equal budget comparisons and rewriting experiments strengthen the evidence that broader attack coverage reveals additional vulnerabilities.
Critical weaknesses
- The fidelity of the runtime defense adaptations is insufficiently established. The results describe adaptations to a common agent loop, but the preservation of the original systems’ key mechanisms remains unclear. CaMeL’s benign task success rate of only 13.3% makes this consequential: the observed trade-off could reflect policy configuration or implementation choices. This limits conclusions about the underlying defenses.
Questions for authors
Which components of the original runtime defenses were retained, modified, or omitted? For CaMeL, were the separation of planning from untrusted-data processing and interpreter-based execution preserved? What accounts for its low benign task success rate?
We thank the reviewer for the careful review. In the revised PDF, all changes are marked in blue, and tables and listings numbered R1, R2, … are new or revised.
Components retained or modified. All key components are retained (App. M); for CaMeL, we add one restriction (below). The main adaptation is to environment-specific configuration: Progent, ARGUS and CaMeL define part of theirs for AgentDojo's tools (which operations are read-only; CaMeL's security policies), and we define it for the operations of our environments. Results characterize these evaluated adaptations. Below, we describe CaMeL's separation, interpreter and policy choices and analyze its benign failures, then report three implementation issues in other defenses that were identified in implementation checks and corrected (App. M.2).
CaMeL's separation and interpreter. Both are preserved. The privileged model writes a program from the user request alone and never sees tool data; the quarantined model parses untrusted content and cannot call tools; programs run in the interpreter from CaMeL's reference implementation, which tracks where every value came from.
Low benign task success.
Revised defense implementations. Three implementation issues were identified in implementation checks and corrected, and the affected defenses were rerun on all five models: Progent now always allows read-only operations, as in its AgentDojo evaluation; DRIFT-full's planner now receives the tools' structured parameter schemas, which it previously received only in part; and ARGUS's input and output length limits are raised from their defaults to fit the longer tool outputs in our environments. Appendix M.2 compares the submitted and revised results (Table R22), and all results for these defenses in the revision use the revised implementations; in brief (five-model means, %):
Attack success is essentially unchanged while task success rises, and the conclusions about these defenses hold: DRIFT-full still leaves observation and state attacks above objective attacks within case (+8.3% and +11.6%, above zero for each model), and Progent's ASR stays close to the undefended rate (16.3% vs 17.0%).