- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces ATTRACK which is a runtime defense against indirect prompt injection in tool using agents. Basically, it scores each proposed tool call with and without the untrusted content and blocks the call if its log probability drops by more than a threshold when the untrusted content is removed. It also adds citation checks which ask a helper model to quote the instruction behind each call and block it if the quote comes from untrusted text. Authors evaluated this on 194 AgentDojo pairs per model, ATTRACK reports zero successful attacks with less utility loss. Also, they adds adaptive attacks and an analysis of how often the bound actually helps.
Critical strengths
- The bound does not depend on the citation model getting things right so the formal guarantee and the empirical detection are nicely separated. I think this is very useful because even if an attacker fools the citation model the bound can still holds.
Critical weaknesses
- I think the main issue is that the certificates don’t say much about whether a full attack goal is safe. Mostly, the strongest bounds are for one exact serialized call. But for instance an attacker can reach the same goal in many ways like for instance different argument values, different formatting of the same call etc. .The authors somewhat mention this but don’t measure how many goals are actually covered. Also, the main runs use greedy decoding but the bound assumes the call is sampled from the scored distribution.
- Moreover, the experiments don’t clearly show what each part adds or how the method compares to the closest work. The main grid uses only 2 injection tasks per suite and most importantly without a defense the attack works on qwen in just 4 of 194 pairs so qwen basically only tells us about utility. and the end-to-end adaptive test covers only 16 Banking cases and it uses a different citation model and prompt than the main runs and lastly the provenance taint baseline leaves out key parts of CAMEL (per-tool policies, user approval, label propagation).
Questions for authors
- Can the certificates be extended to cover a full attacker goal (including other calls and equivalent formats of the same call)? What fraction of the tested goals gets a useful bound?
- How do you check that the action tokens you score match the ones the model actually generated?
- Can you show end-to-end results for the influence test alone, the citation checks alone and the full method along with comparisons to CausalArmor and AttriGuard?
- Does the adaptive robustness still hold with the same citation model and prompt as the main runs and on suites other than Banking?
Thank you for the review. In short: the bound applies to any set of calls, and we now measure it for whole goals (W1, Q1); Table 1 has each layer on its own, CausalArmor, AttriGuard and the official CaMeL on three models, with an end-to-end adaptive attack on all four suites (W2, Q3, Q4); every check now compares the scored call with the generated tokens (Q2).
W1, Q1 (certificates for a whole goal). Proposition 1 bounds the probability of any set of calls, so it applies to a whole goal: every call, with any argument values and formatting, that makes AgentDojo's success check pass (Section 5, Appendix H). Whether the bound is informative depends on how probable the goal is under the trusted context, and we now measure this. Seven of the eight injection tasks succeed with any call that reaches the attacker's destination. For these goals the median log-probability under the trusted context is −49 on Qwen, −79 on Gemma and −44 on Mistral. At τ = 100 the goal-level bound is therefore above one for almost every pair. At this threshold such goals are protected by the citation checks and not by the bound, and Section 6.2 now says so.
The bound does cover these goals at lower thresholds. At τ = 40 it is below 1e-4 for 49% of Qwen pairs, 87% of Gemma pairs and 14% of Mistral pairs. We ran τ = 40 on tools that change state as a new row of Table 1. It costs 6 to 23 points of utility without attack. No static attack succeeds, and the adaptive attack succeeds in none of Mistral's 104 pairs. These figures assume the destination is the call's first argument. Appendix B gives estimates for other positions, which are lower on Qwen (12% at τ = 40).
Greedy decoding. We reran all pairs at temperature 1 with no truncation (Qwen and Gemma). The attack succeeds in 4 Qwen pairs and 39 Gemma pairs undefended, and in none with ATTRACK.
W2 (what each part adds; closest work; evaluation size). Please see Q3 for the ablations and the closest work, and Q4 for the adaptive attack.
Q2 (do the scored tokens match the generated ones?). Every check now logs the token ids and log-probabilities that the agent generated, and aligns the scored call against them (Appendix D). In the runs behind the ATTRACK rows of Table 1, the scored tokens are the generated ones in 99.6% of Qwen's calls and 99.7% of Gemma's. On Qwen the two log-probabilities agree within 0.73 nats. On Gemma the median difference is 0.03 nats and 99% of calls are within 3.3. Using the generated values would change at most 7 of Gemma's 3,562 verdicts, none of them on a call to an attacker's destination. On Mistral the two agree within 0.72 nats on every call.
The check found one gap in our first runs. A later call in a turn with several parallel calls was scored without the calls generated before it. We fixed this for the ATTRACK rows of Table 1. The adaptive attacks and the ρ-only rows on Qwen and Gemma used the earlier scoring, which lowers ρ on such calls by a median of 9 nats on Qwen, so those rows tend to overstate what gets through.
Q3 (end-to-end ablations; CausalArmor and AttriGuard). Table 1 now has all four, run end to end on the 194 pairs and under the adaptive attack.
causal-armorpackage; AttriGuard is ported from the authors' artifact.Q4 (adaptive robustness with the main citer and prompt, beyond banking). The adaptive attack now uses the main citer and prompt on all four suites, end to end, on three models. It succeeds against ATTRACK in none of 40 Qwen pairs, none of 39 Gemma pairs and 2 of 104 Mistral pairs, one of them by replaying a payload found against citation only. Both are bank transfers with ρ below τ (Section 6.2, Appendix A). Two further adaptive attacks on Mistral's 104 pairs succeed in none: one whose objective targets the citation checks directly, and one against τ = 40 on tools that change state.