acceptodds
Withdrawn from ICLR 2027

DeepSift: Attributing Malicious Tool Invocations in LLM Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

An LLM agent augments a backbone LLM with tools and memory, and its context accumulates the user query, memory records, demonstrations, and tool observations while it acts. By injecting malicious texts into one of these inputs, for instance a poisoned memory record or an injected prompt, an attacker can drive the agent to invoke a tool of the attacker's choice, and existing defenses cannot prevent all such attacks. Once such an attack succeeds, identifying the texts in the context that are responsible for the malicious tool invocation is crucial for post-attack analysis and mitigation, a task we call attribution for LLM agents. Existing attribution methods score the entire context at once or target one type of input, so they are inaccurate for LLM agents: unrelated context dilutes the contribution of the malicious texts, and content generated after the attack masks it. To this end, we propose DeepSift, which organizes the context as a segmentation tree, recursively prunes it by Shapley scores to counter dilution, and excludes later-generated content to avoid masking, yielding a compact set of candidate texts. DeepSift then performs interaction-aware Shapley attribution over these candidates, so that each fragment of a split malicious text is still identified even if it is ineffective alone, and adaptively selects the malicious texts at the largest score gap. Theoretically, we bound how much the malicious texts distort the scores of benign texts and give conditions under which DeepSift returns exactly the malicious texts. Extensive experiments across diverse agent scenarios, attack families, and backbone LLMs show that DeepSift accurately attributes the malicious texts and outperforms existing methods, even under strong adaptive attacks.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agent Security, Attribution, Malicious Tool Invocation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Vp1yOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors introduce DeepSift, a framework that uses Shapley values on hierarchically partitioned context for post-hoc provenance localization.

Critical strengths

I definitely think the high-level idea is interesting and deserves some merit. The paper introduces a way to localize malicious inputs using Shapley values that I think will be more valuable as we move to longer and longer horizon tasks. The wide range of empirical results also help the case for it being useful in various settings, although there are parts I am not confident about.

Critical weaknesses

The main problem I have with the paper is that it seems to be framed in a more theoretic way than it actually is, and I am not sure if the parts that are formalized actually hold up as strongly as the paper suggests. I list my main concerns below and leave more in the questions below.

  1. The Shapley values and all associated with it are defined in terms of a surrogate LLM, and to my understanding there's no theoretical aspect that suggests this surrogate is connected to the actual model. This means even with empirical evidence, there's no actual theoretical guarantee that makes this a causal attribution. Thus the main contribution as I stated above become more of a way to localize malicious inputs using Shapley values on a surrogate model rather than anything causal or theoretical. In the same way, my understanding of Theorem 1 is that if malicious elements score much higher than benign ones, the gap will separate them. But the values it uses seem to depend on things you would not actually know during the attribution (I may be wrong on this). Finally, Shapley values are generally as meaningful as the function over the coalitions, but with the feature-removals in the counterfactual experiments the method may be sound mathematically but in practice it may be too OOD to the surrogate to avoid having a caveat written about it.

  2. The empirical validation also seems a little circular, as Equation 5 holds at 93% when it holds, but the theorem itself should already guarantee the exact selection whenever Equation 5 holds. In addition, this requires knowing which children are malicious and copmuting the exact Shapley values and the bound over all the subsets, meaning that this is not doing much theoretical validation.

Separate from the two points, I do generally think that the paper was not easy to follow, following the earlier comment about the paper maybe being more formalized than it needs to be. Note that this is more of a personal comment as a reviewer, not something that speaks for the contributions of the paper.

Questions for authors
  1. [Clarification] The paper defines S∗S^* as the spans written by the attacker but also motivates DeepSift as something that identifies the texts that are "responsible for"/"cause" this. But benign task context can be causally necessary for an attack to succeed, while injected spans may end up being redundant. Could the authors clarify what the actual intended task is?

  2. [Clarification] The theoretical analysis heavily uses Shapley values of vX(Q)=log⁡sθ(ϕ∣B(X)⊕Q)v_X(Q)=\log s_\theta(\phi\mid B(X)\oplus Q), where sθs_\theta is a surrogate model. Is the assumption that there is some sort of formal relationship between the surrogate and the actual agent? Without this, it seems like what Theorems 1 and 2 are saying is that they can only recover when the malicious spans already show different scores under the surrogate, rather than establishing some sort of causal guarantee of the agent's actions. These seem like two notably different claims to be making, and would be worth addressing.

  3. [Clarification] DeepSift computes Shapley values on counterfactual contexts that remove portions of the agent context. But why would we expect the likelihoods on these potentially OOD contexts to provide causal attribution for the original execution? It seems like this would only create semantically incoherent contexts in the subset and thus would be difficult to maintain reliability under these conditions.

  4. [Clarification] The post-hoc certificate in Theorem 3 requires parameters ϵ\epsilon and Γ\Gamma. Is it possible to have these values without access to malicious-span ground truth?

Reviewer_jFxhOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes DeepSift, a post-hoc attribution framework for identifying malicious text responsible for erroneous tool invocations in LLM agents. It combines hierarchical Shapley-based pruning, a base-context rule to reduce masking effects, and interaction-aware span attribution. The authors provide theoretical recovery guarantees and evaluate the method against 11 baselines across multiple attack types and backbone models.

Critical strengths
  1. The paper addresses an important and practical problem in LLM agent security: tracing malicious tool invocations back to their source texts.

  2. The paper considers diverse attack types, multiple backbone and surrogate models, and adaptive attacks, reporting consistently strong results.

  3. The theoretical analysis provides recovery conditions, while removal-based experiments offer additional evidence of attribution effectiveness.

Critical weaknesses
  1. Limited novelty. DeepSift's recursive Shapley pruning closely resembles TracLLM's hierarchical search. Related approaches, including Owen values, Partition SHAP, and h-Shap, are insufficiently discussed. The authors should clarify what is fundamentally new.
  2. Inconsistent experimental configurations. Appendix M specifies three malicious memory records or backdoored demonstrations, yet Figures 4–5 vary the total count from 1 to 5 while claiming the malicious content remains fixed. Table 37's single-malicious-span results also exactly match the default Table 1. These inconsistencies require clarification, particularly given the explanation of LOO-Node's failure in Appendix O.4.
  3. Insufficient ablations and baseline fairness. The ablations mainly compare Shapley with LOO, omitting critical controls: flat Shapley without pruning, removal of the base-context rule, and top-K or fixed-threshold selection instead of the gap rule. TracLLM should also be evaluated with the same hierarchical structure to isolate DeepSift's actual contributions.
  4. Limited generalizability. Evaluation is restricted to ASB, with near-perfect results (FPR = 0, DACC ≥ 0.98). Explicit tool names and arguments in attack texts may favor likelihood-based attribution. Stronger validation on AgentDojo, InjecAgent, longer trajectories, multi-turn interactions, and semantically implicit attacks is needed.
  5. Unverified surrogate attribution fidelity. DeepSift attributes actions using an 8B surrogate rather than the backbone that actually generated them. Although the authors test different backbones and perform removal-based validation, they never directly measure agreement between surrogate and backbone attributions. Thus, attribution fidelity to the actual agent remains uncertain.
  6. Poor readability. The paper is unnecessarily lengthy and difficult to follow, with excessive notation, repetitive discussions, and extensive technical details obscuring the central ideas. Frequent cross-references further hinder comprehension. A more concise, intuitive presentation would substantially improve clarity.
Questions for authors
  1. What are the key technical differences between DeepSift and existing hierarchical attribution methods, particularly TracLLM and Partition SHAP?

  2. How is the surrogate temperature implemented in likelihood scoring, and why does T = 0 outperform T = 0.2 despite the trend in Table 17?

  3. Can the authors clarify the inconsistent malicious record/span counts and provide component-level ablations to isolate the contributions of pruning, the base-context rule, and gap-based selection?

  4. How well do surrogate-based attribution scores align with those of the actual backbone model?

Reviewer_uTugOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DeepSift, an attribution method that identifies which sentences in the agent’s context contributed to the malicious tool invocation. DeepSift organizes the context into a tree, excludes the content generated by the agent after the attack succeeded from the evaluation, and scores each node with Shapley values, thereby identifying the sentences that induce the tool invocation. As a result, it outperforms all baselines.

Critical strengths
  1. The paper exploits the structure of an agent's context to define a segmentation tree. Also, the paper identifies that content generated after the attack can mask the injected text and proposes the base-context rule to remove that content from scoring.

  2. DeepSift consistently outperforms eleven baselines across three scenarios and twelve attacks, whereas each baseline fails on at least one attack category.

Critical weaknesses

No ablation isolates the base-context rule, the paper’s central contribution. Section 4.1 motivates the rule by masking, but no experiment runs DeepSift with the rule removed, i.e., with the full context as the base context. It leaves open whether the improvement comes from the base-context rule or from the tree search and gap rule alone.

Questions for authors
  1. What are the results of DeepSift with tree search and the gap rule but without the base-context rule, i.e., when B(X) is replaced with the full context?

  2. Why is the top-1 rule used at the root instead of the gap rule?

  3. Several baselines, such as ContextCite, TracLLM, and AttnTrace, can operate at the sentence level, but the experiments run them on context elements. Why were they not run at the sentence level, and how do their results change when they are?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.