acceptodds
Under review as a conference paper at ICLR 2027

WITNESS: Benchmarking Dependency Analysis in LLM Agents Where Taint Tracking Ends

Abstract

The safe deployment of large language model (LLM) agents faces the threat of untrusted content delivered through prompts. Information flow tracking (IFT) is proposed for defense, but it requires estimating which inputs each output of the agent-driving LLM depends on, and no benchmark evaluates this estimation against human ground truth. We introduce Witness, a benchmark that treats an agent-driving LLM's output–input dependences as a standalone, testable oracle. Spanning eleven agents in diverse scenarios, Witness provides fine-grained human-annotated dependences, each typed as data or control, over both benign and prompt-injected trajectories. Evaluating twelve frontier and open-weight models, we find that they systematically miss true dependences: even the best model recovers an output's exact dependence set only half the time. This under-identification is the failure mode IFT cannot tolerate, since a missed dependence lets untrusted influence reach a sensitive action untracked.

Primary area:
datasets and benchmarks
Keywords:
LLM agents, Prompt injection, Information flow control, Dependency analysis, Benchmark

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_AhqzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces Witness, a benchmark for evaluating dependency analysis in LLM agents. The motivating problem is that information-flow tracking (IFT) defenses against prompt injection require knowing which parts of an agent’s input context each output decision depends on. For conventional programs, such dependencies can often be determined through program analysis, but no analogous reliable mechanism exists for LLM inference. Existing systems either conservatively assume that every LLM output depends on every input, which causes severe label propagation, or rely on approximate dependence estimation. Witness formalizes the task by segmenting an agent trajectory into context units and output units and annotating whether each output unit depends on each context unit. The benchmark distinguishes data dependence from control dependence, with finer-grained subtypes such as supply, compute, filter, etc.

Critical strengths
  1. The problem is important and clearly motivated. The paper identifies a concrete trust assumption in IFT-based defenses: if the dependency analyzer misses a true dependence on untrusted content, malicious influence may escape tracking. The connection between dependency recall and prompt-injection security is clear and practically relevant.
  2. The benchmark fills a plausible evaluation gap. The contribution is not another prompt-injection defense, but a benchmark for the dependency-estimation component that several defenses implicitly rely on. I found this framing useful because it isolates a component that can otherwise be hidden inside a larger security system.
  3. The annotation methodology is careful. The authors define dependence using a relatively concrete counterfactual criterion and distinguish direct value provenance from decision/control dependence. Two annotators independently label the data and achieve high agreement before adjudication.
Critical weaknesses
  1. The notion of “ground-truth dependence” is still partly semantic and subjective Although the paper provides a detailed rubric and reports strong inter-annotator agreement, the labels are not ground truth in the same sense as dependencies in a deterministic program. The counterfactual criterion sometimes requires annotators to judge whether an output would “plausibly” change, and the rubric contains explicit exceptions for multi-source decisions and control relations. Though this does not invalidate the benchmark, but I think the paper could more clearly distinguish human semantic dependence from causal dependence inside the model. In particular, an LLM may internally rely on a context unit even when human annotators judge that unit unnecessary under the task-level counterfactual definition.

  2. The benchmark is dominated by AgentDojo Although the paper includes 11 agents, AgentDojo contributes 824 of 1,560 output units and 2,880 of 4,715 labeled dependencies, which is a large fraction of the benchmark. This raises a question about how much the aggregate results reflect the diversity of real-world agent frameworks versus one benchmark ecosystem.

Questions for authors
  1. How should we interpret the Witness labels relative to causal dependence inside the LLM? Could there be context units that genuinely affect the model’s internal computation but fail the benchmark’s human counterfactual test, or vice versa?

  2. Why are reasoning-disabled results used as the primary evaluation? Since Appendix Table 13 shows meaningful recall improvements from reasoning for some models, would the main conclusion change if each model were evaluated under its strongest available inference setting?

2 comments
AuthorsOpenReview ↗

Response to Reviewer Ahqz (Part 1)

We thank the reviewer for the constructive feedback. We address the concerns and questions below.

Answers to Required Questions:

Q1. How should we interpret the Witness labels relative to causal dependence inside the LLM?

Witness labels are based on human judgments about task semantics. Many of these labeled dependencies also affect the model’s internal computation. We support this claim with two experiments: direct input interventions and context masking.

For direct input interventions, we sample labeled dependency pairs (c, o), where o is performed at step t. We modify c according to the dependency subtype, then provide the modified trajectory before step t to an LLM, and check whether o changes in its output. We use the following modification rules:

  • Supply: Replace c with a valid alternative of the same type.
  • Compute: Change operand c while preserving the operation and other operands.
  • Filter: Change an eligibility condition c so that o becomes eligible or ineligible.
  • Rank: Change a ranking value or tie-breaking rule while keeping o eligible for comparison.
  • Disambiguate: Change the identifying information c to point to another existing candidate.
  • Schema: Rename the function while preserving its functionality and parameter schema.
  • Request: Change the requested action or requirement while preserving unrelated task constraints.
  • Govern: Change the relevant state, prerequisite, or workflow rule that controls the target action.

We sample more than 15% of the labeled relations in each subtype. We use GPT-5.6-Luna for both modifications and validation. We request GPT-5.6-Luna to analyze each trajectory three times and consider a dependency is validated only if o changes in all three runs.

Overall, the input interventions confirm 645 of the 732 sampled relations (88.11%). The breakdown by subtype is shown in Table R3.1.

We conduct a similar experiment to validate independent annotations. We sample pairs (c, o) where c appears in the trajectory before o is performed and has no labeled relation to o. We use GPT-5.6-Luna to modify c, then ask the same LLM to analyze the modified trajectory three times. We confirm an independent annotation only if o remains unchanged in all three runs. Overall, 667 of the 732 sampled (91.12%) pairs met this criterion.

Table R3.1. Validating the consistency between human-annotated dependencies and actual model behavior.

SubtypeDirect Input Interventions: # of SamplesDirect Input Interventions: # of Validated SamplesIndependent Annotations: # of SamplesIndependent Annotations: # of Validated Samples
supply184171 (92.93%)178170 (95.51%)
compute3733 (89.19%)4141 (100.00%)
filter8273 (89.02%)5747 (82.46%)
rank5448 (88.89%)4741 (87.23%)
disambiguate97 (77.78%)1212 (100.00%)
schema120109 (90.83%)11093 (84.55%)
request110104 (94.55%)140128 (91.43%)
govern136100 (73.53%)147135 (91.84%)
Total732645 (88.11%)732667 (91.12%)

For context masking, we follow Sia et al. [1]. For each output unit o, we provide the preceding trajectory to the model and mask attention to all tokens corresponding to o’s annotated dependent units. We then check whether o changes in the model’s output.

We use Qwen3-8B because its open implementation and standard self-attention architecture make context masking straightforward to apply across all layers and attention heads.

We first identify steps that Qwen3-8B can reproduce without masking. For each of the 704 steps in Witness-Gold, we provide the preceding trajectory to the model three times. We retain the 166 steps that are reproduced in all three runs. These steps contain 342 output units: 120 with only data dependencies, 164 with only control dependencies, and 58 with both.

For each retained output unit, we jointly mask all its annotated context units and repeat the experiment three times. We count an output unit as affected only if it changes in all three runs.

Overall, masking changes 117/120 output units with only data dependencies, 154/164 with only control dependencies, and 50/58 with both. These results suggest that the annotated context units influence the model’s internal computation when generating these outputs.

[1] Where does In-context Learning Happen in Large Language Models?, NeurIPS’2024.

AuthorsOpenReview ↗

Response to Reviewer Ahqz (Part 2)

Q2. Why are reasoning-disabled results used as the primary evaluation.

We set temperature to zero in the main evaluation. Some reasoning APIs reject or ignore it, so we disable reasoning to ensure consistent settings across models.

During rebuttal, we evaluated all 12 models with reasoning enabled and their default temperature settings.

Table R3.2 compares reasoning-enabled results with Table 2 of the submission. Most models improve in precision and recall, but key findings hold: dependency recovery remains unreliable, complete sets are recovered for at most half the outputs, and control recall remains much lower than data recall.

Table R3.2. Evaluation results with reasoning enabled (EM: exact match, P: precision, R: recall).

ModelEMOverall POverall RData PData RControl PControl R
Claude-Opus-50.522(0.076)0.810(0.072)0.835(-0.006)0.899(0.087)0.928(-0.001)0.771(0.066)0.794(-0.008)
Claude-Sonnet-50.518(0.037)0.830(0.095)0.739(-0.010)0.907(0.050)0.861(-0.002)0.792(0.111)0.684(-0.014)
Claude-Haiku-4.50.506(0.138)0.867(0.087)0.698(0.144)0.904(0.134)0.863(0.113)0.845(0.058)0.625(0.158)
GPT-5.6-Sol0.445(-0.071)0.697(-0.122)0.867(0.124)0.877(0.034)0.936(0.059)0.632(-0.174)0.836(0.153)
GPT-5.6-Terra0.475(-0.004)0.717(-0.117)0.838(0.171)0.871(0.018)0.920(0.084)0.658(-0.165)0.801(0.209)
GPT-5.6-Luna0.425(0.019)0.733(0.016)0.817(0.165)0.888(0.147)0.915(0.097)0.672(-0.032)0.772(0.195)
DeepSeek-V4.1-Flash0.520(0.075)0.831(0.056)0.779(0.092)0.865(0.125)0.904(0.040)0.812(0.014)0.722(0.115)
DeepSeek-V4-Pro0.492(0.081)0.754(0.153)0.760(0.020)0.841(0.218)0.895(0.015)0.712(0.123)0.699(0.022)
DeepSeek-V4-Flash0.470(0.090)0.767(0.132)0.790(0.106)0.863(0.333)0.916(0.058)0.722(-0.005)0.733(0.127)
Qwen3.8-27B0.417(-0.048)0.694(-0.116)0.703(0.040)0.782(-0.045)0.847(0.036)0.650(-0.150)0.638(0.042)
Qwen3.5-27B0.491(0.053)0.787(0.104)0.691(0.022)0.801(0.094)0.844(0.024)0.778(0.109)0.622(0.021)
Qwen3.5-9B0.307(0.085)0.612(0.187)0.557(0.062)0.685(0.268)0.634(-0.057)0.578(0.146)0.522(0.115)

Cells show value (Δ). Δ = reasoning enabled − reasoning disabled.

Further Clarification of the Identified Weakness

W1. The paper should more clearly distinguish human semantic dependence from causal dependence inside the model.

See Q1.

W2. AgentDojo contributes a large fraction of the benchmark. The aggregate results may not reflect the diversity of real-world agent frameworks.

We acknowledge that AgentDojo contributes over half of the dependency relations. However, excluding AgentDojo leaves many patterns and observations unchanged.

As shown in Table 1 of the submission, subtype distributions remain similar after excluding AgentDojo, except for Filter and Rank.

Table R3.3 reports scores after excluding AgentDojo and differences from Table 2 of the submission. Most changes are small, and many findings hold: dependency identification remains unreliable, models have lower control recall than data recall, and more capable models perform better.

The remaining 10 agents cover common tasks: question answering, coding, and personal assistance. These results suggest our findings reflect broader patterns in real-world agent frameworks rather than being specific to AgentDojo.

Table R3.3. Dependence attribution excluding AgentDojo (EM: exact match, P: precision, R: recall).

ModelEMOverall POverall RData PData RControl PControl R
Claude-Opus-50.410(-0.034)0.726(-0.012)0.834(-0.002)0.783(-0.016)0.870(-0.058)0.692(-0.018)0.811(0.016)
Claude-Sonnet-50.492(0.010)0.786(0.039)0.735(-0.011)0.817(-0.051)0.796(-0.064)0.766(0.072)0.697(0.002)
Claude-Haiku-4.50.374(0.001)0.789(0.011)0.581(0.029)0.777(0.011)0.670(-0.073)0.798(0.012)0.525(0.059)
GPT-5.6-Sol0.519(0.005)0.824(0.001)0.759(0.016)0.872(0.029)0.808(-0.069)0.793(-0.018)0.729(0.046)
GPT-5.6-Terra0.479(0.004)0.821(-0.018)0.705(0.039)0.855(-0.005)0.747(-0.075)0.798(-0.027)0.679(0.083)
GPT-5.6-Luna0.394(-0.009)0.667(-0.046)0.691(0.045)0.701(-0.048)0.766(-0.043)0.644(-0.048)0.644(0.071)
Qwen3.8-27B0.481(0.018)0.851(0.046)0.659(-0.005)0.848(0.035)0.717(-0.098)0.852(0.052)0.623(0.026)
Qwen3.5-27B0.450(0.010)0.760(0.075)0.677(0.010)0.770(0.060)0.756(-0.065)0.753(0.082)0.628(0.029)
DeepSeek-V4-Pro0.446(0.034)0.651(0.050)0.770(0.031)0.646(0.022)0.854(-0.026)0.655(0.066)0.718(0.042)
DeepSeek-V4.1-Flash0.463(0.019)0.747(-0.021)0.724(0.037)0.746(0.020)0.797(-0.064)0.748(-0.050)0.679(0.070)
DeepSeek-V4-Flash0.387(0.003)0.651(-0.009)0.713(0.034)0.608(0.057)0.815(-0.029)0.690(-0.065)0.650(0.045)
Qwen3.5-9B0.210(-0.013)0.361(-0.063)0.465(-0.030)0.359(-0.059)0.603(-0.089)0.366(-0.065)0.380(-0.027)

Cells show value (Δ). Δ = excluding AgentDojo − full benchmark.

Reviewer_b8FYOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

WITNESS is a human-labeled benchmark for analyzing input–output dependence in LLM agents, motivated by information-flow tracking (IFT) defenses against prompt injection. The paper argues that IFT depends on identifying which inputs each agent-driving LLM output depends on. Unlike conventional program code, LLM reasoning does not expose these dependences directly, and the authors identify a lack of reliable methods and human-labeled benchmarks for evaluating dependence estimators.

The benchmark formulates dependence identification as a binary classification task. It contains trajectories from 11 agents, segmented into 27,217 context units and 1,560 output units. Two authors annotate 4,715 dependence relations, distinguishing data dependence from control dependence and assigning fine-grained subtypes. The authors also construct 269 Witness-Derived injection trajectories, for which the dependence of an injection-triggered tool call is known by construction.

The authors evaluate 12 frontier and open-weight models as third-party analyzers in Witness mode, and evaluate DeepSeek-V4-Flash reporting its own dependences during execution in vouching mode. The reported findings include that the best model, GPT-5.6-Sol, achieves exact match on only 51.3% of output units; control-dependence recall is consistently lower than data-dependence recall; and, for the single model evaluated in vouching mode, recall decreases by 0.238. The paper argues that missed dependences are a serious concern for IFT because they can leave untrusted influence untracked.

Critical strengths
  1. Addresses an important gap by providing a human-labeled benchmark for data and control dependence in LLM-agent trajectories.

  2. Includes a substantial annotation effort, fine-grained dependence subtypes, diverse agents, and injection trajectories with dependences known by construction.

  3. Establishes a security-relevant negative finding: models have particular difficulty recovering control dependences, an important channel through which prompt injections affect behavior.

  4. Provides useful experimental comparisons, including multiple LLMs, non-LLM baselines, a select-all reference, and analyses of model scale and generation.

  5. Makes annotation rubrics, prompts, and additional error analyses available in the appendix, supporting scrutiny and follow-up work.

Critical weaknesses
  1. The validity of the gold standard would benefit from further characterization. The benchmark includes dependences supported by observable single-unit counterfactual changes as well as dependences labeled on the basis of joint participation, but the paper does not quantify the proportion of each. The reported inter-annotator agreement is encouraging and supports consistent application of the rubric; additional validation could further clarify how well the labels capture the intended dependence semantics. This distinction matters when interpreting the benchmark’s model-performance results.

  2. The scope of the evidence for the broader conclusions could be strengthened. The vouching experiment evaluates DeepSeek-V4-Flash, so additional models would help establish how broadly the observed degradation applies. An end-to-end evaluation of an IFT defense could also clarify how dependence-identification errors relate to attack success. These extensions would provide stronger support for the paper’s broader security implications, while the benchmark itself remains a useful contribution.

Questions for authors
  1. How Was the Counterfactual Test Carried Out? The paper describes an annotator judgment about whether an output would plausibly change under a type- and role-preserving replacement. Was this test executed through trajectory replay and intervention on any subset? If not, could the authors perform an intervention-based validation on a stratified sample and report measured agreement with the human labels? Please also clarify how retrieval results, database state, and side-effectful tools are held fixed during any such replay.

  2. What Proportion of Labels Rely on Joint Participation? Could the authors report the number and proportion of dependence edges supported by an observable type-preserving single-unit intervention versus those labeled through the joint-participation rule for filter, rank, or compute? A breakdown by subtype, together with annotator agreement for these categories, would clarify the operational meaning of the gold standard.

  3. How Robust Is the Vouching Result Across Models? Could the authors repeat the vouching evaluation on two or three additional models, including at least one strong frontier model and one open-weight model from a different model family? If this is not feasible, could the abstract and conclusion explicitly limit the claim to the observed result for DeepSeek-V4-Flash rather than generalizing across LLMs?

3 comments
AuthorsOpenReview ↗

Response to Reviewer b8FY (Part 1)

We thank the reviewer for the constructive feedback. We address the concerns and questions below.

Answers to Required Questions:

Q1. How was the counterfactual test carried out? Could the authors perform an intervention-based validation?

The counterfactual test described in the rubric guides annotators’ dependency judgments: they mentally assess whether changing a context unit would affect an output unit.

As suggested by the reviewer, we perform an intervention-based validation and confirm the consistency between the human labels and observations through trajectory replay.

We sample labeled dependency pairs (c, o), where o is performed at step t. We modify c according to the dependency subtype, then provide the modified trajectory before step t to an LLM and check whether o changes in its output. We used the following modification rules:

  • Supply: Replace c with a valid alternative of the same type.
  • Compute: Change operand c while preserving the operation and other operands.
  • Filter: Change an eligibility condition c so that o becomes eligible or ineligible.
  • Rank: Change a ranking value or tie-breaking rule while keeping o eligible for comparison.
  • Disambiguate: Change the identifying information c to point to another existing candidate.
  • Schema: Rename the function while preserving its functionality and parameter schema.
  • Request: Change the requested action or requirement while preserving unrelated task constraints.
  • Govern: Change the relevant state, prerequisite, or workflow rule that controls the target action.

We sample more than 15% of the labeled relations in each subtype. We use GPT-5.6-Luna for both modifications and validation. We request GPT-5.6-Luna to analyze each trajectory three times and consider a dependency is validated only if o changes in all three runs.

Overall, the input interventions confirm 645 of the 732 sampled relations (88.11%). The breakdown by subtype is shown in Table R2.1.

We conduct a similar experiment to validate independent annotations. We sample pairs (c, o) where c appears in the trajectory before o is performed and has no labeled relation to o. We use GPT-5.6-Luna to modify c, then ask the same LLM to analyze the modified trajectory three times. We confirm an independent annotation only if o remains unchanged in all three runs. Overall, 667 of the 732 sampled (91.12%) pairs met this criterion.

Notably, we validate annotations by modifying recorded trajectories, keeping retrieval results fixed unless they are selected for intervention. We do not query databases or execute tools with side effects, so database states and live systems remain unchanged.

Table R2.1. Validating the consistency between human-annotated dependencies and actual model behavior.

SubtypeDirect Input Interventions: # of SamplesDirect Input Interventions: # of Validated SamplesIndependent Annotations: # of SamplesIndependent Annotations: # of Validated Samples
supply184171 (92.93%)178170 (95.51%)
compute3733 (89.19%)4141 (100.00%)
filter8273 (89.02%)5747 (82.46%)
rank5448 (88.89%)4741 (87.23%)
disambiguate97 (77.78%)1212 (100.00%)
schema120109 (90.83%)11093 (84.55%)
request110104 (94.55%)140128 (91.43%)
govern136100 (73.53%)147135 (91.84%)
Total732645 (88.11%)732667 (91.12%)

Q2. How many labels rely on single unit evidence or joint participation?

We revisit all dependencies in the Filter, Rank, and Compute subtypes and classified them as involving single-unit evidence or joint participation.

For Filter, joint participation involves multiple equally important eligibility criteria that jointly determine an output unit. For Rank, it involves multiple rules for comparing eligible candidates or multiple eligible candidates considered together. For Compute, it involves multiple data items used in the same calculation to produce the final result. We classified the remaining dependencies in these three subtypes as involving single-unit evidence.

Table R2.2 reports the number and proportion of dependencies in each category, along with the initial annotator agreement rates, for each subtype. Agreement patterns vary across subtypes. For example, agreement is higher for single-unit evidence than for joint participation in Filter and Compute, but lower in Rank. The overall agreement rate is consistent across single unit evidence and joint participation.

Table R2.2. Single and joint groups within filter, rank, and compute.

SubtypeEdgesSingle countSingle shareSingle agreementJoint countJoint shareJoint agreement
filter43117741.07%81.36%25458.93%60.24%
rank3449928.78%69.70%24571.22%77.55%
compute217146.45%100.00%20393.55%94.09%
Total99229029.23%78.28%70270.77%76.07%
AuthorsOpenReview ↗

Response to Reviewer b8FY (Part 2)

Q3. Repeating the vouching evaluation on two additional models.

Following the reviewer’s suggestion, we repeated the vouching evaluation with two additional models: GPT-5.6-Sol, a strong frontier model, and Qwen3.8-27B, an open-weight model from a different family. The results show broadly similar patterns across the three models, with some differences in task completion and precision.

As in Table 3 of the submission, we compute precision, recall, and exact match on aligned trajectories. We use the same aligned trajectories to calculate differences from the witness mode. Table R2.3 reports the detailed results. Its first row reproduces the last row of Table 3 for the original model, DeepSeek-V4-Flash, while the next two rows report results for the new models.

Both new models have task completion rates and trajectory alignment rates similar to that of DeepSeek-V4-Flash. Their recall and exact match are lower in the vouching mode than in the witness mode, the same as DeepSeek-V4-Flash. Their precision also decreases relative to the witness mode, whereas DeepSeek-V4-Flash’s precision increases.

We will incorporate these results in the next paper version and modify the abstract and the introduction accordingly.

Table R2.3. Vouching evaluation across models.

ModelComp.AlignedPrecision (delta)Recall (delta)Exact Match (delta)
DeepSeek-V4-Flash92.0%49.5%0.816 (+0.103)0.457 (-0.238)0.324 (-0.145)
GPT-5.6-Sol98.7%43.9%0.791 (-0.062)0.619 (-0.139)0.387 (-0.136)
Qwen3.8-27B90.3%45.4%0.856 (-0.009)0.493 (-0.166)0.294 (-0.191)

delta denotes Vouching minus Witness.

Further Clarification of the Identified Weakness

W1. The paper does not quantify the proportions of dependencies involving single-unit counterfactual changes and joint participation.

See Q2.

W2. The paper does not clearly explain how well the labels capture the intended dependency semantics.

See Q1.

W3. The vouching experiment only evaluates one model.

See Q3.

W4. The paper lacks an end-to-end evaluation of an IFT defense.

Following the reviewer’s suggestion, we perform an end-to-end evaluation of an IFT defense using AgentDojo trajectories in Witness-Gold. We observe that treating each LLM output as dependent on all its inputs reduces task completion rates. We also observe that a frontier model can miss dependencies of tool calls on untrusted sources, allowing calls that violate security policies.

Specifically, we implement the defense following policies in FIDES [1]. P-T requires tool selection to depend only on trusted sources. P-F permits data transmission only when the recipient is authorized and the outgoing message contains no untrusted links. We follow FIDES when assigning these policies to tools. FIDES evaluates AgentDojo prompt-injection trajectories and treats the injected prompts as untrusted. Since the 97 AgentDojo trajectories in Witness-Gold do not contain prompt injection, we instead designate outputs from selected tools as untrusted, based on our understanding of the tools, including two banking tools, four Slack tools, 20 travel tools, and seven workspace tools. For example, we treat the output of get_rating_reviews_for_car_rental() as untrusted because reviews can contain injected prompts, as illustrated in Figure 1 of our submission.

We compare three dependency propagation methods: (1) Conservative, which treats each LLM output as dependent on all its inputs; (2) Witness-Gold, which uses human-annotated dependencies in our dataset; and (3) GPT-5.6-Luna, which uses dependencies predicted by the model. We stop each trajectory at the first blocked tool call to reflect how the defense would operate in practice. The results are shown in Table R2.4.

Using Witness-Gold annotations increases task completion from 51.55% to 65.98% compared with Conservative, demonstrating the value of fine-grained dependency annotations.

We define the dependency miss rate as the proportion of evaluated tool calls for which at least one dependency on an untrusted-tool output is missed. Evaluated calls and dependencies differ across methods because they stop at different points. GPT-5.6-Luna misses dependencies on untrusted sources for 62 sensitive calls. Under security policies, 44 calls should be blocked, but the defense using GPT-5.6-Luna’s predictions blocks only 11. It also incorrectly blocks one call because of falsely predicted dependencies.

Table R2.4. Evaluation Results with an IFT Defense Framework

MetricConservativeHuman goldGPT-5.6-Luna
Task success rate50/97 (51.55%)64/97 (65.98%)85/97 (87.63%)
Dependency miss rate0/57 (0.00%)0/61 (0.00%)62/80 (77.50%)
Missing blocking rate0/31 (0.00%)0/33 (0.00%)33/44 (75.00%)
False blocking rate16/38 (42.11%)0/40 (0.00%)1/48 (2.08%)

[1] Securing AI Agents with Information-Flow Control. https://arxiv.org/abs/2505.23643

Thank you for the thorough response. The additional experiments directly address my concerns and strengthen the paper. The validation rates are encouraging overall; the lower validation rate for govern (73.53%) and the lower agreement for joint-participation filter labels (60.24%) are notable but do not undermine the benchmark. The additional vouching models support the broader degradation claim, and the end-to-end defense results reinforce the security motivation. I maintain my original score.

Reviewer_8ZWaOpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
2
Presentation
1
Soundness
1(−1)
Read review
Paper summary

This paper introduces WITNESS, a benchmark for evaluating the ability of LLM agents to identify dependencies between their outputs and preceding inputs, with particular emphasis on distinguishing data dependencies from control dependencies. The benchmark covers 11 agents and consists of 221 human-annotated trajectories containing 4,715 dependency relations, along with 269 constructed prompt-injection trajectories. The authors evaluate 12 LLMs under two settings: third-party dependency analysis and self-reported dependency analysis. These findings highlight potential limitations of current dependency analysis methods in supporting information flow tracking defenses.

Critical strengths
  1. The paper focuses on dependency identification, a critical component of information flow tracking, and formulates it as an independently evaluable task. This provides a valuable perspective for assessing the reliability of prompt-injection defenses.

  2. The benchmark covers diverse agent architectures and defines eight dependency subtypes with detailed annotation guidelines. The use of independent annotations followed by adjudication strengthens the credibility of the dataset, supported by a reported Cohen's κ of 0.899.

  3. The authors evaluate 12 LLMs alongside non-LLM baselines and investigate different dependency types, self-reported attribution, and injection-source identification. These analyses provide useful insights into the failure modes of existing dependency identification approaches.

Critical weaknesses
  1. The ground-truth dependencies are primarily established through human counterfactual judgments based on observable trajectories, rather than direct interventions on the underlying models. Although inter-annotator agreement is relatively high, Table 7 reports an exact dependency-set agreement of 0.740, suggesting that some dependency judgments remain uncertain.

  2. Witness-Derived constructs attack trajectories by replaying reference tool-call sequences and labels only the tool-name dependencies of injection-triggered calls, leaving their argument dependencies unlabeled. Consequently, the RQ3 evaluation may not fully capture the complex dependencies that arise when agents actually execute under prompt-injection attacks.

  3. The evaluation uses only DeepSeek-V4-Flash, and just 49.5% of steps remain aligned with the reference trajectories. Moreover, the two modes differ in their prompting and execution procedures. Therefore, the observed reduction in recall may not be entirely attributable to the model's self-reporting capability.

  4. AgentDojo contributes 2,880 of the 4,715 human-annotated dependency relations, while some dependency subtypes contain relatively few examples. Additionally, treating an entire webpage as a single input unit may limit the evaluation of fine-grained attribution to localized malicious content.

  5. Metrics such as Exact Match and Dependency Inflation Ratio measure attribution quality but do not directly quantify how missed dependencies translate into security violations or how additional dependencies affect legitimate operations. The lack of end-to-end defense evaluation may limit the practical interpretation of the security-related findings.

Questions for authors
  1. Annotation reliability: Could the authors further validate the consistency between human-annotated dependencies and actual model behavior through direct input interventions or additional independent annotations?

  2. Prompt-injection evaluation: Could the evaluation be extended to trajectories generated by agents actually executing under prompt-injection attacks, covering both tool selection and argument dependencies, to better assess the applicability of Witness-Derived?

  3. Vouching Mode: Given the trajectory alignment rate of only 49.5%, could the authors provide controlled experiments with additional models and further disentangle the effects of execution variability, prompt design, and self-reporting on attribution performance?

  4. Practical security implications: Could the authors report dependency miss rates specifically for sensitive actions influenced by untrusted tool outputs and evaluate the trade-off between security enforcement and false blocking of legitimate tasks within an actual IFT defense framework?

8 comments
AuthorsOpenReview ↗

Response to Reviewer 8ZWa (Part 1)

We thank the reviewer for the constructive feedback. We address the concerns and questions below.

Answers to Required Questions:

Q1. Validating human-annotated dependencies via direct input interventions and additional independent annotations.

Following the reviewer’s suggestion, we add experiments and have validated the consistency between human-annotated dependencies and actual model behavior.

For direct input interventions, we sample labeled dependency pairs (c, o), where o is performed at step t. We modify c according to the dependency subtype, then provide the modified trajectory before step t to an LLM, and check whether o changes in its output. We use the following modification rules:

  • Supply: Replace c with a valid alternative of the same type.
  • Compute: Change operand c while preserving the operation and other operands.
  • Filter: Change an eligibility condition c so that o becomes eligible or ineligible.
  • Rank: Change a ranking value or tie-breaking rule while keeping o eligible for comparison.
  • Disambiguate: Change the identifying information c to point to another existing candidate.
  • Schema: Rename the function while preserving its functionality and parameter schema.
  • Request: Change the requested action or requirement while preserving unrelated task constraints.
  • Govern: Change the relevant state, prerequisite, or workflow rule that controls the target action.

We sample more than 15% of the labeled relations in each subtype. We use GPT-5.6-Luna for both modifications and validation. We request GPT-5.6-Luna to analyze each trajectory three times and consider a dependency is validated only if o changes in all three runs.

Overall, the input interventions confirm 645 of the 732 sampled relations (88.11%). The breakdown by subtype is shown in Table R1.1.

We conduct a similar experiment to validate independent annotations. We sample pairs (c, o) where c appears in the trajectory before o is performed and has no labeled relation to o. We use GPT-5.6-Luna to modify c, then ask the LLM to analyze the modified trajectory three times. We confirm an independent annotation only if o remains unchanged in all three runs. 667 of 732 sampled (91.12%) pairs met this criterion.

Table R1.1. Validating the consistency between human-annotated dependencies and actual model behavior.

SubtypeDirect Input Interventions: # of SamplesDirect Input Interventions: # of Validated SamplesIndependent Annotations: # of SamplesIndependent Annotations: # of Validated Samples
supply184171 (92.93%)178170 (95.51%)
compute3733 (89.19%)4141 (100.00%)
filter8273 (89.02%)5747 (82.46%)
rank5448 (88.89%)4741 (87.23%)
disambiguate97 (77.78%)1212 (100.00%)
schema120109 (90.83%)11093 (84.55%)
request110104 (94.55%)140128 (91.43%)
govern136100 (73.53%)147135 (91.84%)
Total732645 (88.11%)732667 (91.12%)

Q2. Extending evaluation to cover both tool-selection and argument dependencies under prompt-inject attacks.

We sampled 12 tool calls triggered by injected prompts in AgentDojo attack trajectories, covering all 4 suites. Following the Witness-Gold methodology, we label how each call’s tool name and arguments depend on units in the preceding trajectory. We label the tool name as depending on the injected prompt. Across the 62 tool names and arguments, we identify 169 dependency relations.

We evaluate all 12 models in Witness mode on these annotations. As shown in Table R1.2, three models (Claude-Opus-5, Qwen3.5-27B, Qwen3.5-9B) miss dependencies on the injected prompts, and all models missed argument dependencies.

Rebuttal time limits prevented full AgentDojo annotation. This experiment shows Witness-Gold applies to injection-triggered calls and their arguments. We will annotate the rest for the next paper version.

Table R1.2. Dependency attribution on actual prompt-injection executions. (P: precision, R: recall)

ModelTool selection PTool selection RArguments PArguments ROverall POverall RInjection misses: CallsInjection misses: Edges
Claude-Opus-552.5481.5882.9870.9166.0475.271/126/52
Claude-Sonnet-548.5386.8493.1098.1869.0593.550/124/52
Claude-Haiku-4.572.7342.1168.6363.6469.8654.840/1217/52
GPT-5.6-Sol78.1265.7975.7196.3676.4783.870/1213/52
GPT-5.6-Terra92.3163.1696.1590.9194.8779.570/1215/52
GPT-5.6-Luna73.5365.7981.3687.2778.4978.490/1215/52
DeepSeek-V4.1-Flash96.0063.1694.2389.0994.8178.490/1214/52
DeepSeek-V4-Pro76.4768.4275.3889.0975.7680.650/1215/52
DeepSeek-V4-Flash68.4268.4272.8692.7371.3082.800/1214/52
Qwen3.8-27B84.6257.8980.7083.6481.9373.120/1219/52
Qwen3.5-27B65.6255.2677.9783.6473.6372.041/1219/52
Qwen3.5-9B26.0931.5842.5367.2736.8452.692/1218/52
AuthorsOpenReview ↗

Response to Reviewer 8ZWa (Part 2)

Q3. Further disentangling the effects of model choice, execution variability, prompt design, and self-reporting for the vouching mode.

We conducted additional experiments to examine these factors in vouching mode. Most observations are consistent with what we report in the submission.

To assess the effect of model choice, we added two models: a strong frontier model, GPT-5.6-Sol, and an open-weight model from a different family, Qwen3.8-27B. The results show broadly similar patterns across the three models, with some differences in task completion and precision. As in Table 3 of the submission, we compute precision, recall, and exact match on aligned trajectories. We use the same aligned trajectories to calculate differences from the witness mode. Table R1.3 reports the detailed results. Its first row reproduces the last row of Table 3 for the original model, DeepSeek-V4-Flash, while the next two rows report results for the new models. Both new models have task completion rates and trajectory alignment rates similar to that of DeepSeek-V4-Flash. Their recall and exact match are lower in the vouching mode than in the witness mode, the same as DeepSeek-V4-Flash. Their precision also decreases relative to the witness mode, whereas DeepSeek-V4-Flash’s precision increases.

Table R1.3. Vouching evaluation across models. (EM:exact match, P: precision, R: recall)

ModelComp.AlignedPREM
DeepSeek-V4-Flash92.0%49.5%0.816(0.103)0.457(-0.238)0.324(-0.145)
GPT-5.6-Sol98.7%43.9%0.791(-0.062)0.619(-0.139)0.387(-0.136)
Qwen3.8-27B90.3%45.4%0.856(-0.009)0.493(-0.166)0.294(-0.191)

Cells show value (Δ). Δ = Vouching − Witness.

To exclude execution variability, we compute all precision, recall, and exact-match scores in Table R1.3 on trajectories aligned with our annotated trajectories. The delta columns compare these scores with the corresponding witness-mode scores on the same aligned trajectories.

To control for prompt design, we modify the witness-mode trajectories by replacing the original task prompts with the vouching-mode prompts, which ask the LLM to both perform the task and report dependencies. We then ask a separate LLM to analyze the modified trajectories and report dependencies, following the witness-mode procedure. Table R1.4 shows the results for the three models. The delta column reports the differences from the original witness-mode results. These results show that using vouching-mode prompts has little effect. We cannot perform the reverse comparison because witness-mode prompts do not ask the task-performing LLM to report dependencies, as required in vouching mode.

Table R1.4. Prompt substitution for the Witness mode. (EM:exact match, P: precision, R: recall)

ModelPREM
DeepSeek-V4-Flash0.669(0.029)0.721(0.024)0.367(-0.017)
GPT-5.6-Sol0.857(0.038)0.709(-0.034)0.518(0.002)
Qwen3.8-27B0.809(-0.001)0.669(-0.004)0.461(0.005)

Cells show value (Δ). Δ = Vouching prompt − Witness prompt.

AuthorsOpenReview ↗

Response to Reviewer 8ZWa (Part 3)

Q4. Evaluating practical security implications.

Following the reviewer’s suggestion, we evaluate the practical security implications by applying an IFT defense framework to AgentDojo trajectories in Witness-Gold. We observe that treating each LLM output as dependent on all its inputs reduces task completion rates. We also observe that a strong frontier model can miss dependencies of tool calls on untrusted sources, allowing calls that violate security policies.

Specifically, we implement the defense following policies in FIDES [1]. P-T requires tool selection to depend only on trusted sources. P-F permits data transmission only when the recipient is authorized and the outgoing message contains no untrusted links. We follow FIDES when assigning these policies to tools. FIDES evaluates AgentDojo prompt-injection trajectories and treats the injected prompts as untrusted. Since the 97 AgentDojo trajectories in Witness-Gold do not contain prompt injection, we instead designate outputs from selected tools as untrusted, based on our understanding of the tools, including two banking tools, four Slack tools, 20 travel tools, and seven workspace tools. For example, we treat the output of get_rating_reviews_for_car_rental() as untrusted because reviews can contain injected prompts, as illustrated in Figure 1 of our submission.

We compare three dependency propagation methods: (1) Conservative, which treats each LLM output as dependent on all its inputs; (2) Witness-Gold, which uses human-annotated dependencies in our dataset; and (3) GPT-5.6-Luna, which uses dependencies predicted by the model. We stop at the first blocked tool call to reflect enforcement (Table R1.5).

Using Witness-Gold annotations increases task completion from 51.55% to 65.98% compared with Conservative, demonstrating the value of fine-grained dependency annotations.

We define the dependency miss rate as the proportion of evaluated tool calls for which at least one dependency on an untrusted-tool output is missed. The evaluated calls and dependencies differ across methods because they stop at different points. GPT-5.6-Luna misses dependencies on untrusted sources for 62 sensitive calls. Under the security policies, 44 calls should be blocked, but the defense using GPT-5.6-Luna’s predictions blocks only 11 of them. It also incorrectly blocks one call because of falsely predicted dependencies.

Table R1.5. Evaluation Results with an IFT Defense Framework

MetricConservativeHuman goldGPT-5.6-Luna
Task success rate50/97 (51.55%)64/97 (65.98%)85/97 (87.63%)
Dependency miss rate0/57 (0.00%)0/61 (0.00%)62/80 (77.50%)
Missing blocking rate0/31 (0.00%)0/33 (0.00%)33/44 (75.00%)
False blocking rate16/38 (42.11%)0/40 (0.00%)1/48 (2.08%)

[1] Securing AI Agents with Information-Flow Control. https://arxiv.org/abs/2505.23643

Comments on the paper

1 comment
AuthorsOpenReview ↗

Note to AC: Rebuttal Summary

Dear Area Chair,

Thank you for overseeing the review of our paper. We also thank the reviewers for reading it carefully and suggesting experiments that helped us test our central claim more directly.

WITNESS is a benchmark for evaluating how well models can identify which inputs influence each part of an agent-driving LLM’s output. It includes trajectories from 11 agents, with fine-grained human annotations of data and control dependencies. We divide each dependency type into several subtypes. We evaluate 12 models and find that even the best model recovers the exact set of dependencies for only about half of the output units. We also find that control dependencies are harder to identify than data dependencies.

All three reviewers recognize the value of the benchmark. Reviewer 8ZWa writes that it offers “a valuable perspective” on assessing the reliability of prompt-injection defenses. Reviewer b8FY says the paper “addresses an important gap” by providing a human-labeled benchmark for data and control dependence in LLM-agent trajectories. Reviewer Ahqz writes that “the benchmark fills a plausible evaluation gap.” Two reviewers also highlight what we learn from the evaluation. Reviewer 8ZWa notes that our analyses shed light on “the failure modes of existing dependency identification approaches.” Reviewer b8FY emphasizes the security relevance of our negative findings.

During the rebuttal, we added six experiments and analyses. 1) We used direct input interventions and context masking to test how closely our labels reflect the inputs LLMs rely on when analyzing trajectories and making decisions. We found strong agreement. 2) We used an IFT defense framework to confirm the security implications of the labeled dependencies. 3) We repeated our analysis after excluding AgentDojo trajectories. Our main observations and conclusions still held. 4) We annotated tool arguments in trajectories containing prompt injections. This showed that our method also applies to tool arguments under prompt injections and that we can include these annotations in the next version of the paper. 5) We repeated the vouching evaluation on two models and confirmed our original observations. 6) We also ran experiments with reasoning enabled and found that our observations held in this setting.

At the current stage of the discussion, reviewer b8FY wrote that our “additional experiments directly address my concerns and strengthen the paper” and kept their original score of 3. On November 17, Reviewer 8ZWa raised a concern about novelty for the first time. In our latest response, we noted that our paper already discusses all the works the reviewer cited. We explained how each relates to our work, clarified the differences, and addressed the reviewer’s other concerns.

We thank you and the reviewers again for your time and consideration.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.