acceptodds
Under review as a conference paper at ICLR 2027

PIPES: Securing Agent Perception with Provenance and Priors

Abstract

Tool-using agents consume external data from sources with different levels of trust, yet tool responses rarely identify who produced each component or what it should convey. We show that this gap enables state-corruption attacks, in which attacker-controlled content makes environmental claims beyond the informational authority of its response component and corrupts the agent’s perceived environment, making the resulting action appear justified to existing guardrails. We introduce PIPES (Provenance-Informed, Prior-Enforced Screening), which screens tool-response units using semantic priors and source provenance. PIPES uses static field contracts when schemas provide stable expectations, and conditions screening of open-ended content on the pre-tool-response trajectory and trusted provenance metadata. It marks units that violate their semantic prior or the provenance hierarchy; deployments may remove, warn, block, or escalate detected violations. We instantiate atomic removal and evaluate PIPES against adaptive PAIR-style attacks. We evaluate Gemma 4 31B IT, GPT-5.6 Luna, and Claude Sonnet 5 across three VitaBench and three AgentDyn splits. PIPES changes average attack success from 83.3% to 6.5% for Gemma, from 27.4% to 4.6% for Luna, and from 11.5% to 4.4% for Sonnet, without degrading benign utility on average.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
prompt injection defense

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.40 over 5 reviews, above 68% of the others.
Reviewer_TLedOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies an “agent perception gap” in tool-using LLM systems: tool responses often do not explicitly encode which source controls each response component or what semantic claims that component is authorized to make. The paper argues that this enables state-corruption attacks, where adversarially controlled content makes environmental claims beyond its informational authority. Unlike conventional prompt injection, these attacks need not contain explicit instructions; instead, they corrupt the agent's perceived state such that the attacker-desired action appears locally consistent with the resulting environment representation.

The authors propose PIPES (Provenance-Informed, Prior-Enforced Screening), which screens tool responses before they enter the agent context. PIPES checks each field or provenance unit for (1) consistency with a semantic prior defining what that component is expected to convey, and (2) consistency with a provenance hierarchy defining which sources are authorized to establish or override particular information. Structured tools use static field-level contracts, whereas open-ended responses such as emails, files, and webpages use a broad tool prior specialized by the pre-response trajectory and trusted provenance metadata.

The paper evaluates PIPES on three VitaBench and three AgentDyn splits with Gemma 4 31B IT, GPT-5.6 Luna, and Claude Sonnet 5. Attacks are optimized separately against each defense using a white-box PAIR-style adaptive attacker. PIPES achieves the lowest mean ASR among the evaluated defenses for all three target models while maintaining benign utility comparable to the undefended agent.

Critical strengths

The paper identifies a meaningful gap in current IPI defenses. The distinction between explicit control-transfer attacks and data-like state corruption is compelling: an attacker does not necessarily need to instruct the model to take an unauthorized action if it can instead corrupt the environmental facts that justify the action.

The decomposition into semantic priors and provenance authority is intuitive and useful. A response field should both remain within its expected informational role and respect the authority of the source that produced it. Importantly, PIPES does not reject content merely because it originates from an untrusted source.

The distinction between static and contextual priors is also well motivated. Structured response fields can often support stable field-level contracts, whereas emails, webpages, and files require task-dependent expectations inferred from the preceding trajectory. The use of source-attributable provenance units allows the method to preserve compliant content instead of discarding entire responses.

The adaptive attack evaluation is a strong aspect of the paper. Payloads are re-optimized against each defense, and the attacker receives substantial feedback, including PIPES assessments and static contracts. Therefore, the security gains cannot be explained merely by poor transfer of fixed attacks across defenses. The attack-instance construction also appears careful: VitaBench surfaces and goals are selected without access to PIPES's semantic priors or provenance hierarchy, and causal validation attempts to avoid labeling benign authoritative state changes as attacks.

The empirical results are strong across the evaluated settings. PIPES substantially reduces ASR for Gemma on both VitaBench and AgentDyn and consistently obtains the lowest mean ASR among the evaluated defenses. The ablations further indicate that priors and provenance are complementary, particularly in open-ended contextual settings.

Finally, the paper provides useful analyses of benign removals and token overhead, and clearly states several limitations.

Critical weaknesses

My primary concern is the amount of trusted tool-specific instrumentation required by PIPES. The system relies on semantic priors, provenance labels/orderings, deterministic source extractors, and sometimes authenticated provenance anchors. Existing tools may not expose these contracts, and in the experiments the missing information is constructed offline using documentation, response structure, and representative benign values. This is a meaningful systems assumption and potentially a substantial deployment cost. I would like to understand the manual effort required to construct and maintain these contracts and the method's sensitivity to imperfect or stale specifications.

Relatedly, the comparison with other defenses conflates two effects: PIPES introduces a new screening algorithm, but it also receives richer trusted semantic/provenance information than several baselines. PromptArmor, for example, is primarily an instruction detector and does not receive the same semantic authority structure. A control in which a generic LLM filter or action validator receives the same contracts and provenance metadata would help isolate the value of the PIPES-specific formulation.

The ablation also suggests that the contribution of provenance differs substantially across settings. On VitaBench, prior-only screening already obtains most of the improvement (14.8% mean ASR versus 11.2% for full PIPES), while provenance-only screening is much weaker. The joint benefit is clearer on AgentDyn. I think the paper should emphasize this asymmetry more explicitly.

A further concern is robustness to specification quality. Real tools often expose broad fields such as descriptions, comments, email bodies, and arbitrary metadata whose legitimate semantics are difficult to characterize narrowly. Overly narrow priors may remove legitimate content, whereas overly broad priors may admit state-corruption attacks. The current benchmarks provide relatively controlled tool environments, and AgentDyn contains only 20 tasks per split.

The adaptive evaluation is strong, but the ten-attempt budget may still limit conclusions. Reporting ASR as a function of the number of PAIR attempts would show whether attack success has saturated. Moreover, Gemma 4 31B IT is used as the fixed attacker even against stronger target models; the very low undefended AgentDyn ASR for Luna and Sonnet suggests a possible attacker bottleneck/floor effect.

Finally, the end-to-end baseline set does not include several conceptually close typed-interface or provenance/information-flow defenses discussed in the related-work section. If these methods cannot be instantiated in the benchmark environments, the paper should explain this more explicitly.

Questions for authors
  1. How much manual effort is required to construct a PIPES contract for a new tool, and how does the approach scale to large or rapidly changing tool ecosystems? Can the authors quantify authoring cost or test automatically generated/noisy contracts?

  2. Can the authors provide a baseline that receives the same semantic priors and provenance metadata as PIPES but uses a generic LLM-based screening/action-validation prompt? This would separate gains from richer trusted metadata from gains due to PIPES's specific formulation.

  3. Does adaptive ASR saturate within the ten PAIR attempts? Could the authors report attack-budget curves and evaluate a stronger or multi-model attacker on a subset of tasks?

  4. How sensitive is PIPES to incorrect, incomplete, overly narrow, or overly broad semantic priors and provenance orderings?

  5. Why are the closest typed/provenance-aware defenses discussed in Related Work not included in the end-to-end comparison? Are they incompatible with the evaluated environments?

2 comments

We thank the reviewer for recognizing the state-corruption threat, the value of semantic priors and provenance, and the strength of our adaptive evaluation. We address the reviewer’s concerns below.

Q1. How much effort is required to construct and maintain contracts for new tools or changing tool ecosystems? Can authoring cost or automatically generated contracts be evaluated?

Existing tools generally do not publish explicit field contracts. We therefore built an automated offline pipeline that constructs semantic priors and provenance annotations from tool documentation, response structure, and benign rollouts. Our reported evaluation uses profiles constructed by this pipeline, frozen before attack evaluation without using attack payloads or defense outcomes.

By demonstrating substantial improvements in robustness, we hope to encourage tool developers to publish semantic priors and provenance information alongside their response schemas. This would reduce the need to reconstruct missing contracts and support wider adoption of the defense.

Q2. How does PIPES compare with a generic LLM screener or action validator given the same semantic priors and provenance metadata?

To test whether richer trusted metadata explains the gains, we evaluated a Gemma contracts-in-agent baseline by inlining the same response schemas, semantic contracts, provenance information, and checking instructions alongside tool descriptions in the agent's system prompt. The agent applies these checks itself, without a separate screener or response removal. We retain the same attack cohorts and ten-attempt adaptive PAIR protocol, including target reasoning feedback.

Mean ASR across splits (%):

MethodVitaBenchAgentDyn
Baseline86.780.0
Contracts in agent32.85.0
PIPES11.21.7

While providing contracts directly to the agent improves robustness over the baseline, a separate screening stage provides further protection.

Q3. Does ASR saturate within ten PAIR attempts? What do attack-budget curves and a stronger or multi-model attacker show?

Attack-budget curves. We reconstructed cumulative ASR at each PAIR attempt budget from the saved adaptive attack traces, keeping the retained cohorts fixed and averaging ASR equally across the six splits. Gemma 4 results (%) are:

DefenseASR@1ASR@3ASR@5ASR@7ASR@10
No defense24.752.165.674.383.3
Action guardrail3.211.216.420.221.8
PromptArmor6.118.128.834.239.9
DRIFT2.57.010.812.414.8
PIPES0.33.44.55.66.5

Across defenses, 70.2–78.7% of the final mean ASR is reached within the first five PAIR attempts. Although ASR does not fully saturate, the average ASR gain per additional attempt is substantially smaller in the second half of the budget. We leave evaluation with stronger or multiple attacker models to future work.

Q4. How sensitive is PIPES to incorrect, incomplete, overly narrow, or overly broad priors and provenance orderings, including noisy contracts?

We expect PIPES’s robustness and utility to depend on the accuracy of its contracts and provenance information. Overly narrow contracts may remove legitimate content, while overly broad contracts may admit attacks. Although our evaluation uses automatically constructed contracts, we did not explicitly inject specification errors to measure sensitivity. We leave this systematic evaluation to future work.

Q5. Why were close typed-interface and provenance-aware defenses omitted? Can they be instantiated in the evaluated environments?

The cited typed-interface and provenance-aware defenses impose restrictions that make them difficult to instantiate in our current benchmarks. We explain why this is the case with specific prior works below.

Untrusted Content Masking. UCM is designed for browser-based web agents: it masks untrusted DOM elements before page rendering and is evaluated on custom websites and WebArena GitLab (Nikolić et al., 2026, Sections 3–5 and Appendix B.2). Our VitaBench and AgentDyn evaluations use tool APIs without this browser interface, so the published UCM implementation is not directly applicable.

Type-directed privilege separation. This method restricts communication from a quarantined agent to the privileged planner to curated types such as integers, floats, booleans, and enums; it explicitly disallows free-form text across this boundary (Jacob et al., 2026, Section III-B). PIPES instead screens free-form tool responses while preserving admissible text. Applying type-directed separation would require redesigning the agent’s handoffs around these restricted types.

Thank you for the response and the additional experiments. I have noted the clarification of contract construction, the contracts-in-agent comparison, and the attack-budget curves. The sensitivity to specification errors and evaluation with stronger attackers remain limitations.

Reviewer_zZxBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies state-corruption attacks against tool-using agents. Instead of issuing explicit instructions, attacker-controlled content makes claims beyond its source’s authority. These claims can distort the agent’s view of the environment, making an unwanted action appear consistent with the user’s request. The authors propose PIPES, a screening mechanism for tool responses. It checks what each part of a response is expected to convey and whether its source has the authority to make those claims. For structured responses, PIPES uses fixed field-level contracts. For open-ended content, it combines source metadata with the preceding interaction history. The experimental implementation removes content units that fail these checks before they enter the agent’s context. The evaluation covers six VitaBench and AgentDyn splits and three target models. Attacks are iteratively optimized against each defense, rather than reused unchanged across defenses. The reported results show lower average attack success than the evaluated baselines, with comparable aggregate performance on benign tasks. Ablations examine the two checks separately, and additional measurements report their token overhead.

Critical strengths
  1. A concrete defense against misleading claims. PIPES checks more than whether external content contains malicious instructions. It also checks whether a claim belongs in a given field and whether its source has the authority to make it. This directly addresses cases where misleading data makes an unwanted action appear justified. The paper applies the idea to both structured tool outputs and open-ended content.

  2. An evaluation that lets attackers adapt to the defense. The attacker can revise its payload using defense feedback, and attacks are optimized separately for each defense. The authors also measure benign utility on the full task suites. Ablations test the two checks separately under renewed attacks. This provides stronger evidence than testing all defenses against a fixed set of payloads.

Critical weaknesses
  1. The provenance hierarchy is under-specified, hand-constructed, and empirically the weaker component: PIPES depends on a profile-local provenance ordering and on "trusted reference context" admitted only when "the frozen profile also authorizes its provenance for that fact." The paper never explains how the hierarchy is derived operationally, how conflicts between equally ranked sources are resolved, or how sensitive results are to hierarchy errors.

  2. Baselines are weak and do not reflect recent related work: The comparison set is limited to no defense, an action guardrail, PromptArmor, and DRIFT. Several recent authority-aware and architectural-confinement defenses are cited in related work but not evaluated: type-directed privilege separation (Jacob et al., 2026), Untrusted Content Masking (Nikolic et al., 2026), CaMeL (Debenedetti et al., 2025), Fides (Costa et al., 2025), and Prompt Flow Integrity (Kim et al., 2025). These are the most directly comparable defenses (they also constrain what untrusted content may influence), so omitting them makes the empirical claim that screening at the claim level is the better design point unsupported.

  3. Benign-utility preservation is only weakly supported: The abstract claims "without degrading benign utility on average," but Appendix F reports removal rates of 0.17% (VitaBench, 16/9,686) and 4.6% (AgentDyn, 42/920), with all 42 contextual removals stripping legitimate but task-irrelevant content (24 in emails). Per-split benign utility with variance is not reported in the main text, so the key selling point, preserving admissible content, is not fully substantiated.

Questions for authors
  1. How is the provenance hierarchy obtained and validated outside the benchmark? Please describe the operational procedure for deriving q_T and X_{ref}, how authority is assigned when multiple sources are plausible, and how sensitive results are to hierarchy errors. Given that provenance-only checking is far weaker than prior-only on VitaBench, please clarify whether provenance does independent work or mainly reinforces the prior signal.

Why are recent authority-aware defenses not evaluated? Please compare against at least one of type-directed privilege separation (Jacob et al., 2026), Untrusted Content Masking (Nikolic et al., 2026), CaMeL, Fides, or Prompt Flow Integrity, and explain whether PIPES's claim-level screening is preferable to interface restriction or information-flow control.

What is the full benign-utility picture, including variance and contextual removals? Please report per-split benign utility for all defenses with confidence intervals and analyze the 42 AgentDyn contextual removals (especially the 24 email cases) to characterize lost legitimate content and whether the "higher utility on removal-affected tasks" result is robust.

2 comments

We thank the reviewer for recognizing PIPES’s protection against misleading claims and the strength of our adaptive evaluation. We address their concerns below.

Q1. How is the provenance hierarchy constructed and validated? How are competing sources ranked, and how sensitive is PIPES to hierarchy errors? Does provenance provide an independent benefit beyond the prior?

How provenance hierarchy is constructed. We assume tool developers provide or approve the provenance metadata in the frozen profile, including which sources are authorized to support particular facts. Many current tools may not expose the required metadata, so PIPES cannot reliably enforce provenance when it is unavailable. By demonstrating the benefits of exposing contracts and provenance, we hope to encourage wider adoption of these capabilities in tool integrations. In our evaluation, profiles are constructed offline from tool documentation, response structure, and representative benign values, then frozen before testing.

Benefits of Provenance over Prior: Priors determine which information is admissible; provenance resolves conflicts among otherwise admissible claims when priors are broad. Appendix D measures provenance’s added benefit under adaptive attacks: adding provenance to prior-only checking lowers mean ASR from 14.8% to 11.2% on VitaBench and from 11.7% to 1.7% on AgentDyn. The smaller gain on VitaBench is consistent with its static, narrower field contracts, where the prior often rules out out-of-scope claims directly; AgentDyn’s broader contextual priors leave more conflicts for provenance to resolve.

Q2. Why were recent authority-aware defenses omitted? Can PIPES be compared with one of them, and what does claim-level screening add over interface restriction or information-flow control?

PIPES allows tool responses to retain free-form text, while checking that their claims stay within the field’s semantic scope and the source’s authority. The cited works also restrict the attack surface, but enforce different boundaries:

  • Type-directed privilege separation uses a dual-LLM design with restricted typed handoffs, such as integers and enums. It does not return unrestricted free-form text to the privileged planner.
  • Untrusted Content Masking masks untrusted webpage regions based on DOM structure. It does not check whether claims in admitted content fall within a field’s semantic scope.
  • CaMeL derives trusted program control flow before processing untrusted data. This limits adaptation when required steps emerge only through environmental feedback.
  • Fides tracks confidentiality and integrity labels, selectively hiding data and revealing constrained outputs. Its enforcement centers on permitted information flows, rather than whether individual free-form claims fit a field’s meaning and source authority.
  • Prompt Flow Integrity isolates untrusted data behind identifiers and requests user approval for unsafe privileged flows. Its guardrails check flow indicators rather than content compliance with field-level semantic contracts.

At the reviewer’s request, we implemented CaMeL and compared it with PIPES using Luna on AgentDyn. Both obtained 0% observed ASR, but PIPES maintained 93.3% mean benign-task utility versus 0% for our CaMeL implementation. CaMeL’s poor utility is consistent with the AgentDyn paper’s reported over-defense on dynamic tasks.

Luna results on AgentDyn (%; higher benign utility and lower ASR are better):

SplitCaMeL benign utilityCaMeL ASRPIPES benign utilityPIPES ASR
Shopping0.00.090.00.0
GitHub0.00.095.00.0
DailyLife0.00.095.00.0
Mean0.00.093.30.0

Q3. What do per-split benign utility and variance show? What legitimate content was removed in the 42 AgentDyn contextual cases, especially the 24 email cases, and is the utility result on affected tasks robust?

Per-split benign utility. Tables 3 and 4 report per-split benign utility for all defenses with Gemma 4 and Luna; Table 7 reports the corresponding Sonnet results. We will repeat the benign evaluations across several independent runs and report mean utility, variance, and confidence intervals across runs.

AgentDyn content removals. We manually examined several contextual removals in AgentDyn and found that the removed content was unrelated to the current task and not needed for its completion. For the 24 that occurred in email results specifically: when the agent reads an inbox, PIPES removes emails unrelated to the current task. This is consistent with our experimental finding that removals do not hurt task utility.

Thank you for your response and clarifications. However, some of my concerns still persist. I will maintain my original score.

Reviewer_xUm5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies state-corruption attacks against tool-using agents, where attacker-controlled content makes environmental claims beyond the authority of its field or source. Such attacks can redirect an agent without explicit malicious instructions because its actions appear reasonable given the corrupted information.

The authors propose PIPES, which screens tool responses before they enter the agent's context. It uses an LLM assessor with static field contracts or trajectory-conditioned expectations and provenance information to identify noncompliant content. The experiments use atomic removal as the enforcement policy. Evaluation against adaptive PAIR attacks on VitaBench and AgentDyn, using Gemma 4 31B IT, GPT-5.6 Luna, and Claude Sonnet 5, shows substantial reductions in mean attack success rates with comparable mean benign utility.

Critical strengths
  1. The paper addresses an important security problem and provides useful evidence that defenses need to look beyond explicit malicious instructions. In the authors' LLM-based analysis, 255 of 347 successful VitaBench payloads observed intact under the three baseline defenses are classified as state-only, without explicit instructions. The adaptive evaluation is also a strength: attacks are optimized separately against each defense, with access to target reasoning and defense feedback.

  2. The defense is simple and effective on the evaluated benchmarks. Checking tool-response content against its expected role and source authority is an intuitive approach. Across the six splits, PIPES reduces mean adaptive ASR from 83.3% to 6.5% for Gemma, from 27.4% to 4.6% for Luna, and from 11.5% to 4.4% for Sonnet, while maintaining comparable mean benign utility. Appendix E also reports lower token usage than the action guardrail and DRIFT configurations.

Critical weaknesses
  1. The most important missing comparison is an information-matched baseline. PIPES receives explicit field contracts and provenance information, but the reported experiments do not give the same profiles and checking instructions directly to the target agent or an existing action guardrail. Without this comparison, it is unclear how much of the gain comes from making these expectations explicit versus applying them in a separate screening stage. Appendix D tests the contribution of the two checks, but does not resolve this distinction.

Relatedly, the evaluation uses removal rather than warnings or annotations. I recognize that the framework is explicitly policy-independent and does not claim removal is necessary. Still, comparing these policies would clarify the security–utility trade-off, particularly when a violating unit also contains information needed for the legitimate task. Appendix F provides useful benign-removal measurements, but does not resolve that comparison.

  1. I find the algorithmic novelty modest, and the practical generality of the approach is not yet clear. The runtime method centers on an LLM-based compliance check, whose usefulness depends on the field contracts, provenance rules, and response-unit extraction. Sections 3 and 4.4 describe constructing profiles offline from documentation and benign examples, but do not clearly establish which steps are manual or automated, how much effort is required for new tools, or how sensitive the results are to imperfect specifications.

The authors acknowledge these dependencies in Section 7, but the evaluation does not quantify the effects of vague or incorrect contracts, unavailable provenance, or extraction failures. This leaves uncertainty about whether the reported security–utility tradeoff transfers to less controlled deployments.

Questions for authors
  1. Additional comparison: Can the authors evaluate a baseline that gives the same contracts, provenance information, and checking instructions directly to the target agent or existing action guardrail? This would help isolate the added benefit of the separate screening stage.

  2. Enforcement comparison: How does atomic removal compare with retaining the content and attaching PIPES's violation warnings? Beyond the benign-removal analysis in Appendix F, what happens when a violating unit also contains information needed to complete the legitimate task?

  3. Implementation clarification and robustness: Which parts of profile and provenance-extractor construction are manual versus automated, and what happens when reliable extraction or attribution is unavailable? Is there evidence on sensitivity to vague or incorrect contracts?

1 comment

We thank the reviewer for their thoughtful comments and for recognizing the importance of state corruption attacks and the effectiveness of our defense in mitigating this threat.

Our responses to the reviewer’s questions along with results from additional experiments are below. We are committed to addressing any additional concerns the reviewer may have.

Q1. Additional comparison: Can the authors evaluate a baseline that gives the same contracts, provenance information, and checking instructions directly to the target agent or existing action guardrail? This would help isolate the added benefit of the separate screening stage.

We evaluated the attack on Gemma-4-31B-IT by inlining the same response schemas, semantic contracts, provenance information, and checking instructions alongside tool descriptions in the agent's system prompt. The agent applies these checks itself, without a separate screener or response removal. We retain the same attack cohorts and ten-attempt adaptive PAIR protocol, including target reasoning feedback. The results show that providing contracts directly to the agent improves robustness over the baseline. However a separate screening stage provides further protection.

Mean ASR across splits (%):

MethodVitaBenchAgentDyn
Baseline86.780.0
Contracts in agent32.85.0
PIPES11.21.7

Q2a. Enforcement comparison: How does atomic removal compare with retaining the content and attaching PIPES's violation warnings?

We evaluated the warning-based alternative on Gemma under the same adaptive PAIR protocol. When PIPES flags a violation, we retain the complete tool response and append a warning identifying the affected fields and violation types, instructing the agent to treat the flagged claims as untrusted.

Mean ASR across splits (%):

MethodVitaBenchAgentDyn
Baseline86.780.0
PIPES warnings66.423.3
PIPES atomic removal11.21.7

Warnings improve robustness over the baseline, but atomic removal provides substantially stronger protection. In 102 of 107 successful VitaBench attacks, the compromised response carried a warning. Warnings leave enforcement to the agent, which can still act on the flagged claims; removal excludes the flagged content before it enters the agent's context.

Q2b. Beyond the benign-removal analysis in Appendix F, what happens when a violating unit also contains information needed to complete the legitimate task?

If an attacker directly controls an authoritative task input, such as the rating used to select a product, it can change the correct decision without violating that field's contract. We instead study claims beyond the controlled field's authority, such as a rating override embedded in merchant tags. VitaBench's causal audit excludes surface–goal pairs where ordinary, non-adversarial field values could legitimately justify the target action.

Q3. Implementation clarification and robustness: Which parts of profile and provenance-extractor construction are manual versus automated, and what happens when reliable extraction or attribution is unavailable? Is there evidence on sensitivity to vague or incorrect contracts?

Construction. We use an automated offline pipeline to construct semantic priors and provenance annotations from tool documentation, response structure, and benign rollouts. Profiles are frozen before evaluation without using attack payloads or defense outcomes. Tool-specific extraction and source-mapping rules are specified in integration code; at runtime, provenance-unit extraction and source attribution are deterministic rather than inferred by the LLM assessor.

Unavailable metadata. Some tools may not expose sufficient structure or trusted metadata to establish unit boundaries and source authority. In these cases, PIPES cannot be applied reliably without additional information from the tool developer. By demonstrating the security benefits, we aim to encourage developers to publish semantic contracts and provenance metadata alongside their response schemas.

Specification sensitivity. Overly narrow contracts may remove legitimate content, while overly broad contracts may admit attacks; incorrect provenance attribution can also weaken authority checks. Although our evaluation uses automatically constructed contracts, we did not explicitly inject specification errors to measure sensitivity. We leave this systematic evaluation to future work.

Reviewer_j6jhOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies state-corruption attacks, where malicious tool responses manipulate an agent's perception of the environment without explicit instructions. The authors propose PIPES, an LLM-based defense that screens tool responses using semantic priors and source provenance, with static and contextual variants. Experiments on VitaBench and AgentDyn show substantial reductions in attack success rates while preserving benign task utility.

Critical strengths
  1. The paper highlights state-corruption attacks that manipulate agents' environmental perceptions without explicit malicious instructions, exposing a limitation of existing defenses.
  2. The proposed method demonstrates encouraging results across multiple models and benchmarks, suggesting its potential for improving agent security.
Critical weaknesses
  1. Limited technical novelty. The proposed framework largely combines existing techniques, including semantic validation, provenance checking, and LLM-based filtering. Static PIPES is essentially schema-guided validation, where many violations can be detected using simple rule-based approaches. The paper fails to establish why an additional LLM-based framework is necessary for such structured settings. Contextual PIPES is more interesting, but its implementation still primarily relies on prompting an LLM with additional task context and provenance metadata, without substantial methodological innovation.

  2. Insufficient evaluation of Contextual PIPES. Contextual PIPES is arguably the more important contribution, since Static PIPES largely handles structured data with predefined constraints. However, the evaluation is dominated by static scenarios. All 165 VitaBench attack instances use Static PIPES, while only 20 AgentDyn GitHub instances are exclusively contextual. The remaining AgentDyn splits mix both settings, without separately reporting their performance. This evaluation is insufficient to substantiate the effectiveness and generalizability of the paper's most important component.

  3. Lack of evaluation on long-horizon agent tasks, especially for Contextual PIPES. The paper does not establish whether PIPES remains efficient and effective in long-running agent workflows. Although the benchmarks allow up to 40 agent steps, the authors do not report the actual average trajectory lengths or evaluate substantially longer tasks. This raises two major concerns:

    (1) Computational overhead: PIPES requires additional LLM assessments during agent execution. For long-horizon tasks involving numerous tool interactions, the accumulated token consumption and latency could become substantial, particularly for Contextual PIPES. The aggregate token statistics in Appendix E do not adequately characterize this overhead, and no end-to-end latency evaluation is provided.

    (2) Contextual assessment accuracy: Contextual PIPES relies on preceding agent trajectories to infer semantic priors and provenance, but only considers up to 12 preceding messages. In long-horizon tasks, critical information such as the original user intent, earlier tool observations, and source authority may fall outside this window. This could lead to both missed attacks and false positives. The paper provides no experiments evaluating how Contextual PIPES's detection accuracy changes with trajectory length or context truncation.

    Without evaluating these two aspects, the paper does not provide sufficient evidence that PIPES can be reliably deployed in practical settings.

  4. Limited evidence of effectiveness on stronger models. The AgentDyn benchmark contains only 20 attack instances per split, and stronger models already exhibit very low attack success rates without defense. Consequently, the improvements introduced by PIPES are marginal, particularly in the fully contextual GitHub setting. The small sample size and low baseline ASRs make it difficult to assess the actual effectiveness of Contextual PIPES on stronger agents. More challenging and larger-scale evaluations are needed.

Questions for authors
  1. How does PIPES compare with simpler baselines, such as rule-based validation for Static PIPES or a context-aware LLM detector without explicit prior/provenance contracts?

  2. Can the authors provide a more comprehensive evaluation of Contextual PIPES, particularly on stronger models and more challenging attack scenarios where the undefended ASR is not already near zero?

  3. What are the average trajectory lengths of the evaluated tasks? How do the computational overhead and detection accuracy of Contextual PIPES change in long-horizon tasks, particularly when relevant information falls outside the 12-message context window?

  4. How well does PIPES generalize to unseen tools and environments, especially when predefined tool profiles or reliable provenance metadata are unavailable?

3 comments
AuthorsOpenReview ↗

Rebuttal-1

We thank the reviewer for their thoughtful review. At the outset, we would like to clarify a potential misunderstanding - the core contribution of Static PIPES is to do semantic validation on tool response fields that contain free-form text. In our experiments we specifically exclude fields that can be checked with rules as the attack surface. We do not claim rule-based checks as part of our contribution.

We answer the reviewer's questions below and are committed to addressing any other concerns the reviewer may have.

Q1. How does PIPES compare with simpler baselines, such as rule-based validation for Static PIPES or a context-aware LLM detector without explicit prior/provenance contracts?

Rule-Based Validation: Simple rule-based validation is insufficient for semantic violations in free-form text fields such as names, descriptions, and tags. A value can satisfy the schema while making claims outside the field's semantic scope or source authority; PIPES uses an LLM to assess these claims. “Static” refers to the semantic contracts being fixed before execution (not rule-based checks). Fields that can be validated with deterministic checks (e.g., numbers, booleans, and enums) are excluded from attack-surface selection in our experiments.

Context aware LLM detector without prior/provenance contracts: Two of the baselines used in our evaluation already incorporate context: DRIFT uses a task-aware detector to screen tool responses, while Action Guardrail uses a context-aware action monitor. PIPES achieves lower mean ASR than both (Gemma VitaBench: 11.2% for PIPES, 24.5% for DRIFT, and 40.3% for Action Guardrail), supporting the value of explicitly defining and checking semantic contracts.

Q2. Can the authors provide a more comprehensive evaluation of Contextual PIPES? More challenging attack scenarios?

On AgentDyn, we group attack instances by whether the injected response uses static or contextual assessment. The ASRs below pool instances within each group, rather than averaging split-level rates. For Gemma's contextual instances, PIPES reduces ASR from 81.6% to 2.0%.

Target agentSettingInstancesUndefended ASRPIPES ASR
GemmaStatic1172.7%0.0%
GemmaContextual4981.6%2.0%
LunaStatic119.1%0.0%
LunaContextual496.1%0.0%
SonnetStatic80.0%0.0%
SonnetContextual521.9%0.0%

We evaluate on established benchmarks using a strong adaptive PAIR attacker that iteratively optimizes payloads using the target and defense models’ reasoning and responses as feedback. We welcome suggestions for additional benchmarks with more challenging contextual attack scenarios.

Q3. What are the average trajectory lengths of the evaluated tasks? How do the computational overhead and detection accuracy of Contextual PIPES change in long-horizon tasks, particularly when relevant information falls outside the 12-message context window?

Trajectory Lengths: The table reports median trajectory lengths in agent steps on the undefended benign runs (100 tasks per VitaBench split and 20 per AgentDyn split, per model). One step is one generated agent turn; synthetic greetings are excluded. Agent steps differ from context messages, since a step can add both assistant and tool messages.

Benchmark splitGemmaLunaSonnet
VitaBench Delivery13.80 (11)11.40 (11)12.49 (11)
VitaBench In-store11.01 (10.5)11.71 (11)10.78 (10.5)
VitaBench OTA11.07 (11)12.75 (12)11.52 (11)
AgentDyn Shopping9.90 (9)8.65 (8)7.20 (6.5)
AgentDyn GitHub9.95 (8)7.20 (6.5)7.95 (6.5)
AgentDyn Daily-life10.00 (10.5)8.55 (8)7.60 (8)

Computational overhead: Contextual PIPES performs one LLM assessment per screened tool response, using the response units and a bounded trajectory window. Its input-token overhead grows with the amount of context and response content supplied. Increasing the window therefore increases per-assessment overhead, while total overhead accumulates with the number of screened responses. This cost structure is similar to that of context-aware action review, as used in Codex Auto-review, which evaluates proposed actions using a compact conversation transcript.

On the evaluated workloads, the measurements in Appendix E show that PIPES has comparable total token usage to PromptArmor and substantially lower usage than Action Guardrail and DRIFT.

Context truncation: Our reported contextual results use a 12-message window. Evaluating how detection accuracy changes when essential information falls outside that window, and how best to retain relevant context, remains a promising direction for future work.

AuthorsOpenReview ↗

Rebuttal-2

Q4. How well does PIPES generalize to unseen tools and environments, especially when predefined tool profiles or reliable provenance metadata are unavailable?

PIPES requires semantic contracts and available provenance metadata for each tool. When contracts are not published, we construct them offline from tool documentation, response structure, and representative benign values, then freeze them before evaluation. Some tools may not expose sufficient information to specify semantic contracts or establish source provenance. In these cases, PIPES cannot be applied reliably without additional information from the tool developer. By demonstrating the security benefits of these contracts, we aim to encourage tool developers to publish semantic priors and provenance metadata alongside their response schemas, supporting broader adoption of provenance-aware screening.

Thanks the authors's reply and additional experiments, however, considering that (1) the effectiveness of contextual defenses on strong models has yet to be validated by sufficient successful attacks. (2) The additional token overhead and latency costs introduced by the PIPES method. (3) The method relies on pre-built tool contracts and information from trusted sources, and its cross-tool applicability remains unclear. Based on these points, I still have concerns about the practical applicability of PIPES, so I will maintain my score.

Reviewer_RXfKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper identifies an interesting attack surface in tool-using LLM agents, where attackers manipulate low-trust tool-response components to corrupt the agent's perceived environment rather than issuing explicit malicious instructions. The authors introduce state-corruption attacks, in which attacker-controlled content makes claims beyond its provenance authority, potentially causing an agent to take unintended actions that appear legitimate under existing guardrails.

To address this, the authors propose PIPES, a provenance-informed screening mechanism that evaluates tool-response units against semantic priors and provenance hierarchies before they enter the agent's context. The evaluation spans three LLMs and six benchmark splits from VitaBench and AgentDyn, demonstrating substantial reductions in attack success rate (ASR) while maintaining comparable benign utility.

Critical strengths
  1. Interesting attack surface: The paper identifies an important distinction between conventional indirect prompt injection and state corruption. It demonstrates how low-trust metadata can introduce unauthorized environmental claims that influence an agent's decisions without explicit instructions.
  2. Simple yet effective defense with strong evaluation: PIPES presents an elegant design that combines semantic priors and provenance hierarchy to identify unauthorized claims at the tool-response boundary. The evaluation across multiple models, tasks, and adaptive attacks is convincing, with PIPES consistently achieving the lowest average ASR. While utility varies across settings, the security-utility tradeoff appears reasonable.
Critical weaknesses
  1. Threat-model realism: The threat model assumes that an adversary can replace a specific component of a tool response while leaving the surrounding response unchanged. Although the authors constrain this to externally writable fields, the paper could better justify how this control maps to realistic tool integrations and distinguish legitimate third-party content manipulation from compromised tool responses or communication channels.
  2. Reliability of provenance-based assessment: PIPES depends on correctly identifying provenance units, associating them with trusted source metadata, and using an LLM to evaluate their compliance with semantic priors and provenance hierarchies. While the design is reasonable, the evaluation does not sufficiently isolate the reliability of these intermediate steps. Errors in extraction, provenance attribution, or LLM assessment could directly affect the effectiveness of the defense.
Questions for authors
  1. Threat model (clarification): Could the authors clarify how the assumed attacker control over a tool-response component manifests in realistic deployments? A concrete end-to-end example would help establish the practical applicability of the threat model.
  2. Assessment reliability (clarification and potential evaluation): How do the authors validate the correctness of provenance-unit extraction, source attribution, and LLM-based compliance assessment as done by past works [1]? In particular, have they measured the assessor's false-positive and false-negative rates against independently annotated ground truth?

[1] Mukherjee, K. and Kantarcioglu, M. (2025) ‘LLM-driven Provenance Forensics for Threat Investigation and Detection’. arXiv preprint, arXiv:2508.21323. Available at: https://arxiv.org/abs/2508.21323

1 comment

We thank the reviewer for their thoughtful review that recognizes the novelty of the attack vector and the strength of our defense. We address the review's concerns below.

Q1. Could the authors clarify how the assumed attacker control over a tool-response component manifests in realistic deployments?

This control arises naturally in real-world APIs that combine content from multiple sources in one response. For example, BigCommerce, an e-commerce platform, exposes merchant-managed product descriptions and customer-authored reviews through its GraphQL Storefront API. A single query can request both description and reviews from the same product, returning them together in one response [1][2]. A shopping-agent tool using this query therefore returns mixed-provenance information. An adversary who authors a review included in that response controls the review text, without controlling the surrounding merchant-maintained product information or compromising the tool or communication channel. This directly illustrates the component-level attacker control assumed in our threat model.

Q2a. How is the correctness of provenance-unit extraction and source attribution validated?

Provenance units and source attribution are specified by the tool designer through the response structure and trusted metadata, rather than inferred by the LLM assessor. In the BigCommerce example above, the integration can identify the product's description and each review's text as separate units and attach their merchant and customer provenance, respectively. The assessor then evaluates the content against those supplied boundaries.

Q2b. How is LLM-based compliance assessment validated? Have the assessor's false-positive and false-negative rates been measured against independently annotated ground truth?

For static PIPES on VitaBench, we estimate the false-positive rate from how often the guardrail intervenes on unmodified benign trajectories, assuming these responses do not violate their static contracts. As reported in Appendix F, static PIPES removed content from only 16 of 9,686 screened responses (0.17%).

Measuring the false-negative rate is more difficult because our adaptive attacker probes the boundaries of the contracts. When PIPES does not flag an adversarial payload, the payload may either remain within the contract or constitute a violation that the assessor missed. Distinguishing these cases requires independent ground-truth annotation, which is challenging for borderline cases given the semantic nature of the contracts. We therefore focus on the downstream security metric, attack success rate (ASR), which is directly measurable using the benchmark's success criteria. PIPES achieves the lowest mean ASR for each target model across the six benchmark splits, demonstrating its benefit over the other evaluated defenses.

References

[1] BigCommerce. Product. GraphQL Storefront API documentation. https://docs.bigcommerce.com/developer/api-reference/graphql/storefront/types/objects/product

[2] BigCommerce. Product Reviews with the GraphQL Storefront API. Developer documentation. https://docs.bigcommerce.com/developer/docs/storefront/guides/graphql-storefront-api/products-and-catalog/product-reviews

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.