- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper identifies an “agent perception gap” in tool-using LLM systems: tool responses often do not explicitly encode which source controls each response component or what semantic claims that component is authorized to make. The paper argues that this enables state-corruption attacks, where adversarially controlled content makes environmental claims beyond its informational authority. Unlike conventional prompt injection, these attacks need not contain explicit instructions; instead, they corrupt the agent's perceived state such that the attacker-desired action appears locally consistent with the resulting environment representation.
The authors propose PIPES (Provenance-Informed, Prior-Enforced Screening), which screens tool responses before they enter the agent context. PIPES checks each field or provenance unit for (1) consistency with a semantic prior defining what that component is expected to convey, and (2) consistency with a provenance hierarchy defining which sources are authorized to establish or override particular information. Structured tools use static field-level contracts, whereas open-ended responses such as emails, files, and webpages use a broad tool prior specialized by the pre-response trajectory and trusted provenance metadata.
The paper evaluates PIPES on three VitaBench and three AgentDyn splits with Gemma 4 31B IT, GPT-5.6 Luna, and Claude Sonnet 5. Attacks are optimized separately against each defense using a white-box PAIR-style adaptive attacker. PIPES achieves the lowest mean ASR among the evaluated defenses for all three target models while maintaining benign utility comparable to the undefended agent.
Critical strengths
The paper identifies a meaningful gap in current IPI defenses. The distinction between explicit control-transfer attacks and data-like state corruption is compelling: an attacker does not necessarily need to instruct the model to take an unauthorized action if it can instead corrupt the environmental facts that justify the action.
The decomposition into semantic priors and provenance authority is intuitive and useful. A response field should both remain within its expected informational role and respect the authority of the source that produced it. Importantly, PIPES does not reject content merely because it originates from an untrusted source.
The distinction between static and contextual priors is also well motivated. Structured response fields can often support stable field-level contracts, whereas emails, webpages, and files require task-dependent expectations inferred from the preceding trajectory. The use of source-attributable provenance units allows the method to preserve compliant content instead of discarding entire responses.
The adaptive attack evaluation is a strong aspect of the paper. Payloads are re-optimized against each defense, and the attacker receives substantial feedback, including PIPES assessments and static contracts. Therefore, the security gains cannot be explained merely by poor transfer of fixed attacks across defenses. The attack-instance construction also appears careful: VitaBench surfaces and goals are selected without access to PIPES's semantic priors or provenance hierarchy, and causal validation attempts to avoid labeling benign authoritative state changes as attacks.
The empirical results are strong across the evaluated settings. PIPES substantially reduces ASR for Gemma on both VitaBench and AgentDyn and consistently obtains the lowest mean ASR among the evaluated defenses. The ablations further indicate that priors and provenance are complementary, particularly in open-ended contextual settings.
Finally, the paper provides useful analyses of benign removals and token overhead, and clearly states several limitations.
Critical weaknesses
My primary concern is the amount of trusted tool-specific instrumentation required by PIPES. The system relies on semantic priors, provenance labels/orderings, deterministic source extractors, and sometimes authenticated provenance anchors. Existing tools may not expose these contracts, and in the experiments the missing information is constructed offline using documentation, response structure, and representative benign values. This is a meaningful systems assumption and potentially a substantial deployment cost. I would like to understand the manual effort required to construct and maintain these contracts and the method's sensitivity to imperfect or stale specifications.
Relatedly, the comparison with other defenses conflates two effects: PIPES introduces a new screening algorithm, but it also receives richer trusted semantic/provenance information than several baselines. PromptArmor, for example, is primarily an instruction detector and does not receive the same semantic authority structure. A control in which a generic LLM filter or action validator receives the same contracts and provenance metadata would help isolate the value of the PIPES-specific formulation.
The ablation also suggests that the contribution of provenance differs substantially across settings. On VitaBench, prior-only screening already obtains most of the improvement (14.8% mean ASR versus 11.2% for full PIPES), while provenance-only screening is much weaker. The joint benefit is clearer on AgentDyn. I think the paper should emphasize this asymmetry more explicitly.
A further concern is robustness to specification quality. Real tools often expose broad fields such as descriptions, comments, email bodies, and arbitrary metadata whose legitimate semantics are difficult to characterize narrowly. Overly narrow priors may remove legitimate content, whereas overly broad priors may admit state-corruption attacks. The current benchmarks provide relatively controlled tool environments, and AgentDyn contains only 20 tasks per split.
The adaptive evaluation is strong, but the ten-attempt budget may still limit conclusions. Reporting ASR as a function of the number of PAIR attempts would show whether attack success has saturated. Moreover, Gemma 4 31B IT is used as the fixed attacker even against stronger target models; the very low undefended AgentDyn ASR for Luna and Sonnet suggests a possible attacker bottleneck/floor effect.
Finally, the end-to-end baseline set does not include several conceptually close typed-interface or provenance/information-flow defenses discussed in the related-work section. If these methods cannot be instantiated in the benchmark environments, the paper should explain this more explicitly.
Questions for authors
-
How much manual effort is required to construct a PIPES contract for a new tool, and how does the approach scale to large or rapidly changing tool ecosystems? Can the authors quantify authoring cost or test automatically generated/noisy contracts?
-
Can the authors provide a baseline that receives the same semantic priors and provenance metadata as PIPES but uses a generic LLM-based screening/action-validation prompt? This would separate gains from richer trusted metadata from gains due to PIPES's specific formulation.
-
Does adaptive ASR saturate within the ten PAIR attempts? Could the authors report attack-budget curves and evaluate a stronger or multi-model attacker on a subset of tasks?
-
How sensitive is PIPES to incorrect, incomplete, overly narrow, or overly broad semantic priors and provenance orderings?
-
Why are the closest typed/provenance-aware defenses discussed in Related Work not included in the end-to-end comparison? Are they incompatible with the evaluated environments?
2 comments
Thank you for the response and the additional experiments. I have noted the clarification of contract construction, the contracts-in-agent comparison, and the attack-budget curves. The sensitivity to specification errors and evaluation with stronger attackers remain limitations.
Rebuttal
We thank the reviewer for recognizing the state-corruption threat, the value of semantic priors and provenance, and the strength of our adaptive evaluation. We address the reviewer’s concerns below.
Q1. How much effort is required to construct and maintain contracts for new tools or changing tool ecosystems? Can authoring cost or automatically generated contracts be evaluated?
Existing tools generally do not publish explicit field contracts. We therefore built an automated offline pipeline that constructs semantic priors and provenance annotations from tool documentation, response structure, and benign rollouts. Our reported evaluation uses profiles constructed by this pipeline, frozen before attack evaluation without using attack payloads or defense outcomes.
By demonstrating substantial improvements in robustness, we hope to encourage tool developers to publish semantic priors and provenance information alongside their response schemas. This would reduce the need to reconstruct missing contracts and support wider adoption of the defense.
Q2. How does PIPES compare with a generic LLM screener or action validator given the same semantic priors and provenance metadata?
To test whether richer trusted metadata explains the gains, we evaluated a Gemma contracts-in-agent baseline by inlining the same response schemas, semantic contracts, provenance information, and checking instructions alongside tool descriptions in the agent's system prompt. The agent applies these checks itself, without a separate screener or response removal. We retain the same attack cohorts and ten-attempt adaptive PAIR protocol, including target reasoning feedback.
Mean ASR across splits (%):
While providing contracts directly to the agent improves robustness over the baseline, a separate screening stage provides further protection.
Q3. Does ASR saturate within ten PAIR attempts? What do attack-budget curves and a stronger or multi-model attacker show?
Attack-budget curves. We reconstructed cumulative ASR at each PAIR attempt budget from the saved adaptive attack traces, keeping the retained cohorts fixed and averaging ASR equally across the six splits. Gemma 4 results (%) are:
Across defenses, 70.2–78.7% of the final mean ASR is reached within the first five PAIR attempts. Although ASR does not fully saturate, the average ASR gain per additional attempt is substantially smaller in the second half of the budget. We leave evaluation with stronger or multiple attacker models to future work.
Q4. How sensitive is PIPES to incorrect, incomplete, overly narrow, or overly broad priors and provenance orderings, including noisy contracts?
We expect PIPES’s robustness and utility to depend on the accuracy of its contracts and provenance information. Overly narrow contracts may remove legitimate content, while overly broad contracts may admit attacks. Although our evaluation uses automatically constructed contracts, we did not explicitly inject specification errors to measure sensitivity. We leave this systematic evaluation to future work.
Q5. Why were close typed-interface and provenance-aware defenses omitted? Can they be instantiated in the evaluated environments?
The cited typed-interface and provenance-aware defenses impose restrictions that make them difficult to instantiate in our current benchmarks. We explain why this is the case with specific prior works below.
Untrusted Content Masking. UCM is designed for browser-based web agents: it masks untrusted DOM elements before page rendering and is evaluated on custom websites and WebArena GitLab (Nikolić et al., 2026, Sections 3–5 and Appendix B.2). Our VitaBench and AgentDyn evaluations use tool APIs without this browser interface, so the published UCM implementation is not directly applicable.
Type-directed privilege separation. This method restricts communication from a quarantined agent to the privileged planner to curated types such as integers, floats, booleans, and enums; it explicitly disallows free-form text across this boundary (Jacob et al., 2026, Section III-B). PIPES instead screens free-form tool responses while preserving admissible text. Applying type-directed separation would require redesigning the agent’s handoffs around these restricted types.