acceptodds
Under review as a conference paper at ICLR 2027

AgentSiphon: Privilege-Crossing Exfiltration through Covert Generation in RAG Agents

Abstract

Retrieval-augmented generation (RAG) agents increasingly combine untrusted knowledge sources with privileged access to protected data and tools, creating an information-flow vulnerability in which an attacker who cannot directly access protected data may induce an authorized agent to retrieve and covertly leak it through seemingly benign responses. Prior poisoning attacks show that compromised retrieval can induce malicious actions, but primarily study observable behavioral failures. Whether retrieval influence can instead cause an authorized agent to acquire protected information and transmit it through otherwise valid responses remains underexplored. To uncover such vulnerabilities, we introduce **AgentSiphon**, a privilege-crossing exfiltration framework that decomposes this threat into three measurable stages: (i) trigger-conditioned retrieval of an attacker-influenced instruction, (ii) agent-mediated acquisition of a dynamically selected secret, and (iii) covert transmission through token choices in otherwise task-valid responses. Unlike attacks that produce fixed malicious actions or explicitly reveal secrets, AgentSiphon treats language generation as a stateful communication channel, supporting different secret instances, multi-interaction payloads, and redundancy-based recovery. We evaluate AgentSiphon using synthetic canaries across healthcare and general-domain RAG tasks, multiple language-model families, and varying payload lengths, interaction budgets, and channel strengths. Our experiments characterize the trade-off among secret recovery, bit error rate, communication cost, task fidelity, and detectability, while ablations isolate the effects of persistent state, phase separation, and error correction. The results demonstrate that privileged information can be reliably recovered while preserving useful, benign-looking responses. AgentSiphon therefore exposes a security boundary overlooked by conventional retrieval-integrity or action-monitoring defenses, in which preserving the visible correctness of an agent’s response does not prevent its generation process from transmitting information obtained from privileged context.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, retrieval-augmented generation, data privacy, data exfiltration, covert channels, knowledge base poisoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WzWYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose AgentSiphon, a RAG agent attack framework consisting of "trigger-conditioned retrieval", "agent-mediated acquisition", and "HGA-based covert generation". The overall idea is that an attacker-controlled content can cause the agent to retrieve a protected document (since it has the authority to do so, unlike the attacker), and following this, under gray-box settings, information on the protected document can be transmitted to the attack through HGA. The main novelty, as i understand, is combining retrieval poisoning and steganographic methods to attack RAG agents.

Critical strengths
  1. Method seems plausible and potentially effective

The paper proposes using HGA-based covert generation to extract information following an indirect prompt injection attack. Such an attack seem plausible in gray-box settings, motivating further research into security mechanisms mitigating such attacks.

  1. Clear empirical results

Results presented in the paper indicate that AgentSiphon works well under the assumed setting.

Critical weaknesses
  1. On writing

Although HGA is a central component of AgentSiphon, the paper does not detail sufficient information on how it works in the AgentSiphon scenario to encode hidden payloads into the generation. The paper does not read in a self-contained way, and should be revised to incorporate such information.

Questions for authors
  1. Minor typo

Line 048: "... privileged operations x Ruan et al. (2024)"

4 comments
AuthorsOpenReview ↗

HGA is not self-contained

We thank the reviewer for pointing out that HGA, despite being central to the method, was not explained in sufficient detail for readers to understand it independently of the cited work.

We have expanded Section 3 (pages 3–4) to describe the hash-then-greedy vocabulary partition, the bias parameter β, the eligibility threshold η, and the seeded decoding procedure. We also explicitly clarify that HGA is unrelated to hierarchical graph methods.

Algorithm 1 (Section 5, page 6) now presents a complete AgentSiphon episode, covering Activate, Acquire, Transmit, and Recover. This provides a self-contained description of how the mechanism is used in our framework without requiring readers to consult Silva et al. (2024) to follow the overall procedure.

AuthorsOpenReview ↗

Typo at line 048

Thank you for catching this issue. We have corrected the citation formatting in the Introduction.

The corrected passage reads:

"Existing work studies related pieces of this problem. Agent security and information-flow research examines tool use and privileged operations (Ruan et al., 2024; Costa et al., 2025; Kolluri et al., 2026), while steganographic methods hide information in generated text (Fang et al., 2017; Ziegler et al., 2019; Silva et al., 2024)."

This appears in Section 1, page 1, lines 048–051 of the revised manuscript.

AuthorsOpenReview ↗

Additional improvements

In response to the other reviews, we have also made three additions that may be useful context for the reviewer:

  • A worked example showing the complete pipeline through a concrete trigger and acquired field (Section 5, page 6).
  • A prompt-injection defense experiment evaluating acquisition under a manually written, non-adaptive defense (Section 6.3, Table 3, page 9).
  • A comparison with the three closest data-exfiltration attacks, clarifying the differences in attacker capabilities, secret acquisition, and transmission mechanisms (Table 1, Section 2, page 4).

Thank you for identifying the presentation issues and for the suggestions that helped us clarify the method. Kindly Let us know if any more clarifications.

Thank you again for your helpful feedback and for the suggestions on improving the presentation of our work.

We have expanded the explanation of HGA, including its underlying mechanism and how it is incorporated into AgentSiphon. We have also added a complete algorithm describing the attack pipeline and addressed the presentation issues identified in your review.

In addition, we have incorporated a worked example and further experimental clarifications arising from the other reviews.

We would be grateful to know whether these additions have sufficiently addressed your concerns or whether there are any further aspects that you feel require clarification.

If you find the revisions satisfactory, we would sincerely appreciate your continued consideration of our submission.

Thank you once again for your time and constructive feedback.

Reviewer_Q4sVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes an attack against RAG systems in which, in addition to responding based on retrieved context, the agent can invoke tools and access a protected database that is not reachable through retrieval. The attack manipulates retrieval so that the agent accesses the protected database, obtains private data, and covertly discloses it to the attacker in its output.

Critical strengths

The core idea of manipulating retrieval to influence the agent’s behavior, causing it to take actions that leak private information through its output, is very interesting.

Critical weaknesses
  • Clarity and presentation. The paper is difficult to follow. The prose is dense and relies on many undefined terms. For example, the paper refers to HGA for the generation step without defining it. I initially assumed it referred to a hierarchical graph, and only the citation in Section 3 clarified that it refers to a different method. Since the cited work is not widely known (2 citations), readers cannot be expected to be familiar with it, especially as the method relies heavily on it. Other terms are also left undefined, such as “episodes” in Section 5.1. In addition, the core pipeline (retrieval -> private data access -> covert generation) is restated many times, but it is never illustrated with a concrete example or connected to real-world systems.

  • System design. The paper relies on a very specific architecture. Given a query, the system retrieves documents and an instruction, invokes two functions (validate and resolve) to obtain private information, and then uses covert generation to encode it. Since the system was not designed to reveal this information, it is unclear why it would follow this protocol. This is not a standard or widely adopted framework, and its design is not justified anywhere in the paper. Why would a system retrieve instructions, and what exactly are these instructions? What role does the retrieved context play? Why would a standard RAG system, whose purpose is to answer queries, include a covert generation component?

  • Threat model. The threat model appears unrealistic. The introduction suggests that the attacker controls documents in the corpus and uses them to induce the agent to retrieve private information. In practice, however, the corpus is not actually used, and the attacker instead controls a model that maps queries to instructions (whose nature remains unclear to me). Why would the attacker have this capability? Even assuming a covert generation process exists, why would the attacker know its inner workings? Overall, the setting seems unrealistic.

  • Figures and formatting. Figures are very dense, font is tiny, figures 1 and 2 that are supposed to explain the method are not referenced in the text. All citations used citet instead of citep.

  • Evaluation. The evaluation is unclear. It is conducted on only two small models, using metrics that are not clearly defined or explained.

Questions for authors
  • Could you justify the design of the system and provide concrete examples of where such a system would be deployed?

  • Could you justify the threat model?

11 comments
AuthorsOpenReview ↗

A component that influences decoding would already reveal the secret

We appreciate the reviewer raising this point. The question highlights an ambiguity in the original description of what the generation-side implant can access and what it can actually do. We have revised Section 4 to make this distinction explicit.

The generation-side implant is egress-restricted. Its only capability is to bias token selection toward one of the two HGA partitions (Section 4, "What the attacker controls," page 5). It cannot independently read application memory, access the network, or determine where the acquired value is used downstream.

The receiver is external and observes only the visible responses Y₁,H (Section 4, "What is observed and trusted," page 5). Although the generation-side component must process the protected value to encode it, it has no independent outbound channel through which to send that value to the receiver. The generated response is therefore the only communication path available to the attacker.

We have added the following clarification:

"A fully compromised host that could read application memory and open a network connection would not need AGENT SIPHON; our model is strictly weaker and better matched to supply-chain and model-provider compromise."

AuthorsOpenReview ↗

The assumption that the attacker influences decoding

We agree that influencing the decoding step is a stronger assumption than what some previous exfiltration attacks require. We now acknowledge this explicitly in the Related Work comparison (Section 2, page 3):

"It also needs a stronger assumption, influence over the decoding step (Section 4), that these attacks do not."

The capability is narrow: the implant can only influence which tokens are sampled and cannot perform other operations (Section 4, page 5).

We also identify plausible sources of this capability, including a tampered decoding library, a backdoored sampling policy, or a model obtained from a compromised provider. These are motivating examples of how such influence could arise, not live compromises that we implement or evaluate.

We make this distinction explicit in Section 4 ("Demonstrated versus assumed," page 5).

Our claim is not that an attacker with unrestricted control can exfiltrate data. Rather, we investigate whether an attacker with this limited decoding capability can cross an information-flow boundary even when retrieval-integrity checks, authorized-action monitoring, and response-correctness checks are applied independently (Section 4, page 5).

AuthorsOpenReview ↗

Authorization and privilege

The reviewer questions why an agent would invoke Validate/Resolve when the system was not designed to disclose the protected information. We would like to clarify the distinction between performing an authorized operation and disclosing its result.

We are not claiming that RAG systems are designed to leak data or that Validate/Resolve is inherently a disclosure mechanism.

In our setting, the agent already has legitimate access to the protected resource through its existing application interface as part of its normal operation. For example, a clinical question-answering agent may legitimately access its EHR database to answer a diagnosis-related question (Section 4, "Setting," page 4).

The attack does not introduce a new application capability or bypass authorization. Instead, it redirects an existing authorized operation. The attacker-controlled retrieval entry contains only an instruction and never embeds the protected value itself (Section 5.1, page 6). Following that instruction, the agent makes a request that the host application would otherwise be permitted to execute.

As stated in Section 5.2 (page 7):

"No protected-resource credentials or authorization bypass are required; the violation occurs when the authorized result enters attacker-observable responses."

The distinction we make in Section 4 (page 4) is between operation authorization, which permits an application operation, and information-flow authorization, which determines whether the result may be disclosed. AgentSiphon respects the former while violating the latter.

Reviewer_nfCoOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes AgentSiphon, a data exfiltration attack against RAG-based systems. The authors propose a combination of distinct primitives to carry out their attack 1) using a trigger to retrieve some attacker-desired instruction, 2) using benign tool calls to fetch sensitive data, and 3) exfiltrating the data through an attacker-observable channel. The authors evaluate their attack on three distinct benchmarks that could feasibly involve sensitive data (i.e., health records and financial data) and find that the attack can recover data from certain models.

Critical strengths
  1. This is a contemporary threat model; as users continue to rely on agents for personal tasks, data exfiltration attacks will become increasingly relevant. I think it is valuable to continue spending research effort in this area, so that as a community we can eventually come up with workable defense mechanisms.
Critical weaknesses
  1. The presentation and writing quality of this submission is convoluted and obscure. The core aim of the attack is clear (i.e., force a RAG agent to exfiltrate data across an attacker-observable channel). However, the remainder of the presentation is dense and difficult to parse; the exact mechanism by which adversarial retrieval and exfiltration occur are not specified in pseudocode or otherwise explained in a clear manner. I would suggest that the authors focus on presenting the core elements of their technique more clearly. One approach is to proactively explain techniques and/or acronyms before they are referenced (i.e., HGA).

  2. The primitives are not described in sufficient detail. For instance, why is HGA preferable for covert transmission? Are there any other approaches that could have plausibly been used instead? Because HGA is central to the paper’s argument, there should be an in-depth discussion about the technique, and how exactly it was combined with the other mechanisms in the submission.

  3. I’m not super convinced by the experimental results as presented in the manuscript.

    • The authors only test on smaller, instruction-tuned models (i.e., Llama 3.23BInstruct and Qwen 3.54B). Most models used in production systems are larger and reasoning-based; the attack would be more convincing if it were to be successful against modern models.
    • The RAG harness used in experiments is not clear. The authors do not explain how they constructed the harness, or why they didn’t use standard off-the-shelf implementations.
    • It looks like the authors evaluate their attack against undefended LLMs. How does the performance of the attack change as standard defense techniques are applied? Even a simple baseline, like a prompt-based defense, would be useful to show.
  4. Ultimately, the submission seems to have limited novelty given the existence of other recent work on data exfiltration. For instance:

    I think the authors should at the least incorporate a brief discussion of these existing attacks and why they are insufficient for the specified task. Preferably, the authors should provide some empirical justification / evidence for these claims as well.

Questions for authors

Thank you for submitting your work to ICLR 2027. Unfortunately, in its current state, I do not think this submission is ready for publication. The core idea is clear but feels limited in novelty given existing work on data exfiltration for LLM agents. The manuscript is also hard to read and is convoluted in presentation. I would encourage the authors to focus more on clearly explaining the crux of their idea, along with the core improvements they introduce compared to existing attack schemes. A couple questions for the authors:

  • How did the authors settle on the attack design presented in this paper? Is there any reason particular methods (i.e., HGA) were used over others, and are there ablations that support these decisions?
  • Is there any reason that all agents were evaluated in an undefended state? I agree that there might not be a super rigorous system out there that can prevent these types of attacks, but at the least it would be prudent to evaluate some basic baselines (i.e., a prompt-based defense)?
9 comments
AuthorsOpenReview ↗

Novelty relative to prior exfiltration attacks

We thank the reviewer for the careful reading. We address the concern that "the submission seems to have limited novelty given the existence of other recent work on data exfiltration," particularly the three papers mentioned: Inadvertent Context Leakage in Language Models, Your LLM Agent Can Leak Your Data, and Simple Prompt Injection Attacks Can Leak Personal Data.

We have added Table 1 (Section 2, page 4) to compare AgentSiphon with these three works along five dimensions: attacker capability, the secret's initial location, acquisition mechanism, transmission channel, and evaluation scope. We also acknowledge the overlap with Back-Reveal (Zhang and Pei, 2026), which uses semantic triggers, a backdoored agent, and multi-interaction leakage. This overlap is stated explicitly in the table caption (page 4) and the "Closest exfiltration attacks" paragraph (Section 2, page 3).

The differences are as follows:

  • Back-Reveal: The secret is already present in session memory. It is obtained through memory-access tool calls and transmitted through disguised outbound requests, which an egress-monitoring mechanism could potentially observe.
  • Alizadeh et al. (2025): The secret is personal data that the agent has already observed while performing the task. It is disclosed directly in the response, without an engineered transmission channel.
  • Fairoze et al. (2026): The secret is placed directly in the model's context and recovered through correlations in its ordinary outputs by an adaptive black-box adversary, without an engineered encoder.

AgentSiphon differs from these approaches along three dimensions simultaneously (Table 1, row 4). First, the protected value is not already present in the model's context. It is resolved at execution time through an authorized application call (Section 5.2, page 7). Second, the value is transmitted through the visible answer using an engineered, decodable channel, without generating an outbound request for an egress filter to inspect (Section 5.3, page 7). Third, the generation-side implant is egress-restricted: it can only influence token selection and has no independent network or application-memory access (Section 4, page 5).

To our knowledge, this particular combination of acquisition through an authorized application interface and transmission without an outbound network request has not been demonstrated in these prior settings. The threat models differ substantially, so we have not conducted a direct empirical comparison with these attacks. We have clarified this distinction rather than suggesting that the underlying mechanisms are entirely unrelated.

AuthorsOpenReview ↗

Use of a custom harness instead of a standard benchmark

We address the question of why we used a custom harness instead of an existing benchmark. Our evaluation requires retrieval, an authorized protected-resource interface, and generation to be independently measurable. We are not aware of an existing agent benchmark, including AgentDojo, that provides this separation in the form required by our threat model.

We now explain this choice in Section 6.1 ("Why a controlled harness," page 8) and provide the implementation and evaluation details in Appendix A (pages 14–15), including the three application settings, model configurations, random seeds, and baseline definitions.

We have also made the limitations of this evaluation explicit. The harness does not evaluate production agent stacks, defenses beyond the one considered in Section 6.3, adaptive attackers, or whether an actual decoding-side compromise would evade platform integrity checks.

AuthorsOpenReview ↗

Evaluation against defenses

The reviewer suggests evaluating the attack against a simple defense, such as a prompt-based defense. We have added this experiment in Section 6.3 and Table 3 (page 9).

For the eICU setting, we introduce a prompt-based defense that marks retrieved entries as untrusted and instructs the agent to call the tool only when the question requires it. Under this defense, Qwen's acquisition rate decreases from 0.452 to 0.108 in the free-form condition, whereas Llama changes from 0.970 to 0.950.

We also include a benign-entry control (Table 3, bottom row; additional details in Appendix L, page 30). Without the injected instruction, Qwen selects the protected target in only 0.5% of trials, while Llama does so in 12.3% of trials.

These results help distinguish the effect of the injected instruction from target selection driven by the query alone. However, we do not interpret them as evidence that the defense generalizes. As clarified in Section 6.3, this experiment evaluates acquisition only, uses one manually written, non-adaptive defense, and does not establish defended end-to-end recovery.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.