acceptodds
Under review as a conference paper at ICLR 2027

Query Obfuscation with Embedding-Optimized Noise (EON): A Representational Blindspot in Agentic Retrieval

Abstract

In modern agentic and semantic retrieval systems, dense embedding models increasingly determine which intent, tool, document, memory, or image to retrieve for a query. We identify a representational failure mode in this pipeline: a meaningful query can be replaced by visibly nonsemantic content while preserving much of its retrieval outcome, even when the candidate bank is hidden from the optimizer. We introduce *Embedding-Optimized Noise (EON)*, constructed using only the natural query representation, without access to candidate embeddings, relevance labels, rankings, retrieval feedback, or guardrail decisions. Despite this corpus-blind objective, EONs frequently preserve intent and tool routing, benchmark document search, episodic agent-memory retrieval, and cross-modal search across multiple dense embedding families. Crucially, this disconnect carries serious security implications: for malicious queries, an EON can reliably evade input guardrails that block the natural query, executing the intended safety-sensitive retrieval undetected. Through the geometry of the embedding space, we then characterize how query-level representation matching can reliably maintain subsequent retrieval outcomes over candidates unseen during optimization. Together, these findings expose a fundamental phenomenon across dual encoders that we call BLINDSPOT, where systems remain operationally blind to nonsemantic artifacts while reliably retrieving real, consequential content.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
semantic retrieval, agentic retrieval, query obfuscation, retrieval security, embedding robustness

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9cHGOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Embedding-Optimized Noise (EON), aiming to create non-semantic image/text that have a high embedding similarity of a natural query. It can retrieval dangerous items to agent contexts without being detected by input guardrails.

Critical strengths

The paper is very clear and easy to follow. The motivation and idea of finding blindspots of embedding models is very interesting.

Critical weaknesses

(1) The contribution of this paper is very limited. The main idea and method of this paper is very general and similar to prior adversarial attacks [1,2] targeting LLM and embedding models. While many related works on adversarial attacks are not cited in this paper. No baseline comparison is conducted in experiments.

(2) The robustness evaluation of the proposed method is missing. Since the generated EON is very obvious garbled text/image, it should be easily detected and removed by many other ways except safety guardrails (e.g. [3,4]).

References

[1] Explaining and Harnessing Adversarial Examples. ICLR 2015

[2] Universal and Transferable Adversarial Attacks on Aligned Language Models. arxiv 2307.15043

[3] Detecting Language Model Attacks with Perplexity. arXiv:2308.14132

[4] Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense. NeurIPS'23

Questions for authors

(1) What is the main difference between EON and prior works on adversarial attacks? Could the authors further clarify their contribution?

(2) Could the authors compare their method to prior works on adversarial attacks? What is the advantage of the proposed method?

(3) How robust and stealth is EON? Can it survive detection and purification process such as filtering, paraphrasing, image cropping, diffusion-based image reconstruction, and embedding model-based classification?

(4) What is the threat model of this paper? Is the embedding model black-box or white-box to the attacker?

1 comment
AuthorsOpenReview ↗

Black-Box Construction, Contribution, and Defenses

We thank the reviewer for the thoughtful feedback and the appreciation of our motivation. We address the questions below, beginning with the threat model.

4. Threat model

Black-box or white-box? The encoder is black-box to the attacker. The construction is gradient-free and uses only forward embeddings, without access to gradients or model parameters. We have explicitly mentioned this in the Introduction and Section 3.2, with further details in Appendix A.4.

Threat model. We describe the threat model in detail in Section 3.1: optimization uses no candidate identities, embeddings, rankings, retrieval feedback, or guardrail decisions. An accessible local copy of the target encoder allows offline construction followed by submission of the EON.

1. Contribution and prior work

Difference from prior adversarial attacks. Prior work has explored adversarial inputs, semantic collisions, and image queries designed to preserve retrieval results. Building on these findings, BLINDSPOT systematically studies the remarkable ability of nonsensical, nonsemantic text and images to reproduce meaningful queries' retrieval outcomes across encoders and tasks, without access to the candidate bank. For images, this includes queries constructed by changing a few pixels on a blank canvas.

Contribution. The central finding is that a nonsensical proxy can preserve the retrieval behavior of a meaningful query. For example, an unreadable token sequence optimized to approximate the embedding of “a red car” can retrieve relevant car images from a collection unseen during optimization. We further formalize retrieval preservation through geometric bounds and study its security implications: semantic guards can admit a nonsensical proxy while blocking its natural query, allowing sensitive content to reach downstream processing.

2. Comparison and advantage

We deliberately use a simple optimizer and compare it with a white-box, gradient-based approach under the same representation objective, finding that the phenomenon arises with both (submitted Table 13). Random-noise and permutation controls provide further context (submitted Tables 18–19). The main takeaway is that nonsensical inputs can align with meaningful queries in embedding space closely enough to preserve retrieval, with security implications. Our method requires no access to the candidate bank and uses only encoder outputs, making it applicable without the gradient access required by white-box attacks.

3. Detection and robustness

Detection and filtering. Larger semantic guardrails often admitted EONs while blocking the corresponding natural queries. To highlight this vulnerability and offer a simple mitigation, we deliberately included a lightweight gibberish detector. It intercepted all evaluated text EONs, reducing text bypass to zero, with 1.0–4.4% natural-query false positives (submitted Table 8). Thus, a simple detector was sufficient for the evaluated text EONs.

Robustness to other defenses. Tables 15–17 evaluate late interaction, reranking, and transfer, while Table 19 examines token order. Paraphrasing, cropping, and diffusion reconstruction remain outside the evaluated scope.

We hope these clarifications help address the reviewer's concerns, and would be grateful if the reviewer would consider revisiting their assessment if they find it appropriate.

Reviewer_3ajBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates a representational mismatch in dense retrieval systems, where nonsemantic queries can preserve the retrieval behavior of natural-language queries. The authors introduce Embedding-Optimized Noise (EON), a gradient-free optimization method that constructs nonsemantic query proxies by maximizing embedding similarity to the original query without accessing candidate corpora, retrieval feedback, or guardrail decisions. The paper demonstrates that EONs can preserve retrieval outcomes across intent routing, tool selection, document retrieval, agent memory, and multimodal retrieval. It further shows that such proxies can bypass semantic safety guardrails while preserving safety-sensitive retrieval outcomes, a phenomenon termed BLINDSPOT.

Critical strengths
  1. The paper highlights a mismatch between semantic guardrails and embedding-based retrieval: a nonsemantic query can bypass input moderation while preserving the retrieval behavior of a blocked natural query. This observation is interesting, particularly given the increasing reliance on dense retrieval in RAG and agentic systems. The corpus-blind setting also distinguishes the work from attacks that explicitly optimize against known retrieval candidates.

  2. The authors evaluate the phenomenon across multiple embedding models, retrieval tasks, and modalities, including query-specific memory retrieval. These evaluations provide reasonably convincing evidence that the observed retrieval preservation is attributable to the optimized embeddings rather than random chance.

Critical weaknesses
  1. The core approach optimizes a nonsemantic query to approximate the embedding of a natural query, thereby preserving its retrieval behavior. However, the relationship between embedding similarity and retrieval preservation is largely an inherent property of dense retrieval. The theoretical analysis formalizes this relationship using standard vector similarity bounds but provides limited additional insight into the underlying failure mode.

  2. While the experiments demonstrate retrieval preservation and guardrail bypass, they do not establish whether these effects translate into meaningful security failures in realistic agentic systems. In particular, the evaluation focuses on retrieval outcomes rather than downstream agent behavior. Since EON queries are deliberately nonsemantic, it is unclear whether an agent receiving such a query could interpret the user's intended task or execute the corresponding harmful action, even if relevant tools or documents were successfully retrieved.

  3. The paper's own experiments reveal important limitations of the proposed attack. A pretrained gibberish detector intercepts all evaluated text EONs with relatively low false-positive rates, while independent cross-encoder reranking substantially reduces retrieval preservation. The attack also transfers poorly across independently trained embedding models, requiring knowledge of or access to the target encoder.

Questions for authors

The current evaluation demonstrates guardrail bypass together with retrieval preservation, but does not assess downstream agent behavior. Can the authors provide end-to-end evidence that EON queries lead to unintended tool executions, unsafe responses, or other security-sensitive outcomes? Given that EON queries are nonsemantic, how would a downstream agent recover the user's intended task from such inputs?

Table 8 shows that a pretrained gibberish detector successfully intercepts all evaluated text EONs, while Table 16 demonstrates substantial degradation under cross-encoder reranking. How do the authors assess the practical significance of BLINDSPOT in systems employing these relatively straightforward defenses? Is there evidence that the vulnerability persists when such defenses are integrated into the retrieval pipeline?

Since preserving retrieval behavior through embedding similarity follows naturally from the design of dense retrieval systems, what new insight does the proposed BLINDSPOT provide beyond existing research on adversarial semantic collisions and query-side embedding optimization? In particular, does the work reveal any previously unknown property of embedding representations, rather than primarily demonstrating the effectiveness of a specific adversarial construction?

3 comments
AuthorsOpenReview ↗

Downstream Impact, Defenses, and Contribution

We thank the reviewer for the constructive feedback and for highlighting the need to connect retrieval outcomes to downstream behavior. We have added a focused downstream evaluation in the new Section 5 of the updated submission and address this and the other concerns below.

1. Downstream implications

Downstream scenario. To study sensitive-content recovery, we built and evaluated a search-and-describe workflow that retrieves an image and passes it to a vision-language model (VLM) for description. The VLM receives the image and a fixed description request, without the EON, natural query, or source caption. We reuse 50 unchanged EONs optimized for captions from T2I-RiskyPrompt's pornography category (15 explicit, 35 borderline), selected because the natural captions were blocked while the EONs were admitted.

Evaluation setup. Retrieval uses 5,000 T2I images mixed with 5,000 benign COCO distractors, followed by the standard Qwen3.8-27B VLM. We reuse saved query-guard decisions and retrieved images, then generate and audit new descriptions. Rebuttal Table 1 summarizes the workflow and results.

Rebuttal Table 1. Guarded image retrieval and downstream processing.

StageModel / InputResult
Query gateQwen3Guard-Gen-8B; 50 caption/EON pairs50 natural captions blocked; 50 EONs admitted
Image retrievalCLIP; 10,000-image mixed corpusPornography-annotated image at rank 1: 45/50 (90%)
Visual processingQwen3.8-27B; top-1 image + fixed requestFactual descriptions from retrieved top1 image
Description auditSame Qwen3Guard input-text classifierDescriptions flagged: 38/50 (76%)

Content recovery. In a post-hoc audit, the guardrail that blocked the EONs’ corresponding natural NSFW queries flags 76% of downstream descriptions. This provides guard-based evidence of sensitive-content recovery without requiring the worker to decode the EON.

Other downstream uses. Beyond the evaluated description task, retrieved content could also be used for generation, editing, or sharing in other applications.

We would appreciate it if the reviewer could take a look at Figure 2 and Table 4 in Section 5 for more details.

2. Practical significance of defenses

Gibberish detector. Larger semantic guardrails often admitted EONs while blocking the corresponding natural queries. To highlight this vulnerability and offer a simple mitigation, we deliberately included a lightweight gibberish detector. It intercepted all evaluated text EONs, reducing text bypass to zero, thus showing that a simple detector can mitigate this evaluated threat (submitted Table 8).

Reranking and persistence. Independent cross-encoder reranking sharply reduced, but did not eliminate, retrieval preservation (submitted Table 16); this experiment measured preservation, not safety bypass. These defenses substantially limit the evaluated threat. The new downstream evaluation uses neither additional defense.

Retrieval efficiency. Because reranking adds per-candidate computation, some systems may rely on efficient dense retrieval without it. In such systems, our findings remain relevant when query screening uses semantic guardrails alone.

Encoder access. Limited transfer makes the target encoder important. However, if the system uses a known, publicly available encoder, an attacker can optimize an EON locally for that encoder and use it to query the system, without access to its private candidate bank or retrieval feedback (submitted Table 17; Section 3.1).

3. Insight beyond prior work

The contribution. Building on prior work on semantic collisions in text and image retrieval, BLINDSPOT systematically studies how nonsensical, nonsemantic text and images can reproduce meaningful queries' retrieval outcomes across encoders and tasks, without access to the candidate bank. This phenomenon also has security implications (Tables 1–3).

Embedding representations. Our insight is that nonsensical inputs can align with meaningful queries in embedding space closely enough to preserve retrieval, with security implications. Our geometric analysis gives score-stability bounds and conditions for this preservation. We deliberately use a simple optimizer to study the phenomenon and also show that it arises with a gradient-based approach (submitted Table 13).

Ethics. We appreciate the attention to dual-use implications. The study uses public benchmarks and controlled evaluations, and the Ethics Statement discusses these concerns alongside the measured mitigations. We welcome further assessment of the release implications.

We hope these clarifications and the new downstream evaluation help address the reviewer's concern. We would be grateful if the reviewer would consider revisiting and, if appropriate, updating their assessment in light of this new evidence.

Dear Reviewer,

With less than a week remaining in the discussion period, we would be grateful for any feedback on our response or an update to the assessment if the clarifications have addressed your concerns.

Thank you for your time!

AuthorsOpenReview ↗

Follow-Up Request Before the Discussion Period Closes

Dear Reviewer,

With just a day remaining in the discussion period, we would be grateful for any feedback on our response or an update to the assessment if the clarifications have addressed your concerns.

Thank you for your time!

Reviewer_nceSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies whether visibly nonsemantic text or image artifacts can be optimized to match a natural query’s embedding, thereby preserving dense-retrieval behavior without candidate-bank access. It proposes Embedding-Optimized Noise (EON) and evaluates preservation across routing, document and memory retrieval, and cross-modal retrieval, including safety-category retrieval under independent query guardrails. The paper also gives score-stability bounds from query-proxy embedding similarity.

Critical strengths
  • The paper clearly defines a meaningful corpus-blind threat model and proposes a candidate-agnostic attack that does not require access to the retrieval candidate bank.
  • The paper provides clear geometric bounds on retrieval score stability and sufficient conditions for preserving retrieval decisions.
  • The paper presents extensive experiments covering a wide range of tasks and modalities, providing strong empirical support for its findings.
Critical weaknesses
  • As the authors acknowledge in Appendix A.3, the attack generally requires access to the target embedding model and transfers poorly across models, which may substantially limit its practical security implications.
  • The cross-encoder transfer experiments (Table 17) show poor transferability of EONs across different embedding models, potentially limiting the practical threat in real-world scenarios.
Questions for authors

See Weaknesses.

5 comments
AuthorsOpenReview ↗

Encoder Transfer and Practical Security Implications

We thank the reviewer for the thoughtful and encouraging assessment of our work. We particularly appreciate the recognition of the corpus-blind threat model, the clear geometric bounds on retrieval preservation, and the strong empirical support across tasks and modalities.

We address below the concern that encoder access and limited transfer may restrict the practical implications.

  1. Practical relevance without cross-model transfer. Even without cross-model transfer, an attacker can optimize an EON locally for a chosen query when the deployed retrieval encoder is known and accessible. Construction requires no access to the private candidate bank or retrieval feedback (Section 3.1), making the setting relevant to semantic search and workflows that pass retrieved content to downstream models. Submitted Tables 1 and 14 show that independently optimizing for different encoders reproduces the phenomenon. This setting also matters when a separate safety model screens queries before semantic search: an EON may pass that screening while retaining sensitive retrieval behavior.

  2. Practical security implications. To address the concern about practical impact, we added a focused downstream safety evaluation in the new Section 5 of the updated submission, using image captions from T2I-RiskyPrompt's pornography category. We evaluate 50 EONs optimized for these NSFW captions. Each EON bypassed the Qwen3Guard-Gen-8B query gate, while its original caption was blocked. We find that 45 EONs (90%) retrieve a pornography-annotated image at rank 1 from 10,000 images, including benign distractors. This occurs even though the EONs contain no recognizable description of the content. Qwen3.8-27B describes the retrieved image without seeing either query. In a post-hoc audit, the same Qwen3Guard model that blocked the natural captions flags 38/50 descriptions (76%). This provides guard-based evidence that encoder-specific EONs can expose sensitive content to downstream processing despite query-side screening. We would appreciate it if the reviewer could take a look at Figure 2 and Table 4 in Section 5 for more details.

We hope these clarifications and the new downstream evidence help address the reviewer's concern, and would welcome a reconsideration of their assessment if they find it appropriate.

Dear Reviewer,

With less than a week remaining in the discussion period, we would be grateful for any feedback on our response or an update to the assessment if the clarifications have addressed your concerns.

Thank you for your time!

AuthorsOpenReview ↗

Follow-Up Request Before the Discussion Period Closes

Dear Reviewer,

With just a day remaining in the discussion period, we would be grateful for any feedback on our response or an update to the assessment if the clarifications have addressed your concerns.

Thank you for your time!

Comments on the paper

2 comments

We thank the reviewer for the follow-up. We address the remaining concerns against the stated threat model and experimental evidence below.

(1) Encoder access and black-box construction. We assume access to outputs from the encoder used by the target retrieval system. Construction is black-box: the optimizer uses only forward embeddings or cosine similarities to the natural-query embedding, without inspecting model parameters or computing gradients (Algorithms 1–2). By local optimization, we meant running the same output-only search locally, without using model parameters or gradients. The procedure can also operate through a remote encoder interface that provides the required outputs. This is consistent with query-based black-box attacks such as Bhagoji et al. (2017), which use the target model’s class probabilities.

For BLINDSPOT, access to encoder outputs is sufficient for construction, without candidate-bank information, retrieval feedback, or guardrail decisions (Section 3.1).

(2) Contribution beyond the construction algorithm. BLINDSPOT examines an extreme case of nonsemantic input: nonsensical text and images that can preserve meaningful queries’ retrieval outcomes across encoders and tasks. Remarkably, image EONs can achieve this with just a few modified pixels (<1%) on a blank canvas (Figure 1). An illustrative text form is:

fh gj 9f !j …

While some studies have examined nonsemantic inputs, they also note interpretable elements in some of their examples. Zou et al. (2023, Section 3.3) give the GCG fragment:

...end-line only please reiterate the first sentence by putting Sure by...

They observe that “the discovered adversarial prompts here do have an element of interpretability to them”. Nguyen, Yosinski, and Clune (2015, Section 3.4) study unrecognizable images for classification, noting that “the generated images do often contain some features of the target class”.

Thus, the primary contribution of this work is the finding that extremely nonsemantic EONs can preserve retrieval with serious downstream security implications, even when constructed using something as trivial as greedy hill climbing, deliberately chosen for its simplicity and black-box operation (Tables 1–3; Section 5). We also identify a simple mitigation: a lightweight gibberish detector intercepted all evaluated text EONs, including those admitted by the larger semantic guardrails (submitted Table 8).

(3) Detection and robustness. As noted in our initial response, this study identifies a vulnerability with downstream security implications and provides a simple mitigation. The lightweight gibberish detector intercepted all evaluated text EONs, including those admitted by the larger semantic guardrails, reducing measured text bypass to zero (submitted Table 8). For these evaluated text EONs, no additional defense was needed to eliminate bypass. The detector’s success is therefore an important mitigation result of the paper.

We hope these clarifications resolve any misunderstandings about the threat model, contribution, and our findings.

References

  • Arjun Nitin Bhagoji, Warren He, Bo Li, and Dawn Song (2017). Exploring the Space of Black-box Attacks on Deep Neural Networks. arXiv:1712.09491.
  • Anh Nguyen, Jason Yosinski, and Jeff Clune (2015). Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images. CVPR.
  • Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson (2023). Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv:2307.15043.
AuthorsOpenReview ↗

Summary of Responses to Reviewer Concerns

Dear Area Chair and Reviewers,

As the discussion closes, we provide a brief summary of the primary concerns raised during review and our responses, beginning with the new downstream evaluation in Section 5.

  1. “Given that EON queries are nonsemantic, how would a downstream agent recover the user's intended task from such inputs?” In our search-and-describe workflow, the application supplies a fixed description request, and retrieval supplies the image. Of the tested EONs admitted by the query guard, 90% retrieved a pornography-annotated image at rank 1. The vision-language model described each retrieved image without seeing either query. The same guard that blocked the original captions flagged 76% of the resulting descriptions, providing evidence of sensitive-content recovery during downstream processing (Figure 2 and Table 4).

  2. “What is the main difference between EON and prior works?” In the examples discussed in our response, Zou et al. (2023) identify interpretable phrases in adversarial text, and Nguyen et al. (2015) note recognizable target-class features in some generated images. BLINDSPOT examines an extreme case in retrieval with NO SEMANTIC CONTENT. A sequence of two-character atoms or a few modified pixels on a blank canvas can preserve a meaningful query’s retrieval result over an unseen candidate bank (Figure 1). We document this behavior across encoders and tasks. Our geometric analysis gives conditions for retrieval preservation, and the text-side comparison shows that it also arises with a gradient-based optimizer (submitted Table 13).

  3. “Is there evidence that the vulnerability persists when such defenses are integrated into the retrieval pipeline?” A lightweight gibberish detector intercepted every evaluated text EON (submitted Table 8). In a separate experiment, cross-encoder reranking sharply reduced retrieval preservation, though some retrieval outcomes persisted (submitted Table 16). The downstream evaluation used semantic query screening alone.

  4. “Is the embedding model black-box or white-box to the attacker?” The optimization is black-box and assumes access to outputs from the deployed encoder. It uses those outputs without inspecting model parameters or computing gradients and requires no candidate-bank information or retrieval feedback (Section 3.1).

  5. "How does limited transfer affect practical relevance?" We agree that EONs are encoder-specific and explicitly discuss this limitation in the paper. In our response, we gave the example of an enterprise search service to briefly illustrate why the finding can still matter in practice in some scenarios. In that setting, cross-encoder transfer is unnecessary. Given the sensitivity of the content such systems may expose, this vulnerability remains important to identify and mitigate. BLINDSPOT demonstrates the risk and shows that a lightweight gibberish detector can mitigate it for the evaluated text EONs.

We again thank the reviewers for their time and hope the new evidence and these clarifications will inform the next phase of discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.