- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Embedding-Optimized Noise (EON), aiming to create non-semantic image/text that have a high embedding similarity of a natural query. It can retrieval dangerous items to agent contexts without being detected by input guardrails.
Critical strengths
The paper is very clear and easy to follow. The motivation and idea of finding blindspots of embedding models is very interesting.
Critical weaknesses
(1) The contribution of this paper is very limited. The main idea and method of this paper is very general and similar to prior adversarial attacks [1,2] targeting LLM and embedding models. While many related works on adversarial attacks are not cited in this paper. No baseline comparison is conducted in experiments.
(2) The robustness evaluation of the proposed method is missing. Since the generated EON is very obvious garbled text/image, it should be easily detected and removed by many other ways except safety guardrails (e.g. [3,4]).
References
[1] Explaining and Harnessing Adversarial Examples. ICLR 2015
[2] Universal and Transferable Adversarial Attacks on Aligned Language Models. arxiv 2307.15043
[3] Detecting Language Model Attacks with Perplexity. arXiv:2308.14132
[4] Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense. NeurIPS'23
Questions for authors
(1) What is the main difference between EON and prior works on adversarial attacks? Could the authors further clarify their contribution?
(2) Could the authors compare their method to prior works on adversarial attacks? What is the advantage of the proposed method?
(3) How robust and stealth is EON? Can it survive detection and purification process such as filtering, paraphrasing, image cropping, diffusion-based image reconstruction, and embedding model-based classification?
(4) What is the threat model of this paper? Is the embedding model black-box or white-box to the attacker?
Black-Box Construction, Contribution, and Defenses
We thank the reviewer for the thoughtful feedback and the appreciation of our motivation. We address the questions below, beginning with the threat model.
4. Threat model
Black-box or white-box? The encoder is black-box to the attacker. The construction is gradient-free and uses only forward embeddings, without access to gradients or model parameters. We have explicitly mentioned this in the Introduction and Section 3.2, with further details in Appendix A.4.
Threat model. We describe the threat model in detail in Section 3.1: optimization uses no candidate identities, embeddings, rankings, retrieval feedback, or guardrail decisions. An accessible local copy of the target encoder allows offline construction followed by submission of the EON.
1. Contribution and prior work
Difference from prior adversarial attacks. Prior work has explored adversarial inputs, semantic collisions, and image queries designed to preserve retrieval results. Building on these findings, BLINDSPOT systematically studies the remarkable ability of nonsensical, nonsemantic text and images to reproduce meaningful queries' retrieval outcomes across encoders and tasks, without access to the candidate bank. For images, this includes queries constructed by changing a few pixels on a blank canvas.
Contribution. The central finding is that a nonsensical proxy can preserve the retrieval behavior of a meaningful query. For example, an unreadable token sequence optimized to approximate the embedding of “a red car” can retrieve relevant car images from a collection unseen during optimization. We further formalize retrieval preservation through geometric bounds and study its security implications: semantic guards can admit a nonsensical proxy while blocking its natural query, allowing sensitive content to reach downstream processing.
2. Comparison and advantage
We deliberately use a simple optimizer and compare it with a white-box, gradient-based approach under the same representation objective, finding that the phenomenon arises with both (submitted Table 13). Random-noise and permutation controls provide further context (submitted Tables 18–19). The main takeaway is that nonsensical inputs can align with meaningful queries in embedding space closely enough to preserve retrieval, with security implications. Our method requires no access to the candidate bank and uses only encoder outputs, making it applicable without the gradient access required by white-box attacks.
3. Detection and robustness
Detection and filtering. Larger semantic guardrails often admitted EONs while blocking the corresponding natural queries. To highlight this vulnerability and offer a simple mitigation, we deliberately included a lightweight gibberish detector. It intercepted all evaluated text EONs, reducing text bypass to zero, with 1.0–4.4% natural-query false positives (submitted Table 8). Thus, a simple detector was sufficient for the evaluated text EONs.
Robustness to other defenses. Tables 15–17 evaluate late interaction, reranking, and transfer, while Table 19 examines token order. Paraphrasing, cropping, and diffusion reconstruction remain outside the evaluated scope.
We hope these clarifications help address the reviewer's concerns, and would be grateful if the reviewer would consider revisiting their assessment if they find it appropriate.