acceptodds
Under review as a conference paper at ICLR 2027

PoisonSieve: Detecting Poisoned Documents in RAG through Matched Local Contrast

Abstract

Retrieval-augmented generation uses an external corpus as inference-time evidence, allowing an attacker to promote a false answer by injecting a handful of documents. Detection must distinguish this manipulation from ordinary relevance without knowing which queries or documents are targeted. Existing detectors use text irregularity, candidate consensus, or corpus-level graph structure, whose reliability varies with the attack and local context. We present PoisonSieve, which constructs a reference matched to each detection scope. At query time, PoisonSieve-Query (PSQ) compares generation candidates with the lower-ranked tail of the same retrieval, exposing answer-token concentration and carrier–payload seams. At corpus time, PoisonSieve-Graph (PSG) compares each document's strongest semantic relations with its own neighborhood floor to measure coordinated density, complemented by script integrity. Neither requires poison labels, a trusted corpus, or training. Across three QA datasets, three dense retrievers, and six poisoning constructions, PSQ reaches 95.2% AUROC and detects 82.2% of poison at a 5% clean-removal budget, against 81.1% and 52.5% for the strongest query-time baseline; PSG reaches 93.3% and 79.8% against 79.4% and 37.6% for the strongest corpus-time baseline, with a 79.6% versus 1.4% detection rate on camouflaged injections. Joint deployment cuts attack success from 67.4% to 16.1% while retaining unpoisoned-retrieval F1 at 41.0%, compared with 42.1% without filtering.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
retrieval-augmented generation, knowledge poisoning, poisoned document detection, RAG security, robust retrieval, corpus inspection, machine learning security

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_cMPGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The manuscript addresses the security vulnerability of RAG systems to corpus poisoning. An adversary who can insert only a few malicious documents into the knowledge base may cause a targeted query to retrieve those documents and consequently obtain an incorrect answer from the generator. The authors formulate this problem as a matched‑control task and propose PoisonSieve, which builds two complementary detectors: 1) PoisonSieve-Query (PSQ) operates after retrieval, comparing top-k passages against lower-ranked references. It combines four signals, answer-anchor concentration, script integrity, LM surprisal, and query-alignment changes, to identify and replace suspicious passages. 2) PoisonSieve-Graph (PSG) operates offline, detecting poisoned documents through neighborhood density contrasts in the embedding space, combined with script-integrity checks.

Critical strengths
  1. The paper proposes matched‑local‑contrast principle. By using the same query’s retrieval tail (or a document’s own neighbourhood floor) as a reference, the detectors avoid reliance on external clean corpora or labeled poison data. This addresses a key gap in prior work that uses global statistics or learned models for normality.
  2. The paper offers dual‑stage complementary defense. PSQ and PSG target orthogonal attack surfaces (query‑time promotion vs. corpus‑level coordination). Their serial combination yields a reduction in ASR while keeping clean‑document removal under the prescribed 5 % budget.
Critical weaknesses
  1. The paper uses a restrictive threat model. The core assumption is that an attacker can inject only less than 5 documents and cannot contaminate the retrieval tail used as reference. The “tail‑seeding” experiment already shows that when this assumption is violated, PSQ’s detection collapses to near‑random performance. Realistic systems often allow bulk uploads or automated crawlers, making the assumption unrealistic.
  2. The proposed method also heavily depends on coordinated density for PSG. PSG assumes that malicious documents will cluster together in the embedding space, so their strongest edges are much higher than the similarity of the “neighbourhood floor.” When an attacker deliberately spreads the poisoned passages, this gap disappears. Consequently the density contrast score becomes near zero and PSG can no longer flag the poison.
  3. Due to above mentioned weakness, PSG may suffer from high false positive rates under some topic surges. When a genuine “hot topic” event suddenly adds many highly similar, clean documents (e.g., breaking news articles) to the corpus, this same pattern will appear and PSG may erroneously label those normal passages as poisoned. The paper reports only an overall 5 % clean removal budget on a static snapshot and does not evaluate how the detector behaves during such abrupt content spikes. Without empirical evidence of false positive rates under realistic, rapidly evolving corpora, the claim that PSG is safe for practical deployment remains unsubstantiated.
  4. The paper lacks systematic hyper‑parameter and scalability analysis. Key parameters (k, n, etc) are fixed across all experiments without sensitivity studies or guidance for new corpora.
  5. The Evaluation is limited to English QA. All datasets are English open‑domain question answering. The script‑integrity branch and the token‑level answer‑anchor statistic may behave very differently on multilingual or code‑mixed corpora, yet no experiments are presented to support the claim of language‑agnostic applicability.
Questions for authors
  1. Larger injection volumes: How does PSQ perform when more than k poisoned documents are inserted (e.g., 8–10 per query)? Have you measured detection degradation and possible mitigation strategies such as increasing retrieval depth or multi‑round refilling?
  2. Are there any mitigation strategies to solve the dependence on coordinated density for PSG?
  3. Could you provide more systematic analysis on selection of some parameters (k, n, τ_e, τ_l, η_Q and α_G)?
  4. Scalability of PSG: What are the time and memory complexities for building the nearest‑neighbour graph on a corpus of order 10⁸ documents?
  5. Cross‑lingual/generalization: Have you tried PoisonSieve on non‑English QA datasets or on multimodal RAG pipelines (e.g., vision‑language retrieval)? If not, what modifications would be required for the script‑integrity and answer‑anchor components to remain effective?
Reviewer_CsszOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
1
Read review
Paper summary

The problem of the exisiting RAG system is that the knowledge of the RAG knowledge base can be written from outside, so when the attacker inject some poisoned document, the generator will generate wrong answer that follow the attacker's intent, and the hardest part for defender is that they don't know the query to be attack, the payload, and the attack method. The author propose RAGSieve, which is a training-free poisoned document detect method that does not need any posioned document label and trusted clean document. The two detect point are RSQ and RSG, where RSQ detect the poisoned message at the query time by comparing the top-5 candidates with the lower-ranked documents (rank 6 to 20) of the same retrieval, and it sums up four scores for each candidate. RSG detects at the corpus time by comparing the similarity of each document’s 4 closaest neighbor with its 16th neighbor as a local floor, then it flags the top 5% most suspicious documents together with all documents that mix multiple scripts. The experiments cover 3 QA datasets, 3 dense retrievers and 6 poisoning attacks, and the evaluation shows a good perfomrance of the defense method comparing to other baselines.

Critical strengths

One major strength is that RAGSieve is training-free and does not need any poison label or trusted clean corpus, which makes it easy to deploy in a real RAG system without massive train, and it cover both the query time and the corpus time, so one detector can still catch the poison when the other one is evaded.

Also, the paper does not only look at the attack success rate, it also reports how many clean documents are removed and the answer quality on unpoisoned retrieval, which is important because a defense can easily get a low ASR by removing everything, and the adaptive attack part is quite honest, the authors directly show that tail seeding drops the RSQ detection to 0 to 3% and near-duplicate injection drops the RSG detection to 0 to 2.2%, and they also discuss the remaining risk on MS MARCO.

Critical weaknesses

Although the method is easy to deploy and show some good result, I still highly concern about the setup and coverage of the evaluation of the method. The following are two major concern.

1. The system budget of the query-time RSQ is not clearly evaluated, and I highly concern that the design is not valid in a more realistic deployment scenario.

As what the paper says, RSQ runs on every incoming query, it needs the top-20 retrieval results instead of the top-5, and it calls two extra models (Qwen3-0.6B and BERT) on every candidate. In Table B4, the detector is only measured alone with the models kept in memory, and the numbers are already not small: 447 ms mean latency, 542 ms P95, 3.87 GiB memory, and only 2.24 queries/s throughput on one RTX 5090. This means a service with 100 queries per second would need about 45 GPUs only for RSQ, while RAGSentinel can handle 43 queries per second.

So the problem is that in practice, expensive checks like PPL or semantic tests are usually done offline instead of on the query path, because they add latency to every user request, so although the paper shows that the answer quality on unpoisoned retrieval is kept, it does not report the end-to-end latency of the whole RAG pipeline with and without RSQ, or how RSQ works under concurrent requests. I would like to see this system-level evaluation, otherwise it is hard to say RSQ is practical.

2. The evaluation does not cover fluent generation-based poison and adaptive attacks well, and the attacks, retrievers and LLM are limited.

RSQ's script and alignment terms all look for surface artifact, but the problem is that fluent text does not have these artifacts, so on fluent attacks only the answer-anchor term is really working, and the paper's own results already show this, for example on CPA-RAG, the fluent generation-based attack, RSQ only detects 47.3% of the poison and the ASR is still around 72 percent after filtering, and also the diluted answer variant drops RSQ. From the attacker's side, a simple adaptation is to write several diverse rewrites of an existing ground-truth passage that carry the false answer, so if the rewrites keep high word overlap with the original passage, RSG removes their edges as near-duplicates, and if the rewrites are lexically diverse, they form a dense group around the real passage, and RSG may even remove the real evidence together with the poison. The adaptive study also only evades one detector at a time, and a combination like tail seeding plus near duplicate injection is not tested.

Besides, the coverege of the evaluation is limited. Four of the six attacks (PR-W, CEM-C, CEM-D and CamoDocs) contain multillingual token artifacts, so only PR-B and CPA-RAG are really fluent. The retrievers (BGE-M3, E5-large-v2 and all-MiniLM-L6-v2) are relatively old, and Qwen3-Embedding is only used as the attacker's proxy but not as a victim retriever. Also, the same deepseek-v4-flash is used to write the payloads, answer the questions and judge the attack success, so it is not clear whether the results transfer to other LLMs.

Questions for authors

Please address the concern raised in the weakness section.

Reviewer_JLuDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PoisonSieve, a training-free defense against RAG document poisoning. Its key contribution is using matched local references for detection at both query and corpus time. Experiments show that the two stages provide complementary protection, substantially reducing attack success with limited degradation in clean QA performance.

Critical strengths
  1. Well-motivated reference design. Query-tail and neighborhood-based comparisons address concrete limitations of candidate consensus and global density thresholds, without requiring trusted clean data.
  2. Strong end-to-end results. Joint filtering reduces ASR from 67.4% to 16.1%, outperforming the staged baseline (26.1%) while retaining most clean QA performance.
  3. Informative empirical analysis. Component ablations and targeted evasions demonstrate where the two stages provide complementary protection and expose their failure modes.
Critical weaknesses
  1. Adaptive robustness remains insufficiently established. Attacks target individual branches rather than jointly optimizing against the complete defense. Even these variants leave 43–60% ASR on MS MARCO under joint filtering.
  2. Some gains depend heavily on attack artifacts. For CamoDocs on BGE-M3, script integrity detects 78.9% of poison, versus 0.7% for graph contrast. This narrows the evidence that local graph structure handles camouflaged attacks and raises questions about multilingual applicability.
  3. The score fusion lacks convincing justification. PSQ’s integrity term contributes 0.2 even when all documents have zero script mixing. Removing it improves average detection under matched or calibrated clean-removal settings (Tables B3–B4), suggesting that fixed-threshold ablations partly reflect threshold effects rather than additional discriminative evidence.
Questions for authors
  1. Correct answers may naturally concentrate in highly relevant top-ranked documents while being absent from the retrieval tail. The paper does not sufficiently establish whether PSQ can distinguish this benign agreement from coordinated poisoning.
  2. Lower-ranked documents are typically less relevant, so PSQ’s contrast may reflect ordinary relevance differences rather than malicious manipulation.
  3. Legitimate documents covering the same event or newly emerging topic may exhibit strong local connectivity. PSG’s ability to distinguish these clusters from malicious coordination is insufficiently evaluated.
  4. Attackers can contaminate the retrieval tail to suppress PSQ’s contrast. The tail-seeding experiments demonstrate this vulnerability, while protection from PSG remains corpus-dependent.
  5. PSQ scores the initial candidates but fills vacancies with unscored lower-ranked documents, allowing poisoned evidence to reach the generator through the refill step.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.