acceptodds
Withdrawn from ICLR 2027

Purify the Source: Soft External Knowledge Detoxification Against Retrieval Pollution

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

In order to address the challenges of hallucination and information missing in Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) has become a standard paradigm. However, RAG systems face a critical threat, that is, the pollution within external corpora, which underscores the urgent need for robust RAG reinforcement. While existing strategies often resort to hard filtering to exclude polluted documents, which is difficult to properly eliminate polluted content and may discard valuable information, undermining overall system effectiveness. In this paper, we propose SeKe, a novel soft external knowledge detoxification approach designed to mitigate retrieval pollution. Inspired by classic convolution and pooling mechanisms, SeKe employs knowledge convolution alongside generative semantic pooling operations, which can progressively aggregate content within retrieved documents by leveraging the distributional differences between polluted and benign documents, thereby reducing the overall density and concentration of pollution and lowering the susceptibility of LLMs to polluted information. Experimental results show that SeKe can achieve promising performance in almost all 15 investigated pollution settings. Furthermore, SeKe can seamlessly integrate with other RAG frameworks, yielding an average absolute accuracy increase of 14% and reducing the misleading rate by 19%, proving its effectiveness and compatibility.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
retrieval-augmented generation, large language model

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_vTPKOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SeKe, a soft knowledge detoxification framework for robust RAG under retrieval pollution. SeKe expands the retrieved documents, groups semantically related documents, and applies generative semantic pooling to construct a detoxified context. Experiments across multiple datasets, attacks, retrievers, LLMs, and RAG frameworks show consistent improvements in accuracy and misleading rate.

Critical strengths
  • The paper addresses a practical and important robustness problem in RAG systems.
  • The evaluation is broad, covering multiple datasets, attacks, retrievers, LLMs, and RAG frameworks.
  • SeKe can be integrated with existing robust RAG methods and generally improves their performance.
Critical weaknesses
  1. The central assumption of SeKe, namely that polluted documents exhibit stronger semantic cohesion than benign documents, is not directly validated. The method groups documents based on semantic-neighbor overlap under the assumption that polluted documents are more mutually cohesive. However, the paper does not provide quantitative analyses of the overlap distributions or separability among benign-benign, polluted-polluted, and benign-polluted document pairs. As a result, it is unclear how reliably this signal distinguishes polluted documents from benign ones.

  2. It is not sufficiently isolated whether the performance gains of SeKe actually come from the proposed detoxification mechanism. Generative Semantic Pooling combines document grouping, query-conditioned summarization, and internal memory elicitation. Knowledge Expansion also increases the amount of available information by retrieving additional documents. Therefore, it remains unclear whether the gains arise from the proposed detoxification mechanism, stronger summarization/reasoning, or simply access to more external knowledge.

  3. Important control baselines are missing for validating the necessity of the proposed mechanism. For example, the paper should compare against a RAG system that directly uses the same expanded top-k+t documents, as well as one that simply summarizes the expanded context. Without such controls, it is difficult to determine whether the more complex Knowledge Convolution and grouping procedures are actually necessary.

  4. The fairness of the comparison in terms of retrieval budget is unclear. SeKe uses the original k=5 retrieved documents and additionally retrieves t=5 documents, resulting in access to 10 documents in total. It is unclear whether the competing baselines are given the same retrieval budget. Therefore, the current results do not fully separate the contribution of the proposed method from the potential benefit of simply retrieving more documents.

  5. The context detoxification evaluation in Figure 6 may be confounded by context length. Figure 6 counts the number of occurrences of polluted answers in the context, but SeKe explicitly compresses multiple documents into shorter representations. As a result, the raw occurrence count may naturally decrease simply because the resulting context is shorter.

  6. Several important details required for evaluation and reproducibility are insufficiently specified. The exact procedures used to determine ACC and MR are unclear, and the paper does not sufficiently explain how the presence of a polluted answer is detected. In addition, the actual value and sensitivity of the key Generative Semantic Pooling hyperparameter τ\tau are not reported. The paper also lacks an analysis of computational overhead introduced by the additional retrieval and LLM generation steps.

  7. The document grouping procedure is not clearly defined. The text states that two documents are assigned to the same group when the overlap between their corresponding filtered yields FiF_i and FjF_j is at least 0.5. However, Eq. 7 instead computes the overlap between a remaining filtered yield FrF_r and a candidate group GcG_c. Moreover, it is unclear whether GcG_c denotes a set of retrieved documents or a set constructed from filtered yields. This inconsistency makes the grouping algorithm difficult to interpret and reproduce.

  8. To better characterize robustness against retrieval pollution, the paper would benefit from explicitly comparing clean RAG, polluted RAG, and polluted RAG with SeKe. The current evaluation mainly compares different defense methods, making it difficult to see how much performance is degraded by pollution itself and how much of that degradation is recovered by SeKe.

  9. Since the proposed method primarily operates on retrieval and context manipulation, retrieval- or context-level metrics would better align the evaluation with the claimed mechanism.

  10. Figures 4 and 5 are difficult to interpret because many experimental settings are shown simultaneously in radar plots, and small differences, particularly in MR, are hard to distinguish. In addition, TrustRAG appears to achieve low MR in some settings without correspondingly high ACC. Further analysis of this behavior would be helpful.

  11. Finally, there are several grammatical issues and typographical errors throughout the paper that should be corrected.

Questions for authors

Please see the weaknesses.

Reviewer_qWsAOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the issue of external knowledge contamination (Retrieval Pollution) in the Retrieval-Augmented Generation (RAG) system, proposing a soft external knowledge detoxification method named SeKe. The authors point out that existing defense methods mostly adopt hard filtering strategies, assuming that contaminated documents can be completely removed. However, this is often limited by the ability of the backbone model to identify contamination, which can easily lead to the deletion of benign knowledge or the retention of misleading content. To solve this problem, SeKe consists of two core stages:

  1. Knowledge Expansion (Knowledge Expansion): Extends the retrieval scope from top-k to top-k+t, initially diluting the contamination density by introducing more documents;
  2. Quasi-Convolutional Detoxification (Quasi-Convolutional Detoxification): Draws inspiration from CNN, first uses the retrieval semantics of each document as a "convolution filter" for content filtering (extracting filtered yield), then groups the documents based on the overlap between yields (Contextual Grouping), and finally compresses each group of documents into short summaries constrained by length and question relevance through generative semantic pooling, and introduces internal knowledge of the LLM to form the final detoxified context. The authors constructed 15 experimental scenarios by combining three contamination attack methods (PoisonedRAG, HijackRAG, AuthChain) on three datasets (NQ, HotpotQA, and MSMARCO), and evaluated them on three retrievers and five LLMs. The experimental results show that SeKe not only can be used as an enhancement component for VanillaRAG but also can be used as a plugin to improve the performance of other existing RAG frameworks (such as RobustRAG, InstructRAG, etc.), achieving an average absolute improvement of 14% in accuracy (ACC) and an absolute decrease of 19% in misleading rate (MR).
Critical strengths
  1. The problem motivation and perspective are reasonable: The author keenly pointed out the limitations of the existing hard filtering strategies in RAG defense, namely, "incomplete pollution localization can lead to the deletion of benign knowledge or residual pollution", and proposed the idea of shifting from "hard removal" to "soft detoxification" (retaining semantic information while destroying misleading features), which is somewhat enlightening and of practical significance.
  2. The method design has certain potential for modularization and pluginization: SeKe is designed as an independent external knowledge processing module (plugin), not relying on the internal structure of a specific RAG framework. The experiments show that it can be seamlessly integrated into different frameworks such as RobustRAG and InstructRAG, demonstrating good generalization and scalability.
  3. The experimental evaluation matrix is relatively comprehensive: The author cross-used three different pollution generation methods (including single-document and multi-document modes) on three mainstream open-domain QA datasets, and covered three mainstream retrievers (MiniLM, MPNet, Contriever) and five different LLM architectures (Llama3.2, Gemma3, Mistral, Qwen3, Phi4), totaling 15 pollution scenarios. This multi-dimensional experimental setup helps to preliminarily observe the performance of the method under different configurations.
  4. Ablation experiments preliminarily verified the effectiveness of the extension module: The RQ3 ablation experiment on Knowledge Expansion (Figure 8) indicates that simple convolutional detoxification is effective, but combining knowledge expansion can further improve ACC and reduce MR, providing an empirical reference for subsequent research on how to alleviate pollution density by adjusting the retrieval boundary.
Critical weaknesses

Writing

W1: Section 3.1 is excessively lengthy and has an unreasonable structure. The 3.1 section merely transforms the original top kk retrieval into top k+tk+t (an extended retrieval), with the core logic being very simple, yet it occupies over 20 lines and is presented as a separate section, appearing overly complicated and lacking balance. (Note: This also implies potential flaws in the method's logic, as seen in M1)

W2: The method may be suspected of excessive packaging. The paper attempts to appear "high-end" by cramming a large number of complex mathematical formulas (such as the conditional probability constraints and optimization formulas for generative semantic pooling) into the methodology section. However, from the actual code implementation perspective, these steps are merely ordinary Prompt instructions. The theoretical formulas are seriously disconnected from the engineering implementation, and the author is advised to simplify the mathematical expressions in the revision with a realistic approach.

Methodology

M1: Knowledge Expansion may introduce the risk of introducing more contamination. Given that the document library already has contamination, blindly extending top kk to top k+tk+t is likely to recall more incorrect/polluted documents, which may not be as "reducing contamination density" as the author claims.

M2: The core assumption lacks empirical verification. The core premise of Section 3.2 is that "polluted documents have higher cohesion (cohesion) than benign documents", but the author does not provide any quantitative statistics or empirical data to verify this assumption. They merely cited relevant literature.

M3: The logical consistency of knowledge convolution (Knowledge Convolution) is questionable. The author claims that knowledge convolution can extract features that are distinguishable between benign and polluted documents. If this convolution module is truly effective, why is a top-k+t extension needed beforehand? Could it be that the top-k+t step makes the number of benign samples sufficient, so the convolution module barely works?

M4: The summary section has information loss and extremely high computational costs. Generative semantic pooling (summary) is a lossy compression, especially under the strict length budget τ. It is prone to losing the key evidence needed for answering multi-hop or complex questions. Additionally, this section adds a large number of additional LLM calls, resulting in significant increases in latency (Latency) and token costs, severely reducing the inference efficiency of RAG.

M5: The sensitivity analysis of the key hyperparameter τ is missing. The summary length budget τ directly determines the degree of information retention and detoxification effect, but the paper does not verify the impact of different τ values on performance.

**M6: The core assumption that "corrupted documents are more coherent" lacks empirical validation based on the dataset of this study. ** In Section 3.2.1, the author presents the core premise: "Since polluting a specific query often requires documents not only to lead to an incorrect answer but also to remain highly relevant to the question, polluted documents tend to exhibit higher cohesion compared to benign ones." Although the author cites Zhou et al. (2025)'s work as support and this paper uses the same base dataset (NQ, HotpotQA, MSMARCO), the experimental data in this paper is enhanced through data augmentation (i.e., by artificially generating a large number of polluted documents using different algorithms such as PoisonedRAG, HijackRAG, AuthChain). The adversarial samples generated by different attack algorithms are likely to have significant differences in semantic distribution, diversity, and cohesion. Therefore, directly applying the observation conclusions from other literature as the core premise of this method is not rigorous. The author must provide quantitative statistical verification (such as calculating the average cosine similarity, clustering coefficient, or cosine distance distribution between the polluted document group and the benign document group) in the 15 enhanced scenarios actually constructed in this paper to prove that the phenomenon of "polluted documents being more cohesive than benign documents" does indeed exist.

Experimental Design & Evaluation

E1: The computational budget is seriously unbalanced (Unfair Comparison).

  • Search Budget: SeKe retrieved k+t documents (since t=k, it actually means top-2k), while the comparison baseline (VanillaRAG, etc.) seems to only use top-k. Using twice the context to outperform one-time context, the performance improvement is likely due to "seeing more beneficial information" rather than the algorithm's merit.
  • Generation Budget: SeKe requires a large number of additional LLM calls (internal knowledge extraction, summary generation for each group, etc.). Under the condition of equal token consumption and LLM call counts, can SeKe still outperform the baseline?

E2: Experimental evaluation lacks statistical significance and reliability. Although 15 scenarios were constructed, each scenario only contained 100 samples. For LLM evaluation, the variance of a single run is extremely large. The paper did not report the results of multiple-seed (multi-seed) runs, lacked standard deviation (std), confidence intervals, and did not conduct significance tests (such as a paired t-test), making it impossible to prove whether the 14% improvement is due to the algorithm's effectiveness or random fluctuations.

E3: The core metric (MR) is defined extremely vaguely. The criteria for determining the misleading rate (Misleading Rate, MR) are not clearly stated. How is the output judged to be misleading? Is it through manual assessment, rule-based string matching, or using GPT-4 as a judge (LLM-as-a-Judge)? The paper does not explain at all.

E4: The contamination count metric in RQ2 has a "length deviation deception". In RQ2 (Figure 6), the contamination level is measured by "the occurrence frequency of contaminated answers in the prompt". This metric is extremely dependent on text length. The summaries generated by SeKe are limited to 30 words, so the absolute probability of contaminated answers appearing is naturally lower than that of the lengthy original documents. This might be a simple shortening of the text, but it cannot prove that it truly "detoxified".

Code & Reproducibility

C1: Hyperparameters are hardcoded (Hardcoding), affecting reproducibility. The paper claims that the length budget τ is "tunable", but in the actual code, τ=30 is hardcoded (Hardcoded) in the Prompt string. Although it can be adjusted by modifying the Prompt at the code execution level, a reasonable engineering implementation should extract it as a configuration file (Config) parameter and support command-line input.

C2: The core formula and code implementation are seriously inconsistent. The paper claims to have implemented the probability space optimization of formulas (9) and (10), but in the provided code, there is no code to calculate log P(s|q, G_i) or sample candidate summaries and perform argmax. The actual code merely composes a Prompt (merge_sys_prompt) to let the LLM generate directly. This is a "misalignment", belonging to a substantive deviation in method description.

Questions for authors

same to weakness

Reviewer_4GosOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SeKe, a framework that defends RAG against corpus poisoning by detoxifying retrieved context. It retrieves extra documents to dilute poisoned ones, groups documents by overlap with their retrieved semantic neighbors, then summarizes each group into a short, query-focused passage and adds its own internal knowledge. Across multiple settings, the numerical results show good performance.

Critical strengths
  1. This paper focuses on a timely threat to RAG.

  2. This work has broad evaluation on several datasets and models.

Critical weaknesses
  1. Insufficient analysis of the role of each component and how they interact. This paper proposes Knowledge Expansion, Knowledge Convolution, and Generative Semantic Pooling. However, Section 5.3 only discusses Knowledge Expansion. The distinctions from existing works on RAG poisoning detoxification are also not sufficiently discussed. The paper does not report accuracy without an attack, so the utility cost of the defense on normal traffic is unknown.

  2. Poor presentation. The radar plots in Figures 4 and 5 are confusing. The main text provides no tables of numerical results.

Questions for authors
  1. Can you report the contribution of each component separately?

  2. Since the poisoned documents are known, what is the grouping purity in the one-shot and multi-shot settings?

  3. How much accuracy does SeKe cost on benign inputs relative to vanilla RAG? Is this cost consistent across datasets, or larger on multi-hop questions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.