acceptodds
Under review as a conference paper at ICLR 2027

SIDRAG: Adaptive Defense against RAG Poisoning via Set-Level Influence Detection

Abstract

Retrieval-augmented generation (RAG) systems are highly vulnerable to poisoning attacks, and existing defenses struggle to maintain robustness across varying attack scales. We observe that successful attacks typically rely on malicious documents exerting either an overwhelmingly strong individual influence on the large language model's (LLM) predictions, or a consistent and strong aggregated influence when acting jointly. Motivated by this, we propose SIDRAG, a defense framework that adaptively mitigates poisoning attacks across different scales by analyzing document influence patterns. Specifically, SIDRAG decomposes the LLM's attention output into document-level influence vectors and constructs an attributed signed graph to capture their strengths and pairwise directional relationships. It then identifies Jointly Consistent Subsets (JCSs) within the graph, grouping documents with aligned influence patterns. This set-level modeling unifies the coordinated impact of multiple documents into a single detection unit, so that the defense does not require prior knowledge of the number of malicious documents. Finally, SIDRAG flags anomalous subsets based on their influence strength and consistency, filtering out malicious documents prior to answer generation. Extensive experiments demonstrate that SIDRAG achieves a 98.7% average exact identification rate for malicious document sets. Across five LLMs and three benchmarks, it achieves the highest average robust accuracy (RACC) and the lowest average attack success rate (ASR) among existing defenses, while incurring the lowest inference latency among defenses on every evaluated model.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Retrieval-Augmented Generation, RAG Poisoning Attacks, LLM Security, Set-Level Influence Patterns, Attributed Signed Graph

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_A8kdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the vulnerability of RAG systems to knowledge‑poisoning and prompt‑injection attacks. The authors propose SIDRAG, which first decomposes the LLM’s attention output at the first generation step into document‑level influence vectors, builds an attributed signed graph whose node attributes encode influence strength and edge weights encode pairwise directional similarity, then discovers Jointly Consistent Subsets (JCSs) by jointly optimizing direction‑affinity and strength‑consistency. The highest‑scoring subset is removed and the remaining documents are used for answer generation. Experiments show that SIDRAG attains high exact malicious‑subset identification, the highest robust accuracy and the lowest attack success rate (ASR) among baselines.

Critical strengths
  1. The paper proposes set‑level influence modeling. It combines vectorized attention influence, an attributed signed graph, and a joint optimization that groups documents with consistent directions and similar strengths. This unified treatment handles both single‑document poisoning (high individual influence) and multi‑document coordinated attacks (strong aggregate influence) without requiring prior knowledge of the number of malicious documents.

  2. The paper achieves high efficiency and strong empirical validation. SIDRAG extracts influence only from early layers (1–16) at the first prediction step, resulting in a low average inference latency. It demonstrates that robustness can be achieved with negligible overhead.

Critical weaknesses
  1. The algorithm selects only the highest‑scoring Jointly Consistent Subset (JCS) and removes it if it passes the strength and consistency thresholds. An adaptive attacker can deliberately split malicious documents into several weaker sub‑sets that each fall below the removal threshold, thereby evading detection. The stress‑test results for poisoning ratios already show a sharp degradation in robust accuracy, indicating that the single‑subset removal is insufficient against coordinated split attacks.

  2. The paper’s innovation is largely a recombination of existing ideas. SIDRAG’s three main components, vectorized attention influence, signed/attributed graphs for document relations, and clustering of documents into consistent subsets, have all appeared independently in prior work. Although the paper includes an ablation section (Appendix C), the experiments only vary peripheral settings (layers, prediction positions), but didn’t demonstrate that each component is indispensable; it therefore risks being perceived as an engineering integration rather than a fundamentally new methodological contribution.

  3. The paper uses a simplistic threshold calibration: both the influence‑strength threshold and the directional‑consistency threshold are calibrated on only 100 clean NQ examples with a fixed retrieval size k = 10. No experiments explore how these thresholds behave when the number of retrieved documents changes, when different domains or datasets are used, or when the underlying model distribution varies. This raises concerns that the same thresholds may cause high false‑removal rates (FRR) or missed detections in realistic settings.

  4. The experimental scope is limited. The empirical evaluation is confined to k = 10 retrieved passages, and a narrow set of baselines (AV Filter, RobustRAG Keyword/Decoding). Recent graph‑based defenses such as GRADA, TrustRAG or RAGRank are not included, nor are larger retrieval depths (k = 20, 50, 100) examined. It would be great if the authors can provide more in-depth analysis and comparison, or at least explain why these defense mechanisms were not appropriate to be used as baselines.

Questions for authors
  1. Could you construct an explicit attack that splits malicious documents into two or three subsets with moderate directional similarity and report SIDRAG’s performance?

  2. Could you demonstrate how the influence‑strength threshold and the directional‑consistency threshold behave when the number of retrieved documents changes?

  3. Could the authors provide more comprehensive analyses and experiments with larger retrieval depths and additional baselines? If certain existing defense mechanisms are considered unsuitable for comparison, please clarify the reasons for their exclusion.

4 comments

W1 & Q1: Adaptive split attacks and high poisoning ratios

Thank you for raising this important concern. We address two points: whether adaptive split attacks can evade SIDRAG, and whether the stress-test degradation indicates that single-subset removal is insufficient.

First, regarding adaptive split attacks, we follow the adaptive-attack protocol used for AV Filter and construct attacks that explicitly target SIDRAG’s partitioning and filtering rules. Using an AutoDAN-style genetic search [1], we optimize six malicious documents within a ten-document context toward two prescribed split configurations: three subsets (2+2+2) and two subsets (3+3). The attack jointly optimizes three goals: inducing the target incorrect answer, forming the prescribed JCS partition with moderate inter-group similarity (the 25th–75th percentiles of inter-JCS similarity on 100 clean NQ contexts), and keeping every subgroup below SIDRAG’s removal thresholds.

For each dataset and split configuration we select 20 attacks that SIDRAG fully blocks before optimization (120 instances in total) and optimize each for up to 100 iterations on Mistral-7B. Each cell reports RACC / ASR (%).

SplitAttackNQHotpotQAMS MARCO
2+2+2Before splitting100 / 0100 / 0100 / 0
After splitting50 / 1535 / 2555 / 5
3+3Before splitting100 / 0100 / 0100 / 0
After splitting55 / 1045 / 545 / 15

Adaptive splitting raises SIDRAG’s ASR from 0% to 15% (2+2+2) and 10% (3+3) on average, demonstrating that split attacks can partially bypass the single-subset removal rule, with three subsets slightly more effective than two. RACC also drops to 35%–55%, as many optimized contexts lead the model to neither the correct nor the target answer. Each attack requires 81–114 minutes of query-specific optimization on an A100, indicating a high computational cost for only partial evasion.

Second, regarding the stress-test degradation at high poisoning ratios, we acknowledge that single-subset removal becomes less effective once malicious documents dominate the context. As a complementary measure, we also explore a lightweight extension that, after the top-scoring JCS passes the original thresholds and is removed, iteratively removes additional JCSs whose directional distance to any already-removed subset is below a calibrated threshold, until no further JCS qualifies. This improves exact removal rates at high poisoning ratios: from 66% to 72% at ϵ=0.8\epsilon=0.8 for Poison; at ϵ=0.9\epsilon=0.9, from 1% to 35% for Poison and from 0% to 75% for PIA. Performance at lower poisoning ratios remains stable. We will include both the adaptive attack results and this extension in the revision.

W2: Novelty and component necessity

Thank you for this comment. We address novelty first and component necessity second.

First, regarding novelty, we agree that vector-based attention attribution (Kobayashi et al., 2020) and correlation clustering are established tools; our contribution is a set-level detection formulation built on them. Single-document poisoning appears as strong individual influence, whereas multi-document poisoning appears as coordinated influence spread across documents (Figure 2). Detecting both without knowing the number of malicious documents requires deciding which documents to assess together, which we address with three adaptations:

  1. From attribution to detection: we use directional relations between document-level contribution vectors, not only their magnitudes, as evidence of coordination.
  2. From clustering to jointly consistent subsets: we extend correlation clustering with log-strength node attributes and a strength-consistency cost, so that documents exerting influence together form one candidate set, including singletons.
  3. A unified anomaly score: S(G)=T(G)C(G)exp⁡[−γEstr(G)]S(G)=T(G)C(G)\exp[-\gamma E_{\mathrm{str}}(G)] reduces to individual strength for a singleton and aggregates distributed influence for larger sets, so one criterion handles both cases.

Appendix E shows that this formulation is principled: splitting an update equally across more documents does not lower the set score (Proposition 1), and the partitioning objective recovers a separated malicious set (Proposition 2). Empirically, neither ingredient suffices alone: grouping by text similarity (SimCSE [2]) reaches an exact removal rate of only 18.67% (Poison) and 58.00% (PIA), versus 99.00% and 100.00% for SIDRAG.

Second, regarding component necessity, we further quantify the contribution of each component, beyond layer and position choices, in Appendix C.3 (Table C5); the results on Mistral-7B and NQ over 800 attacked contexts are shown below (%):

Removed componentMSDMSIASRFRR
None (full SIDRAG)99.8899.887.634.50
Partitioning: strength consistency96.5096.5010.133.60
Partitioning: directional relations25.0025.0064.753.00
Scoring: aggregate strength TT99.8825.1365.630.10
Scoring: directional consistency CC99.8899.887.639.80
Scoring: strength consistency99.8899.887.6310.60

Directional relations are needed to discover the malicious subset and aggregate strength to identify it (removing either raises ASR to about 65%), while the two scoring consistency terms preserve benign evidence (FRR rises to 9.80%–10.60% without them). Partitioning strength-consistency has a smaller effect in this setting (MSD/MSI falls from 99.88% to 96.50% and ASR rises to 10.13%), but each component still contributes to set-level detection.

W3 & Q2: Transferability of fixed thresholds

Thank you for this question. For each change you raise, we re-estimate the influence-strength threshold τT\tau_T and the directional-consistency threshold τC\tau_C with the original procedure (95th percentile on 100 clean examples), and we test whether keeping the original thresholds (τT=0.14\tau_T=0.14, τC=0.49\tau_C=0.49) causes false removals or missed detections. All values except thresholds are in %.

First, regarding models and datasets, each cell reports the re-estimated (τT,τC)(\tau_T,\tau_C):

ModelNQHotpotQAMS MARCO
Mistral-7B(0.14, 0.49)(0.15, 0.48)(0.13, 0.49)
Llama-2-7B(0.18, 0.50)(0.19, 0.49)(0.17, 0.48)
Llama-3.1-8B(0.25, 0.49)(0.28, 0.50)(0.23, 0.50)
Qwen3-8B(0.23, 0.51)(0.25, 0.49)(0.21, 0.49)

τC\tau_C remains relatively stable across models and datasets (0.48–0.51), as expected for a cosine-based quantity that is invariant to the scale of influence vectors. τT\tau_T shows larger variation (0.13–0.28), consistent with its dependence on vector norms and each model's representation scale.

The next table reports the clean-context FRR (Table A3) and the exact removal rate (ER), i.e., the share of attacked contexts in which exactly the malicious documents are removed, pooled over the three datasets and ϵ=0.1\epsilon=0.1–0.40.4. Values are for the original thresholds; arrows show the change after re-estimating both thresholds per model on 100 clean NQ examples (Mistral-7B is unchanged, as its NQ thresholds are the original ones).

ModelFRR↓ER (Poison)↑ER (PIA)↑
Mistral-7B3.8799.25100.00
Llama-2-7B9.70 → 4.3099.42 → 98.85100.00 → 99.75
Llama-3.1-8B11.03 → 6.2097.67 → 96.10100.00 → 98.80
Qwen3-8B10.90 → 6.5093.08 → 91.4099.67 → 98.20

Under the original thresholds, ER is at least 93.08% and FRR is below AV Filter's 12.83%–35.10% on every model (Table A3), although FRR rises from 3.87% on Mistral-7B to 9.70%–11.03% on the other models. Because the re-estimated thresholds on Llama-2-7B, Llama-3.1-8B, and Qwen3-8B are stricter than the original pair (τT\tau_T is higher), ER decreases slightly while FRR drops to 4.30%–6.50%.

Second, regarding retrieval depth (Q2), we repeat the calibration for Mistral-7B on NQ at each kk and evaluate on 100 clean and 400 attacked contexts per depth (Poison, ϵ=0.1\epsilon=0.1–0.40.4, so the number of malicious documents grows with kk). FRR, ER, RACC, and ASR use the original thresholds; the last two columns use the thresholds re-estimated at that kk:

kkRe-estimated (τT,τC)(\tau_T,\tau_C)FRR↓ER↑RACC↑ASR↓ER (re-est.)FRR (re-est.)
10(0.14, 0.49)4.5099.7557.507.7599.754.50
20(0.12, 0.47)3.8098.5055.258.5098.755.10
50(0.10, 0.45)2.6095.2550.0012.7597.505.80
100(0.09, 0.44)1.9091.5044.5016.0096.756.40

As kk increases, the re-estimated thresholds become more lenient. Using the original thresholds, ER and RACC degrade gradually while ASR rises. Re-estimating the thresholds at each depth recovers most of the lost ER (to 96.75%–98.75%) at the cost of a modestly higher FRR (5.1%–6.4%).

Overall, a single calibration on 100 clean NQ examples transfers across the four open-source models and three datasets: missed detections remain rare (ER of at least 93.08%), and FRR stays below AV Filter's on every model. Because calibration requires only clean examples and no attack data, it can be repeated offline when the model or retrieval depth changes. We will add these results and this guideline to the revised manuscript. Note that we have not yet run the full set of baselines at k=50k=50 and k=100k=100; the RACC numbers at those depths are therefore reported only for SIDRAG.

W4 & Q3: Additional baselines

Thank you for highlighting these relevant baselines. Larger retrieval depths (k=20,50,100k=20,50,100) are addressed in W3 & Q2. Our selection prioritized two representative inference-time defenses: AV Filter, the attention-based filter most related to SIDRAG, and RobustRAG, a widely used certified aggregation defense; both operate directly on a fixed retrieved set. Following your suggestion, we have further extended the comparison to include GRADA, TrustRAG, and RAGRank. Since no official code is available for RAGRank, we use an approximate reproduction.

Results follow the setting of Table 2, averaged over 1–4 malicious documents; cells report RACC / ASR (%), latency in seconds.

LLMDefenseMS MARCO PIAMS MARCO PoisonHotpotQA PIAHotpotQA PoisonNQ PIANQ PoisonLatency (s)↓
Mistral-7BGRADA52.0/33.233.0/59.023.0/55.022.8/58.534.8/38.827.5/52.01.11
TrustRAG30.2/8.228.5/7.822.2/11.023.8/11.028.0/5.027.2/5.23.62
RAGRank3.5/92.010.5/77.03.0/93.84.8/92.04.8/90.06.2/87.26.12
SIDRAG71.5/6.270.5/6.852.2/12.852.2/12.857.5/7.557.5/7.80.85
Llama2-7BGRADA47.5/36.223.0/68.512.8/59.813.0/63.027.0/40.815.0/56.50.83
TrustRAG27.2/4.825.8/5.219.5/14.020.5/12.233.2/3.033.0/3.23.15
RAGRank0.8/68.87.2/39.80.2/79.05.2/54.50.0/50.55.2/31.25.85
SIDRAG69.0/5.568.8/5.837.5/6.237.5/6.245.5/2.045.5/2.20.58
Llama-3.1-8BGRADA58.0/25.225.2/62.230.0/48.827.5/48.046.2/25.230.8/45.51.15
TrustRAG63.0/6.561.5/6.836.0/12.235.5/13.052.2/4.551.5/5.23.71
RAGRank31.2/54.521.5/68.215.8/62.213.2/57.218.2/56.214.5/57.86.22
SIDRAG66.0/6.863.2/8.850.8/8.250.2/9.058.5/6.558.2/7.00.84
Qwen3-8BGRADA43.0/33.525.2/58.519.8/59.017.0/61.833.5/38.026.8/49.51.39
TrustRAG41.0/10.240.5/12.026.8/13.826.8/13.245.2/4.244.8/4.84.93
RAGRank1.2/56.03.2/11.20.8/80.22.5/28.22.0/62.51.8/9.56.44
SIDRAG64.8/5.558.8/14.041.2/6.840.5/8.851.8/3.051.0/4.51.05

The gaps follow from the detection signals: GRADA reranks by similarity and query relevance, so mutually similar malicious documents can remain in the context; TrustRAG suppresses attacks but discards benign evidence (on Mistral-7B, its first stage falsely removes 25.23% of clean documents, versus 3.87% for SIDRAG); and RAGRank's citation-based authority, designed for CTI pipelines, transfers poorly to open-domain QA. We will clarify the baseline selection in Appendix D.1, add GRADA and TrustRAG to the main comparison, and report RAGRank with its reproduction details in Appendix D.1. Overall, SIDRAG achieves the highest RACC in all 24 model–dataset–attack settings, the lowest average ASR (7.10%, versus 8.22% for the next-best TrustRAG, whose RACC is only 35.17%), and the lowest latency on every model (0.58–1.05 s).

We hope these results address your concerns. We would be grateful if you would consider updating your assessment.

[1] Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models. ICLR 2024.

[2] Tianyu Gao, Xingcheng Yao, and Danqi Chen. SimCSE: Simple Contrastive Learning of Sentence Embeddings. EMNLP 2021, pp. 6894–6910.

Reviewer_cKyBOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SIDRAG, a defense against RAG poisoning without prior knowledge of the number of poisoned documents. The authors borrow ideas from mechanism interpretability by decomposing the attention outputs into a document-level influence vector and building an attributed graph accordingly. SIDRAG then detects poison suspects according to strength and consistency thresholds. The experiments report the identification rate and compare it with baseline defense methods.

Critical strengths
  1. The problem the paper addressed is interesting and practical. It is important that a defender does not need to know the poisoning fraction.

  2. The proposed method is straightforward and intuitive. I think it is an interesting idea to borrow ideas from attributed graphs to construct the defense. It appears that the proposed method can be practical, and it is compatible with frontier close models.

  3. The experiments are generally extensive, and the ablations are informative.

Critical weaknesses
  1. The paper lists GRADA, TrustRAG, and Reliability RAG as key reference papers, but these methods were not evaluated. Can the authors discuss further what the criteria for selecting baseline methods are? I only found a relevant sentence in Appendix D.1, but GRADA seems to satisfy the criterion of an "end-to-end" defense?

  2. Following the above, is it possible to combine the proposed method with LLM-judge methods?

  3. According to Appendix B.1, the influence vectors are extracted at the first answer position with a fixed template. I am wondering if a different system prompt would shift the attention patterns and causes a different calibrated threshold?

  4. It appears to me that the attack the authors evaluated is relatively easy to group. For example, the documents generated by PoisonedRAG are paraphrases; PIA documents use templates that are similar; and malicious documents seem to appear always at the end of the context. I think it is worthwhile to further discuss where malicious documents are placed and how they affect the results.

Questions for authors

Please see the weaknesses above. I am open to raising my score if the questions can be clarified.

2 comments

W1: Baseline selection

Thank you for highlighting these relevant baselines. Our selection prioritized two representative inference-time defenses: AV Filter, the attention-based filter most related to SIDRAG, and RobustRAG, a widely used certified aggregation defense; both operate directly on a fixed retrieved set. We agree that GRADA also meets the end-to-end defense criterion in Appendix D.1. We have therefore added GRADA as a baseline and further compared against TrustRAG and ReliabilityRAG, all under the setting and evaluation protocol of Table 2 and averaged across poisoning scales.

Across all settings, SIDRAG achieves the highest average RACC (55.01%) and the lowest ASR (7.10%); TrustRAG reaches a comparable ASR (8.22%) but a much lower RACC (35.17%). SIDRAG also achieves the lowest query latency across all four models (0.58–1.05 s). We will further clarify the baseline selection criteria in Appendix D.1 and incorporate these additional comparisons, including per-model, per-dataset, and per-poisoning-scale results, into the final version.

LLMDefenseMS MARCO PIAMS MARCO PoisonHotpotQA PIAHotpotQA PoisonNQ PIANQ PoisonLatency (s)↓
Mistral-7BGRADA52.0/33.233.0/59.023.0/55.022.8/58.534.8/38.827.5/52.01.11
TrustRAG30.2/8.228.5/7.822.2/11.023.8/11.028.0/5.027.2/5.23.62
ReliabilityRAG50.5/37.846.0/42.531.0/60.030.5/61.831.8/49.832.0/49.06.17
SIDRAG71.5/6.270.5/6.852.2/12.852.2/12.857.5/7.557.5/7.80.85
Llama2-7BGRADA47.5/36.223.0/68.512.8/59.813.0/63.027.0/40.815.0/56.50.83
TrustRAG27.2/4.825.8/5.219.5/14.020.5/12.233.2/3.033.0/3.23.15
ReliabilityRAG53.0/34.552.2/35.526.5/58.830.0/55.233.5/44.834.8/40.83.80
SIDRAG69.0/5.568.8/5.837.5/6.237.5/6.245.5/2.045.5/2.20.58
Llama-3.1-8BGRADA58.0/25.225.2/62.230.0/48.827.5/48.046.2/25.230.8/45.51.15
TrustRAG63.0/6.561.5/6.836.0/12.235.5/13.052.2/4.551.5/5.23.71
ReliabilityRAG50.0/37.549.0/38.226.5/66.825.8/65.834.0/48.832.5/47.57.75
SIDRAG66.0/6.863.2/8.850.8/8.250.2/9.058.5/6.558.2/7.00.84
Qwen3-8BGRADA43.0/33.525.2/58.519.8/59.017.0/61.833.5/38.026.8/49.51.39
TrustRAG41.0/10.240.5/12.026.8/13.826.8/13.245.2/4.244.8/4.84.93
ReliabilityRAG30.2/51.033.5/43.515.0/78.215.2/75.822.5/54.824.2/52.89.24
SIDRAG64.8/5.558.8/14.041.2/6.840.5/8.851.8/3.051.0/4.51.05

W2: Combination with LLM-judge defenses

Thank you for the suggestion. Yes, SIDRAG can be combined with LLM-judge defenses. We test this by replacing TrustRAG’s first-stage filtering (text clustering and similarity) with SIDRAG, while keeping TrustRAG’s second-stage reliability assessment and answer-generation pipeline unchanged.

We evaluate on Mistral-7B using NQ, HotpotQA, and MS MARCO, with 1–4 malicious documents under prompt injection (PIA) and content poisoning (Poison), totaling 300 clean and 2,400 attacked contexts.

MethodACCRACCASR
TrustRAG29.0026.678.04
SIDRAG65.0060.258.96
SIDRAG + TrustRAG Stage 231.0031.466.42

The combination improves over original TrustRAG on all three metrics: RACC rises from 26.67% to 31.46%, ASR falls from 8.04% to 6.42%, and ACC rises from 29.00% to 31.00%. Filtering quality also improves substantially—the false-removal rate on clean contexts drops from 25.23% to 3.87%, and the fraction of attacked contexts still containing malicious documents drops from 24.17% to 0.25%.

At the same time, the combined method remains less accurate than SIDRAG alone. Both retain identical documents across all 2,700 contexts, so the gap arises in TrustRAG’s later integration and generation stages, which are conservative on their own (29.00% ACC for TrustRAG alone). This shows that SIDRAG can serve as an effective front-end filter for LLM-judge pipelines, while final accuracy still depends on the quality of the subsequent judgment and answer-generation steps.

W3: Threshold stability under prompt changes

Thank you for raising this question. We vary the task instruction in the fixed template of Appendix B.1 and test whether the original thresholds still apply.

We compare the original instruction (Orig×1) with a paraphrased instruction repeated twice (Para×2), an evidence-constrained instruction repeated three times (Evid×3), and a defensive instruction to ignore instructions inside documents repeated four times (Def×4), replacing the instruction in all demonstrations and the query. We evaluate four models on 100 questions each from NQ, HotpotQA, and MS MARCO, with clean contexts and Poison/PIA contexts (7 benign and 3 malicious documents), keeping τC=0.49\tau_C=0.49 and τT=0.14\tau_T=0.14 fixed.

The table reports FRR and the exact removal rate (ER, the share of contexts whose removal set equals the malicious set) under the original prompt, with the range over all four prompts in parentheses (%).

ModelFRRER (Poison)ER (PIA)
Mistral-7B3.87 (3.87–4.33)99.00 (98.33–99.33)100.00 (unchanged)
Llama-2-7B9.70 (9.13–9.70)99.33 (98.67–99.33)100.00 (unchanged)
Llama-3.1-8B11.03 (11.03–11.30)97.67 (97.67–98.67)100.00 (99.33–100.00)
Qwen3-8B10.90 (10.90–11.40)93.00 (92.00–93.33)100.00 (99.67–100.00)

Across the three new prompts, ER changes by at most 1.00 percentage point and FRR increases by at most 0.50 percentage points. The influence of the same document also changes little: the median cosine similarity of its influence directions before and after the change is 0.946–0.989, and the median ratio of its layer-averaged influence norms is 0.965–1.113.

Re-estimating the thresholds with Mistral-7B following the original procedure gives direction thresholds of 0.49, 0.46, 0.48, and 0.49 and strength thresholds of 0.14, 0.15, 0.13, and 0.15 for Orig×1, Para×2, Evid×3, and Def×4, all close to the original values. Therefore, threshold recalibration is unnecessary for the tested prompt variations.

W4: Attack groupability and document placement

Thank you for raising both points. We address textual similarity first and document placement second.

Regarding textual similarity, SIDRAG groups documents by their influence within the model rather than by surface text. Appendix B (Sensitivity to Attack Text Characteristics) already shows that varying malicious-document length or removing their shared question text leaves the results unchanged. We add two comparisons with Mistral-7B on 300 clean contexts and 300 attacked contexts per attack across three datasets (7 benign and 3 malicious documents each).

First, a text-only baseline applies complete-linkage clustering to SimCSE [1] embeddings and removes the most similar cluster of 2–4 documents (SIDRAG's size cap), with its threshold (0.895) calibrated on clean NQ data to keep FRR at most 5%. It reaches an ER of only 18.67% (Poison) and 58.00% (PIA) with 6.23% FRR, versus 99.00% and 100.00% with 3.87% FRR for SIDRAG, so similarity from paraphrases or templates cannot reproduce SIDRAG's detection.

Second, in each attacked context we select the three benign documents most similar under SimCSE alone. In the 57 contexts where this group is at least as textually similar as the malicious group, the malicious group still has higher directional consistency CC in 55 and higher aggregate strength TT in all 57. In the corresponding clean contexts, none of the 41 distinct benign groups is removed entirely (36 fully retained, five partially). Textual similarity is thus not equivalent to the joint influence of malicious documents.

Regarding placement, we move the three malicious documents, which the main experiments place at the end, to other positions in the same contexts while keeping everything else fixed (positions indexed from 1):

LayoutPosER (Poison)ER (PIA)
End8, 9, 1099.00100.00
Beginning1, 2, 397.6799.00
Middle4, 5, 698.33100.00
Dispersed2, 5, 899.0099.33

Relative to the end placement, ER drops by at most 1.33 (Poison) and 1.00 (PIA) percentage points, so under the tested layouts SIDRAG's detection does not depend on malicious documents being grouped at the end of the context.

We will clarify the placement used in the main experiments in Appendix B.1 and add these similarity and placement analyses to Appendix B. Overall, SIDRAG's detection relies on neither textual similarity nor document position: it keeps an ER of at least 97.50% across all tested layouts, far above text-similarity filtering.

We hope these clarifications address your questions and would appreciate it if you would consider raising your score.

[1] Gao, Yao, and Chen. SimCSE: Simple Contrastive Learning of Sentence Embeddings. EMNLP 2021.

Reviewer_RtGwOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SIDRAG, an adaptive defense framework against retrieval-augmented generation (RAG) poisoning attacks. The authors observe that malicious documents may exhibit strong and directionally consistent influence patterns, motivating a set-level approach to detecting coordinated poisoning.

SIDRAG decomposes Transformer attention outputs into document-level influence vectors and constructs an attributed signed graph based on influence strength and directional similarity. It then identifies Jointly Consistent Subsets (JCSs) through graph partitioning and removes suspicious subset according to a composite anomaly score.

Experiments across five LLMs, three QA datasets, and different poisoning ratios demonstrate competitive robustness and computational efficiency.

Critical strengths
  1. An important research problem with a novel perspective on set-level influence modeling. The paper addresses the challenge of defending RAG systems against poisoning attacks with an unknown number of malicious documents. By jointly modeling the strength and direction of attention contributions, SIDRAG provides an interesting set-level detection perspective beyond conventional scalar attention-based anomaly detection. The motivation is clear, and the proposed framework is logically designed.

  2. Competitive experimental results supported by comprehensive analyses. SIDRAG achieves substantial improvements in robustness across multiple LLMs, QA datasets, and poisoning ratios while maintaining relatively low inference latency. The paper also includes analyses of document influence patterns, component ablations, layer-selection studies, and stress tests under high poisoning ratios, providing substantial empirical support for the proposed method.

Critical weaknesses
  1. SIDRAG builds its detection mechanism on the observation that malicious documents tend to exhibit strong and directionally consistent influence patterns. However, multiple benign documents supporting the same correct answer may also produce highly consistent influence directions.

  2. The filtering mechanism has limitations when malicious documents are partitioned into multiple subsets. Appendix B.3 shows that malicious documents may be divided into multiple Jointly Consistent Subsets (JCSs) at high poisoning ratios, resulting in a significant degradation in identification performance. Since SIDRAG removes at most the highest-scoring subset, some malicious documents may remain in the retrieved context.

  3. The stability of fixed filtering thresholds across different models and datasets remains unclear. The filtering thresholds are calibrated using 100 clean NQ examples with Mistral-7B and subsequently applied across different models, datasets, and attack settings. Although this design simplifies deployment, different models may exhibit different internal representation scales, retrieval distributions, and attention behaviors.

Questions for authors
  1. Regarding highly consistent benign documents. Could multiple benign documents providing similar and mutually supporting evidence also exhibit highly consistent influence directions? Could the authors further analyze the false-removal cases in the clean-context experiments and clarify how their influence patterns differ from those of coordinated malicious documents?

  2. Appendix B.3 shows that malicious documents may be split into multiple JCSs at high poisoning ratios. Could the authors investigate whether an iterative filtering strategy could mitigate this limitation?

2 comments

W1 & Q1: Influence consistency of benign documents and false-removal analysis

Thank you for your question. We analyze it from two perspectives.

First, how do benign groups supporting the same answer differ from malicious groups? We identified 197 benign document groups supporting the same answer in 300 clean contexts across three datasets. We separately paired the 152 groups containing 2–4 documents with Poison and PIA malicious groups for the same question and with the same number of documents, and used Mistral-7B to compute influence metrics within the full context containing each group.

The table below reports the means of CC, group-size-normalized dispersion of log influence strengths (V=Estr/∣G∣V=E_{\mathrm{str}}/\lvert G\rvert), and mean per-document influence strength (T/∣G∣T/\lvert G\rvert).

Document groupCCVVT/∣G∣T/\lvert G\rvert
Benign0.4710.1390.063
Poison0.6620.0320.128
PIA0.6920.0210.145

The results show that benign groups supporting the same answer have lower mean directional consistency and per-document influence strength, but greater strength dispersion, than both types of malicious groups. This indicates that semantic agreement on an answer does not necessarily correspond to an influence pattern with jointly consistent directions and strengths within the model.

Second, if the overall patterns differ, why are some benign groups still falsely removed? We analyzed falsely removed benign groups from the same clean contexts and paired them with malicious groups matched by question and group size. The table below includes correctly retained benign groups as a reference to compare falsely removed groups with both types of documents; N denotes the number of groups, and the remaining values are the means of the metrics above.

Document groupNCCVVT/∣G∣T/\lvert G\rvert
Correctly retained benign groups2890.4600.0510.035
Falsely removed benign groups290.5650.0740.089
Poison3000.6580.0450.106
PIA3000.6940.0360.119

The results show that falsely removed benign groups have higher mean directional consistency and per-document influence strength than correctly retained groups, but lower values than malicious groups, along with greater strength dispersion. Thus, falsely removed groups retain an overall pattern distinct from that of typical malicious groups, while exhibiting stronger and more consistent influence than other benign groups.

W2 & Q2: High poisoning ratios and residual malicious subsets

Thank you for raising this limitation. First, we would like to clarify that SIDRAG extends effective defense to higher poisoning ratios than existing defenses. As shown below, SIDRAG substantially outperforms the baselines under the same settings at ϵ=0.5\epsilon=0.5–0.80.8. Cells report RACC / ASR (%) as in Table 2.

Attackϵ\epsilonSIDRAGAVFilterRobustRAG KeywordRobustRAG Decoding
PIA0.549 / 713 / 7520 / 5316 / 60
PIA0.650 / 80 / 9721 / 5616 / 62
PIA0.743 / 82 / 9618 / 6012 / 66
PIA0.826 / 281 / 9717 / 6410 / 67
PIA0.95 / 810 / 9815 / 627 / 71
Poison0.547 / 98 / 8217 / 659 / 73
Poison0.650 / 85 / 8714 / 6810 / 74
Poison0.744 / 74 / 9114 / 707 / 77
Poison0.825 / 282 / 919 / 746 / 78
Poison0.93 / 871 / 979 / 744 / 79

The limitation of single-subset removal that you identified becomes apparent under extreme poisoning: at ϵ=0.9\epsilon=0.9 for PIA, RobustRAG Keyword's RACC/ASR (15/62) exceeds SIDRAG's (5/81), because residual malicious subsets beyond the single removed one survive in the context.

Second, inspired by your suggestion of iterative filtering, we extend SIDRAG to iteratively remove residual malicious subsets, directly targeting this ϵ=0.9\epsilon=0.9 regime. After the top-scoring JCS passes the original thresholds and is removed, SIDRAG repeats the check on the remaining JCSs, removing any whose directional distance to an already-removed subset falls below a threshold τD\tau_D, until no further JCS qualifies; τD\tau_D is calibrated on 100 clean NQ contexts disjoint from the test set so that no clean JCS pair would be merged. This works because split malicious subsets remain directionally close: under extreme poisoning, the median distance from the removed malicious subset is 0.457 to other malicious groups versus 0.814 to benign groups. On NQ with Mistral-7B (1,800 contexts, 1–9 malicious documents), the exact removal rate (all and only malicious documents removed) stays at 98%–100% for ϵ≤0.7\epsilon\leq0.7 and rises substantially at high ratios (original: SIDRAG as submitted, Appendix B.3; extended: with τD\tau_D):

ϵ\epsilonPoison: originalPoison: extendedPIA: originalPIA: extended
0.866726281
0.9135075

We will include this extension and its analysis in the revision.

W3: Stability of fixed thresholds across models and datasets

Thank you for pointing out this issue. Using 100 additional calibration examples from each of NQ, HotpotQA, and MS MARCO, we re-estimated the directional-consistency and influence-strength thresholds separately for each of the four models. The table below reports (τT,τC)(\tau_T,\tau_C) in that order.

ModelNQHotpotQAMS MARCO
Mistral-7B(0.14, 0.49)(0.15, 0.48)(0.13, 0.49)
Llama-2-7B(0.18, 0.50)(0.19, 0.49)(0.17, 0.48)
Llama-3.1-8B(0.25, 0.49)(0.28, 0.50)(0.23, 0.50)
Qwen3-8B(0.23, 0.51)(0.25, 0.49)(0.21, 0.49)

The directional-consistency threshold is relatively stable, ranging from 0.48 to 0.51, whereas the influence-strength threshold varies more noticeably, ranging from 0.13 to 0.28.

To assess the practical impact of these differences, we further report filtering results obtained by continuing to use the original fixed thresholds. Exact removal rate (ER) is the proportion of attacked contexts in which all malicious documents are removed without removing any benign documents. The table below summarizes results across the three datasets, with attack settings covering poisoning ratios of 10%–40%. All values are percentages.

ModelClean-context FRR↓ER (Poison)↑ER (PIA)↑
Mistral-7B3.8799.25100.00
Llama-2-7B9.7099.42100.00
Llama-3.1-8B11.0397.67100.00
Qwen3-8B10.9093.0899.67

The original fixed thresholds still maintain high exact removal rates: 93.08%–99.42% for Poison and 99.67%–100% for PIA. In comparison, FRR on clean contexts increases from 3.87% for Mistral-7B to 9.70%–11.03% for the other models.

We therefore agree that the stability of the fixed thresholds varies across models. This variation is primarily reflected in FRR on clean contexts, with a maximum increase of 7.16 percentage points; meanwhile, exact removal rates for malicious documents remain high, and the overall variation is relatively manageable.

We hope these clarifications address your questions and would appreciate it if you would consider raising your score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.