- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the vulnerability of RAG systems to knowledge‑poisoning and prompt‑injection attacks. The authors propose SIDRAG, which first decomposes the LLM’s attention output at the first generation step into document‑level influence vectors, builds an attributed signed graph whose node attributes encode influence strength and edge weights encode pairwise directional similarity, then discovers Jointly Consistent Subsets (JCSs) by jointly optimizing direction‑affinity and strength‑consistency. The highest‑scoring subset is removed and the remaining documents are used for answer generation. Experiments show that SIDRAG attains high exact malicious‑subset identification, the highest robust accuracy and the lowest attack success rate (ASR) among baselines.
Critical strengths
-
The paper proposes set‑level influence modeling. It combines vectorized attention influence, an attributed signed graph, and a joint optimization that groups documents with consistent directions and similar strengths. This unified treatment handles both single‑document poisoning (high individual influence) and multi‑document coordinated attacks (strong aggregate influence) without requiring prior knowledge of the number of malicious documents.
-
The paper achieves high efficiency and strong empirical validation. SIDRAG extracts influence only from early layers (1–16) at the first prediction step, resulting in a low average inference latency. It demonstrates that robustness can be achieved with negligible overhead.
Critical weaknesses
-
The algorithm selects only the highest‑scoring Jointly Consistent Subset (JCS) and removes it if it passes the strength and consistency thresholds. An adaptive attacker can deliberately split malicious documents into several weaker sub‑sets that each fall below the removal threshold, thereby evading detection. The stress‑test results for poisoning ratios already show a sharp degradation in robust accuracy, indicating that the single‑subset removal is insufficient against coordinated split attacks.
-
The paper’s innovation is largely a recombination of existing ideas. SIDRAG’s three main components, vectorized attention influence, signed/attributed graphs for document relations, and clustering of documents into consistent subsets, have all appeared independently in prior work. Although the paper includes an ablation section (Appendix C), the experiments only vary peripheral settings (layers, prediction positions), but didn’t demonstrate that each component is indispensable; it therefore risks being perceived as an engineering integration rather than a fundamentally new methodological contribution.
-
The paper uses a simplistic threshold calibration: both the influence‑strength threshold and the directional‑consistency threshold are calibrated on only 100 clean NQ examples with a fixed retrieval size k = 10. No experiments explore how these thresholds behave when the number of retrieved documents changes, when different domains or datasets are used, or when the underlying model distribution varies. This raises concerns that the same thresholds may cause high false‑removal rates (FRR) or missed detections in realistic settings.
-
The experimental scope is limited. The empirical evaluation is confined to k = 10 retrieved passages, and a narrow set of baselines (AV Filter, RobustRAG Keyword/Decoding). Recent graph‑based defenses such as GRADA, TrustRAG or RAGRank are not included, nor are larger retrieval depths (k = 20, 50, 100) examined. It would be great if the authors can provide more in-depth analysis and comparison, or at least explain why these defense mechanisms were not appropriate to be used as baselines.
Questions for authors
-
Could you construct an explicit attack that splits malicious documents into two or three subsets with moderate directional similarity and report SIDRAG’s performance?
-
Could you demonstrate how the influence‑strength threshold and the directional‑consistency threshold behave when the number of retrieved documents changes?
-
Could the authors provide more comprehensive analyses and experiments with larger retrieval depths and additional baselines? If certain existing defense mechanisms are considered unsuitable for comparison, please clarify the reasons for their exclusion.
4 comments
W2: Novelty and component necessity
Thank you for this comment. We address novelty first and component necessity second.
First, regarding novelty, we agree that vector-based attention attribution (Kobayashi et al., 2020) and correlation clustering are established tools; our contribution is a set-level detection formulation built on them. Single-document poisoning appears as strong individual influence, whereas multi-document poisoning appears as coordinated influence spread across documents (Figure 2). Detecting both without knowing the number of malicious documents requires deciding which documents to assess together, which we address with three adaptations:
- From attribution to detection: we use directional relations between document-level contribution vectors, not only their magnitudes, as evidence of coordination.
- From clustering to jointly consistent subsets: we extend correlation clustering with log-strength node attributes and a strength-consistency cost, so that documents exerting influence together form one candidate set, including singletons.
- A unified anomaly score: reduces to individual strength for a singleton and aggregates distributed influence for larger sets, so one criterion handles both cases.
Appendix E shows that this formulation is principled: splitting an update equally across more documents does not lower the set score (Proposition 1), and the partitioning objective recovers a separated malicious set (Proposition 2). Empirically, neither ingredient suffices alone: grouping by text similarity (SimCSE [2]) reaches an exact removal rate of only 18.67% (Poison) and 58.00% (PIA), versus 99.00% and 100.00% for SIDRAG.
Second, regarding component necessity, we further quantify the contribution of each component, beyond layer and position choices, in Appendix C.3 (Table C5); the results on Mistral-7B and NQ over 800 attacked contexts are shown below (%):
| Removed component | MSD | MSI | ASR | FRR |
|---|---|---|---|---|
| None (full SIDRAG) | 99.88 | 99.88 | 7.63 | 4.50 |
| Partitioning: strength consistency | 96.50 | 96.50 | 10.13 | 3.60 |
| Partitioning: directional relations | 25.00 | 25.00 | 64.75 | 3.00 |
| Scoring: aggregate strength | 99.88 | 25.13 | 65.63 | 0.10 |
| Scoring: directional consistency | 99.88 | 99.88 | 7.63 | 9.80 |
| Scoring: strength consistency | 99.88 | 99.88 | 7.63 | 10.60 |
Directional relations are needed to discover the malicious subset and aggregate strength to identify it (removing either raises ASR to about 65%), while the two scoring consistency terms preserve benign evidence (FRR rises to 9.80%–10.60% without them). Partitioning strength-consistency has a smaller effect in this setting (MSD/MSI falls from 99.88% to 96.50% and ASR rises to 10.13%), but each component still contributes to set-level detection.
W3 & Q2: Transferability of fixed thresholds
Thank you for this question. For each change you raise, we re-estimate the influence-strength threshold and the directional-consistency threshold with the original procedure (95th percentile on 100 clean examples), and we test whether keeping the original thresholds (, ) causes false removals or missed detections. All values except thresholds are in %.
First, regarding models and datasets, each cell reports the re-estimated :
| Model | NQ | HotpotQA | MS MARCO |
|---|---|---|---|
| Mistral-7B | (0.14, 0.49) | (0.15, 0.48) | (0.13, 0.49) |
| Llama-2-7B | (0.18, 0.50) | (0.19, 0.49) | (0.17, 0.48) |
| Llama-3.1-8B | (0.25, 0.49) | (0.28, 0.50) | (0.23, 0.50) |
| Qwen3-8B | (0.23, 0.51) | (0.25, 0.49) | (0.21, 0.49) |
remains relatively stable across models and datasets (0.48–0.51), as expected for a cosine-based quantity that is invariant to the scale of influence vectors. shows larger variation (0.13–0.28), consistent with its dependence on vector norms and each model's representation scale.
The next table reports the clean-context FRR (Table A3) and the exact removal rate (ER), i.e., the share of attacked contexts in which exactly the malicious documents are removed, pooled over the three datasets and –. Values are for the original thresholds; arrows show the change after re-estimating both thresholds per model on 100 clean NQ examples (Mistral-7B is unchanged, as its NQ thresholds are the original ones).
| Model | FRR↓ | ER (Poison)↑ | ER (PIA)↑ |
|---|---|---|---|
| Mistral-7B | 3.87 | 99.25 | 100.00 |
| Llama-2-7B | 9.70 → 4.30 | 99.42 → 98.85 | 100.00 → 99.75 |
| Llama-3.1-8B | 11.03 → 6.20 | 97.67 → 96.10 | 100.00 → 98.80 |
| Qwen3-8B | 10.90 → 6.50 | 93.08 → 91.40 | 99.67 → 98.20 |
Under the original thresholds, ER is at least 93.08% and FRR is below AV Filter's 12.83%–35.10% on every model (Table A3), although FRR rises from 3.87% on Mistral-7B to 9.70%–11.03% on the other models. Because the re-estimated thresholds on Llama-2-7B, Llama-3.1-8B, and Qwen3-8B are stricter than the original pair ( is higher), ER decreases slightly while FRR drops to 4.30%–6.50%.
Second, regarding retrieval depth (Q2), we repeat the calibration for Mistral-7B on NQ at each and evaluate on 100 clean and 400 attacked contexts per depth (Poison, –, so the number of malicious documents grows with ). FRR, ER, RACC, and ASR use the original thresholds; the last two columns use the thresholds re-estimated at that :
| Re-estimated | FRR↓ | ER↑ | RACC↑ | ASR↓ | ER (re-est.) | FRR (re-est.) | |
|---|---|---|---|---|---|---|---|
| 10 | (0.14, 0.49) | 4.50 | 99.75 | 57.50 | 7.75 | 99.75 | 4.50 |
| 20 | (0.12, 0.47) | 3.80 | 98.50 | 55.25 | 8.50 | 98.75 | 5.10 |
| 50 | (0.10, 0.45) | 2.60 | 95.25 | 50.00 | 12.75 | 97.50 | 5.80 |
| 100 | (0.09, 0.44) | 1.90 | 91.50 | 44.50 | 16.00 | 96.75 | 6.40 |
As increases, the re-estimated thresholds become more lenient. Using the original thresholds, ER and RACC degrade gradually while ASR rises. Re-estimating the thresholds at each depth recovers most of the lost ER (to 96.75%–98.75%) at the cost of a modestly higher FRR (5.1%–6.4%).
Overall, a single calibration on 100 clean NQ examples transfers across the four open-source models and three datasets: missed detections remain rare (ER of at least 93.08%), and FRR stays below AV Filter's on every model. Because calibration requires only clean examples and no attack data, it can be repeated offline when the model or retrieval depth changes. We will add these results and this guideline to the revised manuscript. Note that we have not yet run the full set of baselines at and ; the RACC numbers at those depths are therefore reported only for SIDRAG.
W4 & Q3: Additional baselines
Thank you for highlighting these relevant baselines. Larger retrieval depths () are addressed in W3 & Q2. Our selection prioritized two representative inference-time defenses: AV Filter, the attention-based filter most related to SIDRAG, and RobustRAG, a widely used certified aggregation defense; both operate directly on a fixed retrieved set. Following your suggestion, we have further extended the comparison to include GRADA, TrustRAG, and RAGRank. Since no official code is available for RAGRank, we use an approximate reproduction.
Results follow the setting of Table 2, averaged over 1–4 malicious documents; cells report RACC / ASR (%), latency in seconds.
| LLM | Defense | MS MARCO PIA | MS MARCO Poison | HotpotQA PIA | HotpotQA Poison | NQ PIA | NQ Poison | Latency (s)↓ |
|---|---|---|---|---|---|---|---|---|
| Mistral-7B | GRADA | 52.0/33.2 | 33.0/59.0 | 23.0/55.0 | 22.8/58.5 | 34.8/38.8 | 27.5/52.0 | 1.11 |
| TrustRAG | 30.2/8.2 | 28.5/7.8 | 22.2/11.0 | 23.8/11.0 | 28.0/5.0 | 27.2/5.2 | 3.62 | |
| RAGRank | 3.5/92.0 | 10.5/77.0 | 3.0/93.8 | 4.8/92.0 | 4.8/90.0 | 6.2/87.2 | 6.12 | |
| SIDRAG | 71.5/6.2 | 70.5/6.8 | 52.2/12.8 | 52.2/12.8 | 57.5/7.5 | 57.5/7.8 | 0.85 | |
| Llama2-7B | GRADA | 47.5/36.2 | 23.0/68.5 | 12.8/59.8 | 13.0/63.0 | 27.0/40.8 | 15.0/56.5 | 0.83 |
| TrustRAG | 27.2/4.8 | 25.8/5.2 | 19.5/14.0 | 20.5/12.2 | 33.2/3.0 | 33.0/3.2 | 3.15 | |
| RAGRank | 0.8/68.8 | 7.2/39.8 | 0.2/79.0 | 5.2/54.5 | 0.0/50.5 | 5.2/31.2 | 5.85 | |
| SIDRAG | 69.0/5.5 | 68.8/5.8 | 37.5/6.2 | 37.5/6.2 | 45.5/2.0 | 45.5/2.2 | 0.58 | |
| Llama-3.1-8B | GRADA | 58.0/25.2 | 25.2/62.2 | 30.0/48.8 | 27.5/48.0 | 46.2/25.2 | 30.8/45.5 | 1.15 |
| TrustRAG | 63.0/6.5 | 61.5/6.8 | 36.0/12.2 | 35.5/13.0 | 52.2/4.5 | 51.5/5.2 | 3.71 | |
| RAGRank | 31.2/54.5 | 21.5/68.2 | 15.8/62.2 | 13.2/57.2 | 18.2/56.2 | 14.5/57.8 | 6.22 | |
| SIDRAG | 66.0/6.8 | 63.2/8.8 | 50.8/8.2 | 50.2/9.0 | 58.5/6.5 | 58.2/7.0 | 0.84 | |
| Qwen3-8B | GRADA | 43.0/33.5 | 25.2/58.5 | 19.8/59.0 | 17.0/61.8 | 33.5/38.0 | 26.8/49.5 | 1.39 |
| TrustRAG | 41.0/10.2 | 40.5/12.0 | 26.8/13.8 | 26.8/13.2 | 45.2/4.2 | 44.8/4.8 | 4.93 | |
| RAGRank | 1.2/56.0 | 3.2/11.2 | 0.8/80.2 | 2.5/28.2 | 2.0/62.5 | 1.8/9.5 | 6.44 | |
| SIDRAG | 64.8/5.5 | 58.8/14.0 | 41.2/6.8 | 40.5/8.8 | 51.8/3.0 | 51.0/4.5 | 1.05 |
The gaps follow from the detection signals: GRADA reranks by similarity and query relevance, so mutually similar malicious documents can remain in the context; TrustRAG suppresses attacks but discards benign evidence (on Mistral-7B, its first stage falsely removes 25.23% of clean documents, versus 3.87% for SIDRAG); and RAGRank's citation-based authority, designed for CTI pipelines, transfers poorly to open-domain QA. We will clarify the baseline selection in Appendix D.1, add GRADA and TrustRAG to the main comparison, and report RAGRank with its reproduction details in Appendix D.1. Overall, SIDRAG achieves the highest RACC in all 24 model–dataset–attack settings, the lowest average ASR (7.10%, versus 8.22% for the next-best TrustRAG, whose RACC is only 35.17%), and the lowest latency on every model (0.58–1.05 s).
We hope these results address your concerns. We would be grateful if you would consider updating your assessment.
[1] Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models. ICLR 2024.
[2] Tianyu Gao, Xingcheng Yao, and Danqi Chen. SimCSE: Simple Contrastive Learning of Sentence Embeddings. EMNLP 2021, pp. 6894–6910.
W1 & Q1: Adaptive split attacks and high poisoning ratios
Thank you for raising this important concern. We address two points: whether adaptive split attacks can evade SIDRAG, and whether the stress-test degradation indicates that single-subset removal is insufficient.
First, regarding adaptive split attacks, we follow the adaptive-attack protocol used for AV Filter and construct attacks that explicitly target SIDRAG’s partitioning and filtering rules. Using an AutoDAN-style genetic search [1], we optimize six malicious documents within a ten-document context toward two prescribed split configurations: three subsets (2+2+2) and two subsets (3+3). The attack jointly optimizes three goals: inducing the target incorrect answer, forming the prescribed JCS partition with moderate inter-group similarity (the 25th–75th percentiles of inter-JCS similarity on 100 clean NQ contexts), and keeping every subgroup below SIDRAG’s removal thresholds.
For each dataset and split configuration we select 20 attacks that SIDRAG fully blocks before optimization (120 instances in total) and optimize each for up to 100 iterations on Mistral-7B. Each cell reports RACC / ASR (%).
Adaptive splitting raises SIDRAG’s ASR from 0% to 15% (2+2+2) and 10% (3+3) on average, demonstrating that split attacks can partially bypass the single-subset removal rule, with three subsets slightly more effective than two. RACC also drops to 35%–55%, as many optimized contexts lead the model to neither the correct nor the target answer. Each attack requires 81–114 minutes of query-specific optimization on an A100, indicating a high computational cost for only partial evasion.
Second, regarding the stress-test degradation at high poisoning ratios, we acknowledge that single-subset removal becomes less effective once malicious documents dominate the context. As a complementary measure, we also explore a lightweight extension that, after the top-scoring JCS passes the original thresholds and is removed, iteratively removes additional JCSs whose directional distance to any already-removed subset is below a calibrated threshold, until no further JCS qualifies. This improves exact removal rates at high poisoning ratios: from 66% to 72% at ϵ=0.8 for Poison; at ϵ=0.9, from 1% to 35% for Poison and from 0% to 75% for PIA. Performance at lower poisoning ratios remains stable. We will include both the adaptive attack results and this extension in the revision.