- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes SAFEG, a two-stage defense for GraphRAG poisoning across the knowledge lifecycle. During KG evolution, each incoming document is hypothetically integrated into the existing graph and scored by the Jensen–Shannon divergence between random-walk stationary distributions before and after insertion. Documents whose structural divergence exceeds a validation-selected threshold are filtered. During inference, SAFEG retrieves nested subgraphs and generates a candidate answer from each evidence view, constructs a retrieval-independent knowledge prior from the LLM’s parametric knowledge, and uses an LLM verifier to select the candidate most consistent with that prior.
Critical strengths
- The lifecycle framing is well motivated and addresses two distinct failure points.
- SAFEG is evaluated across multiple datasets, GraphRAG frameworks, LLMs, attacks, and a comparatively large baseline set. The authors also include threshold sensitivity, component ablations, cross-attack tests, and a second LLM-judge agreement check.
Critical weaknesses
- The evolution-time detector relies on a fragile assumption that poisoned documents induce unusually large global structural change. The core decision rule treats large stationary-distribution divergence as suspicious. However, an adaptive attacker can plausibly distribute a false claim over low-impact entities/relations, mimic the structural footprint of benign updates, or split one attack across multiple small documents. Conversely, benign but important updates can legitimately introduce new hubs or strongly alter graph connectivity. The current attacks are not adapted to the proposed divergence statistic.
- The inference-time verifier assumes that the LLM’s parametric knowledge is a trustworthy reference, which conflicts with a core motivation for RAG. The verifier favors answers consistent with an internally generated knowledge prior. This can help when poisoning contradicts well-known facts, but it is potentially harmful exactly when retrieval is needed most: recent events, long-tail facts, domain-specific knowledge, or information absent/incorrect in the base model.
Questions for authors
- Resolve the issues mentioned in the Weakness
- Can the authors evaluate Stage II on facts that are deliberately absent from or incorrect in the verifier model’s parametric knowledge—for example, post-training temporal facts or private facts that are present only in the clean KG?
4 comments
Rebuttal by Authors (Part 2/2)
R3: False Rejections of Benign Knowledge Updates (W1/Q1)
Thank you for highlighting this concern. Figure 11 examines the trade-off between poisoning detection recall and benign false-positive rate (FPR), with the detection threshold selected to balance security and knowledge preservation. To further evaluate the impact on legitimate knowledge evolution, we conducted an additional analysis focusing on high-impact benign updates, defined as the top 20% of benign documents exhibiting the largest structural changes.
| Dataset | Overall Benign FPR ↓ | High-impact Benign FPR ↓ | High-impact Acceptance ↑ |
|---|---|---|---|
| Graph-Medical | 18.50% | 25.00% | 75.00% |
| Graph-Novel | 23.00% | 32.50% | 67.50% |
| HotpotQA | 20.50% | 27.50% | 72.50% |
| 2WikiMultihopQA | 21.50% | 30.00% | 70.00% |
| Average | 20.88% | 28.75% | 71.25% |
The results show that Stage I preserves the majority of legitimate knowledge updates even when they introduce substantial structural changes. Although high-impact benign updates exhibit a higher FPR than the overall benign population (28.75% vs. 20.88%), 71.25% of these updates are still accepted. These findings demonstrate that substantial structural changes do not necessarily trigger rejection and quantify the trade-off between poisoning detection and legitimate knowledge preservation.
R4: Reliability of Parametric Knowledge in Stage II (W2/Q1)
Thank you for your comment. Stage II generates candidate answers from retrieved KG evidence and uses independently generated parametric knowledge as an auxiliary reference for credibility verification. This design allows the verifier to assess candidate reliability while retaining external knowledge as the basis for answer generation.
To evaluate its effectiveness when the LLM's internal knowledge is insufficient, we conducted additional experiments using LightRAG with Llama-3.1-8B across four datasets. Specifically, we selected queries where No-GraphRAG answers incorrectly but clean GraphRAG answers correctly, representing cases where external knowledge is essential for accurate responses.
| Dataset | # Queries | Clean GraphRAG ACC ↑ | + Stage II ACC ↑ |
|---|---|---|---|
| Graph-Medical | 32 | 100.00% | 84.38% |
| Graph-Novel | 36 | 100.00% | 83.33% |
| HotpotQA | 46 | 100.00% | 84.78% |
| 2WikiMultihopQA | 43 | 100.00% | 81.40% |
| Overall | 157 | 100.00% | 83.44% |
The results show that Stage II preserves 131 of 157 correct KG-supported answers (83.44%) on queries where the base LLM fails without retrieval. This demonstrates that credibility verification can retain most valid external knowledge even when the model's parametric knowledge is insufficient.
R5: Performance under Unknown or Incorrect Parametric Knowledge (W2/Q2)
Thank you for your suggestion. To directly evaluate Stage II when the verifier's internal knowledge is absent or incorrect, we further analyzed the knowledge references generated for 68 queries from Graph-Medical and Graph-Novel.
We classify these queries into two categories: (1) Unknown Prior, where the generated knowledge reference lacks the facts required to answer the query, and (2) Incorrect Prior, where the reference contains information inconsistent with the ground truth. We then evaluate whether Stage II preserves the correct answers supported by the clean KG.
| Dataset | Prior Condition | # Queries | Stage II Correct | Stage II ACC ↑ |
|---|---|---|---|---|
| Graph-Medical | Unknown Prior | 18 | 16 | 88.89% |
| Graph-Medical | Incorrect Prior | 14 | 11 | 78.57% |
| Graph-Novel | Unknown Prior | 21 | 17 | 80.95% |
| Graph-Novel | Incorrect Prior | 15 | 13 | 86.67% |
| Overall | Unknown Prior | 39 | 33 | 84.62% |
| Overall | Incorrect Prior | 29 | 24 | 82.76% |
Stage II preserves 84.62% and 82.76% of correct KG-supported answers under unknown and incorrect parametric knowledge, respectively. These results demonstrate that Stage II can retain most valid external knowledge even when the generated reference is incomplete or inaccurate, while also quantifying the remaining influence of unreliable priors.
We sincerely thank you again for the careful evaluation and constructive suggestions. We believe these additional experiments and clarifications have strengthened the manuscript.
Thanks for your rebuttal. it solve most of my concerns. However, I still suggest that the author should further enhance the theoretical analysis. I will raise my score to 3
We sincerely thank you for your recognition of our rebuttal and for raising your score to 3. We greatly appreciate your constructive feedback and will further strengthen the theoretical analysis in the revised manuscript. Thank you again for your valuable time and support.
Rebuttal by Authors (Part 1/2)
Thank you very much for your careful and constructive review. We greatly appreciate your recognition that "the lifecycle framing is well motivated and addresses two distinct failure points" and that "SAFEG is evaluated across multiple datasets, GraphRAG frameworks, LLMs, attacks, and a comparatively large baseline set." We also greatly appreciate your insightful suggestions for further strengthening our work. We will address your comments and questions individually in the following five parts:
Thank you for your comment. Stage I is motivated by the observation that poisoning attacks often introduce abnormal entity-relation associations, leading to measurable changes in global KG information propagation. By analyzing shifts in the stationary distribution, SafeG captures the collective structural impact of incoming knowledge updates rather than simply counting newly introduced entities or edges.
As shown in
Figure 10, poisoned documents exhibit larger structural deviations than benign documents across four datasets, achieving AUROC values of 0.847, 0.769, 0.814, and 0.797, respectively. These results demonstrate the effectiveness of structural deviation as a discriminative signal for identifying suspicious knowledge updates. Stage II further complements structural screening by mitigating residual poisoning during answer generation.Thank you for your suggestion. To evaluate whether adaptive attackers can exploit low structural deviations to bypass Stage I, we conducted additional experiments considering two strategies: (1) Relation Rewiring, which manipulates relationships among potentially existing entities to limit structural changes, and (2) Split-Payload Poisoning, which distributes malicious knowledge across multiple documents to reduce per-document structural deviations. Both attacks operate under the original black-box document-injection threat model.
We evaluated both strategies alongside Regular Poisoning on Graph-Novel and HotpotQA using LightRAG with Llama-3.1-8B, with 50 attack attempts per strategy and dataset. We report the Stage I pass rate, defined as the proportion of attack attempts in which all injected documents pass structural screening, together with the ASR of Vanilla GraphRAG and SafeG.
The results reveal a clear trade-off between structural evasion and attack effectiveness. Compared with Regular Poisoning, Relation Rewiring and Split-Payload Poisoning increase the average Stage I pass rate from 26.00% to 67.00% and 66.00%, respectively, while reducing Vanilla ASR from 45.00% to 25.00% and 30.00%. This indicates that reducing structural deviations improves screening evasion but does not necessarily strengthen end-to-end poisoning. Furthermore, SafeG reduces the ASR of Relation Rewiring and Split-Payload Poisoning to 12.00% and 15.00%, respectively, demonstrating its effectiveness against both adaptive strategies. We have incorporated these additional experiments and analyses into
Appendix E.