- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies targeted data poisoning attacks on Graph Retrieval-Augmented Generation (GraphRAG) systems and proposes TopoPoison, a black-box attack framework. The core idea is to treat the reasoning subgraph for each candidate answer as the basic unit for poisoning intervention. The full pipeline runs in three stages: query structure recovery, candidate support manipulation, and poisoning text realization, and it works purely using response feedback from the victim, with no access to internal data. Experiments across 3 multi-hop QA benchmarks and 2 mainstream GraphRAG systems show the method clearly outperforms existing baselines like GragPoison and KEPo on targeted attack success rate.
Critical strengths
- The reasoning subgraph corresponding to the candidate answer is used as the basic unit of poisoning intervention, and the attack strategy is designed from the perspective of topological structure support, which has positive reference significance for understanding the structural vulnerability of GraphRAG.
- The experimental setup is thorough. It covers multiple datasets, and includes component ablations, budget sensitivity analysis, cross-model transferability tests, and impact measurements on non-target queries.
Critical weaknesses
- All experiments run on just two systems: MS-GraphRAG and LightRAG. Those are both fairly classic implementations: one uses hierarchical community summaries, the other mixes graph and vector retrieval. But the field has moved on: there are newer state-of-the-art methods like GFM-RAG, HippoRAG 2, and G-Reasoner that use graph foundation models, personalized PageRank for associative retrieval, structured multi-hop reasoning chains, and so on. These have very different graph construction and retrieval mechanics, and very different topological properties. The paper never tests on any of these more advanced architectures, or analyzes how different graph-building strategies affect poisoning robustness.
- This paper sets the attack as a purely reactive black box attack, but the query structure recovery stage requires an average of nearly 20 relevant queries to the victim, which is easily recognized by access control and anomaly detection mechanisms in real deployment scenarios, and the concealment and practicability of the attack are in doubt.
- Although the paper is named "topology poisoning", the core intervention methods are still traditional association enhancement, fact negation and answer replacement. The utilization of graph topology only stops at the level of subgraph scoring and path ranking, and does not explore the unique attack points of graph structure (such as bridge nodes, community hubs, reasoning bottlenecks, etc.). Compared with the existing GraphRAG poisoning methods, the core differences are mainly in the granularity of attack organization, the underlying poisoning logic has no essential innovation, and the technical increment is limited.
Questions for authors
- In the real scenario, high-frequency queries on the same topic are easy to trigger anomaly detection. How can the author evaluate the concealment and feasibility of this attack in the actual deployment? Are attack performance results under low query budget provided?
- The candidate support score is a heuristic on the attacker side, not aligned with the victim’s internal retrieval ranking logic. How do you know the budget allocation strategy based on this score is actually optimal? Could scoring bias lead to misallocation of poisoning resources?
- Experiments only cover two classic GraphRAG architectures. Why did you not include more recent state-of-the-art systems like GFM-RAG or HippoRAG 2 in your evaluation? Do you expect this attack to generalize equally well across graph architectures with very different topological properties?
8 comments
W2 & Q1 — Query Budget and Deployment Feasibility
We thank the reviewer for highlighting the practical implications of repeated victim queries. Although Table 17 in the revised manuscript reports averages of 18.5 and 19.4 queries per target question on MS-GraphRAG and LightRAG, respectively, these interaction costs are not prerequisites for effective poisoning under the evaluated setting. Our additional experiments show that a single-query variant achieves TASRs of 61.0% and 85.0%, compared with 63.0% and 85.0% under the full query budget. We detail the experimental settings and discuss the implications for deployment feasibility below.
(1) Attack performance under limited query budgets.
We evaluated TopoPoison on MuSiQue with different limits on the number of victim queries per target question, denoted by . This budget includes the initial interaction and subsequent follow-up queries used for structure recovery. Whereas the default recovery procedure uses separate queries to obtain entity and relation information, the single-query variant obtains both through one victim interaction. All remaining settings were kept unchanged. The following tables report the query limit, the average number of queries actually issued, and the resulting attack performance.
Table 3: Effect of the victim query budget against MS-GraphRAG on MuSiQue.
| Avg. Queries | TASR | C-TASR | UASR | C-UASR | |
|---|---|---|---|---|---|
| 1 | 1.00 | 61.0% | 61.7% | 85.0% | 74.5% |
| 2 | 2.00 | 62.0% | 59.6% | 82.0% | 66.0% |
| 6 | 5.76 | 61.0% | 63.8% | 87.0% | 74.5% |
| 10 | 9.32 | 61.0% | 55.3% | 81.0% | 59.6% |
| 14 | 12.00 | 63.0% | 55.3% | 84.0% | 68.1% |
| 18 | 13.98 | 64.0% | 61.7% | 85.0% | 70.2% |
| 22 | 15.39 | 61.0% | 63.8% | 86.0% | 72.3% |
| 26 | 16.39 | 65.0% | 57.4% | 83.0% | 63.8% |
| 30 | 17.27 | 65.0% | 61.7% | 83.0% | 66.0% |
| 34 | 17.95 | 63.0% | 57.4% | 81.0% | 61.7% |
| 38 | 18.47 | 61.0% | 53.2% | 80.0% | 59.6% |
| 42 | 18.50 | 63.0% | 61.7% | 82.0% | 63.8% |
Table 4: Effect of the victim query budget against LightRAG on MuSiQue.
| Avg. Queries | TASR | C-TASR | UASR | C-UASR | |
|---|---|---|---|---|---|
| 1 | 1.00 | 85.0% | 82.5% | 92.0% | 86.0% |
| 2 | 2.00 | 87.0% | 84.2% | 93.0% | 89.5% |
| 6 | 5.49 | 85.0% | 82.5% | 94.0% | 89.5% |
| 10 | 8.36 | 87.0% | 86.0% | 93.0% | 89.5% |
| 14 | 10.76 | 87.0% | 86.0% | 95.0% | 91.2% |
| 18 | 12.63 | 86.0% | 89.5% | 93.0% | 91.2% |
| 22 | 14.25 | 86.0% | 84.2% | 92.0% | 87.7% |
| 26 | 16.17 | 84.0% | 82.5% | 94.0% | 89.5% |
| 30 | 17.30 | 85.0% | 86.0% | 93.0% | 87.7% |
| 34 | 18.23 | 88.0% | 86.0% | 94.0% | 89.5% |
| 38 | 19.03 | 86.0% | 87.7% | 94.0% | 91.2% |
| 42 | 19.40 | 85.0% | 86.0% | 91.0% | 91.2% |
TASR ranges from 61.0% to 65.0% on MS-GraphRAG and from 84.0% to 88.0% on LightRAG across the tested budgets, with no consistent upward trend. Reducing from 42 to 1 changes TASR by −2.0 and 0.0 percentage points, respectively, while reducing the average interactions from 18.5 and 19.4 to one.
The reason is the limited scope of the intervention. Under the approximately 300-word poisoning budget, TopoPoison reinforces the target answer and redirects at most one competing-answer subgraph . In this setting, a small number of queries can recover enough local structure to plan the intervention. A larger query budget permits more extensive exploration, but the present results do not establish that such exploration is necessary under this constrained intervention budget.
W2 & Q1 — Query Budget and Deployment Feasibility (continued)
(2) Implications for deployment feasibility and detectability.
We agree that repeated, closely related queries could trigger rate limits or anomaly detection in deployed systems. TopoPoison’s follow-up queries seek clarification about entities, relations, or intermediate facts identified in earlier responses, and can therefore resemble ordinary clarification requests.
Prior studies have shown that user-visible GraphRAG responses can expose entity and relation information [1], and that adaptive follow-up queries can support incremental graph recovery [2]. These findings motivate feedback-based recovery, while our objective is narrower: recovering local information relevant to a target question for subsequent poisoning. The single-query results provide direct evidence that, for the evaluated setting, this objective can be achieved with substantially fewer interactions than those reported for our default recovery procedure. They address the concern that the attack necessarily depends on repeated querying, while leaving its detectability under specific deployment controls as an open empirical question.
The results address the interaction-cost concern in the evaluated setting; concealment remains an open empirical question. We have added the budget sweep to Appendix E and stated the deployment scope in the Limitations.
References
[1] Jiale Liu, Jiahao Zhang, and Suhang Wang. 2026. Exposing Privacy Risks in Graph Retrieval-Augmented Generation. In Findings of the Association for Computational Linguistics: ACL 2026, pages 18073–18093, San Diego, California, United States. Association for Computational Linguistics.
[2] Shuhua Yang, Jiahao Zhang, Yilong Wang, Dongwon Lee, and Suhang Wang. 2026. Query-Efficient Agentic Graph Extraction Attacks on GraphRAG Systems. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16017–16041, San Diego, California, United States. Association for Computational Linguistics.
W1 & Q3 — Evaluation on Additional GraphRAG Architectures.
We thank the reviewer for suggesting a broader evaluation.
We extended our experiments to HippoRAG 2 and GFM-RAG, using the G-reasoner-34M configuration for the latter. These systems extend our evaluation beyond community-based and hybrid graph–vector retrieval: HippoRAG 2 uses Personalized PageRank for associative retrieval, whereas the evaluated GFM-RAG configuration uses a pretrained graph foundation model for graph reasoning and retrieval. G-reasoner-34M serves as the graph model, with a separate LLM responsible for answer generation. On MuSiQue, we compared TopoPoison against GragPoison, KEPo-S, and KEPo-M. The results are reported below.
Table 1: Effectiveness of GraphRAG poisoning attacks against HippoRAG 2 on MuSiQue.
Table 2: Effectiveness of GraphRAG poisoning attacks against GFM-RAG (with G-reasoner-34M) on MuSiQue.
Cross-system interpretation. TopoPoison outperformed all evaluated baselines on all four metrics on both systems. Its TASR reached 84.0% on HippoRAG2 and 78.0% on GFM-RAG exceeding the best baseline on each system by 18 and 16 percentage points, respectively.
These results show that TopoPoison retains its targeted attack advantage on two architectures with different retrieval mechanisms. We have added the new results, their experimental configurations and this interpretation to Section 5 and Appendix E.