- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies whether query-relevant passages without answer support can strengthen targeted RAG poisoning. The proposed method combines two passages supporting an attacker-chosen answer with three answer-free passages that compete for retrieval slots. It uses relation-aware generation, semantic checks, and proxy retrieval scores to construct and select these passages. Experiments on three QA datasets report improved attack success, with additional analyses across generators, retrievers, and defenses.
Critical strengths
1.The paper examines a useful distinction between query relevance and answer support. Its results suggest that passages need not directly support the attacker’s answer to contribute to targeted poisoning.
2.The evaluation covers three datasets, with additional comparisons across four generators and six retrievers on NQ. Transferring the same attack passages across retrievers provides evidence that the observed gains are not specific to one retriever.
3.The paired analysis fixes the target-bearing passages and tracks changes in retrieval, filtering, and answers. Reporting both gains and losses, including transitions from mixed answers to exclusive target adoption, helps explain the observed ASR changes.
Critical weaknesses
- The contribution beyond existing multi-document poisoning is insufficiently established. The main comparison increases the injection budget from two to five passages while keeping retrieval at top-5. This demonstrates that additional answer-free passages can help, but does not establish their advantage over additional target-bearing passages. A comparison between five target-bearing passages and two target-bearing plus three dilution passages, constructed with comparable resources, is missing.
2.The performance gap over PoisonedRAG is not adequately explained. The authors’ two target-bearing passages already outperform five PoisonedRAG passages on all three datasets. On HotpotQA, they achieve 83% versus 67%, while dilution adds only another four percentage points. Thus, much of the reported advantage exists before dilution is introduced. The paper does not sufficiently document baseline construction settings or compare retrieved attack counts to explain this gap. This makes it difficult to assess whether the advantage comes from dilution, stronger target-bearing passages, or differences in baseline configuration.
Questions for authors
1.Why do your two target-bearing passages outperform five PoisonedRAG passages on all three datasets, particularly on HotpotQA (83% vs. 67%)? What accounts for this gap before any dilution passages are added?
2.Which PoisonedRAG variant and construction settings were used? What explains the advantage of your two target-bearing passages over its five passages? Please provide comparable retrieval statistics and generation budgets.