- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper explores the issue of provenance poisoning in multi-agent reasoning systems powered by LLMs. The authors point out that even when a system ultimately reaches a correct consensus, compromised evidence can continue to contaminate the chain of citations.The main contributions of this work include defining a novel evaluation metric to quantify citation contamination, revealing the limitations of conventional accuracy metrics in detecting such vulnerabilities, and demonstrating that downstream applications suffer quantifiable harm when they reuse these contaminated citations.The authors validate these findings through extensive experiments across various datasets, such as MINT CWQ and MuSiQue, using models including Qwen3 and Mistral, and they further evaluate a mitigation strategy based on evidence provenance.
Critical strengths
- The authors introduce and formalize the concept of provenance poisoning, highlighting a significant flaw in current security assessments of multi-agent systems. The study compellingly demonstrates that standard accuracy metrics entirely overlook the potential for contamination within chains of provenance. This finding is highly significant, as it reveals a subtle security risk: even when upstream systems produce correct consensus results, downstream applications can still suffer quantifiable harm as a consequence.
- This research goes beyond merely identifying poisoned citations by explicitly testing the impact of these citations on subsequent readers. Through extensive experiments conducted across multiple datasets such as MuSiQue and involving various models like Qwen3 and Mistral, the authors demonstrate that removing these planted, false pieces of evidence significantly improves performance on downstream tasks. This comprehensive validation elevates the study from a purely theoretical exploration of vulnerabilities to a compelling empirical demonstration of real-world, systemic harm.
Critical weaknesses
- Why does the proposed EGA defense mechanism exhibit such inconsistent performance across different models, even leading to an increase in the poisoning rate and a decline in overall answer accuracy (Answer F1) when applied to the Gemma model?
- The experiments rely solely on static, pre-set misinformation. How effective would this system and its defense mechanism be against adaptive attackers capable of dynamically adjusting their rhetoric based on multi-agent debates?
- The experiments show that the baseline poisoning rate for advanced, closed-source models like GPT-4o is already extremely low (1–5%). Does this suggest that the source poisoning threat highlighted in this study does not, in fact, pose a significant challenge for state-of-the-art large language models?
- Although tests using opaque IDs were conducted, they were limited to a single model and dataset (MINT/Qwen) and still retained certain header cues (e.g., "Web result"). Can we definitively rule out the possibility that the models rely on metadata cues rather than the text content itself when generating citations?
- The EGA defense mechanism significantly increases the number of calls required for LLMs (e.g., from 16 to 35–39 calls per question). Is such a high computational and temporal cost feasible for practical deployment in multi-agent systems?
Questions for authors
See the above weaknesses.