- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Update-Equivariant Lineage Coding (UELC), a black-box watermarking framework for provenance attribution in evolving Retrieval-Augmented Generation (RAG) systems. The authors introduce the concept of Provenance Laundering Attacks (PLA), in which an adversary steals a protected RAG knowledge base and subsequently modifies its contents or system components while preserving task utility and inherited knowledge. To address this threat, UELC embeds epoch-specific watermarks into fact-preserving semantic carriers using error-correcting codes, periodically refreshes the watermark during authorized knowledge-base updates, and employs cryptographic commitment chains to authenticate historical lineage relationships. The framework enables knowledge-base owners to recover watermark evidence through black-box queries without accessing the suspect system's internal components. Experiments on FARAD and NQ-50K demonstrate improved robustness over RAG-WM, WARD, and SentinelRAG under several knowledge-base transformations.
Critical strengths
- The paper introduces the problem of provenance laundering in evolving RAG systems, extending traditional snapshot-based ownership verification to lineage attribution. This is an interesting research direction for protecting RAG knowledge bases against unauthorized reuse and modification.
- The proposed UELC framework combines semantic watermarking, error-correcting codes, authorized watermark refresh, and cryptographic commitment chains to support black-box provenance attribution across multiple knowledge-base versions.
- The paper is well-written.
Critical weaknesses
- The multi-epoch auditing mechanism has a worst-case query complexity of , raising scalability concerns for continuously evolving RAG systems. The evaluation covers only six epochs, which may not reflect real-world deployments with frequent updates. The authors should evaluate longer evolution histories and explore more efficient epoch identification methods.
- The method relies on linguistic characteristics of generated responses for watermark extraction, which may be affected by common RAG deployment practices such as concise answers, fixed output formats, and paraphrasing. In the A6-C experiment, UELC's TPR drops to 0.270 despite retaining 97% of task utility and inherited knowledge, raising concerns about its practical robustness. The authors should evaluate more diverse response formats and clarify the assumptions required for reliable watermark extraction.
- In static ownership verification, the proposed method achieves performance comparable to RAG-WM, WARD, and SentinelRAG. Although UELC introduces authorized watermark refresh, existing baselines can also be extended to periodically re-protect updated knowledge bases. The authors consider such rolling baselines in Appendix C.6, where Rolling-WARD achieves a lineage TPR of 0.946 compared with 0.954 for UELC. This relatively small difference raises questions about the extent of the improvement in attribution effectiveness. Moreover, the rolling baselines rely on from-scratch re-protection, whereas UELC uses a specifically designed incremental refresh mechanism. The paper should better isolate the benefits of its individual components and provide a clearer justification for the fairness of the update-cost comparisons.
- The experimental evaluation is restricted to two relatively small open-source language models, Qwen3-8B and Llama-3.1-8B. Although black-box access can be simulated using locally deployed models, the paper does not demonstrate the effectiveness of UELC on commercial RAG services powered by closed-source LLM APIs. Since watermark extraction depends on how different generators express retrieved knowledge, its effectiveness may vary significantly across model architectures, scales, and generation policies. Experiments using commercial LLM APIs, such as GPT, Gemini, or Claude, would strengthen the generalizability and practical relevance of the proposed framework.
- The paper defines successful provenance laundering under task-utility and inherited-knowledge preservation constraints of and . However, the practical justification for selecting these thresholds remains unclear. Although the authors evaluate different sanitization intensities, the results suggest that substantial watermark suppression can occur while most of the original knowledge and utility are also preserved. For example, even with 100% canonicalization, the RAG system retains 93.7% task utility and 86.2% inherited knowledge, while the TPR drops further to 0.204, although this setting falls outside the predefined preservation constraints. These results raise concerns about the practical security value of the proposed watermarking mechanism.
Questions for authors
See Weaknesses above.