- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SILK, a framework for implicit knowledge extraction from RAG systems through a black-box query interface. SILK combines two components: semantic-network-based anchor selection and constrained query rewriting. The first component uses atomic claims to estimate knowledge gain, propagates observed scores to unexplored neighboring anchors, and balances topic continuity with exploration. The second component uses a shadow intent detector and an LLM judge to guide constrained beam search over candidate queries.
The evaluation compares SILK with six baselines across four datasets, multiple defense settings, and alternative model configurations. The paper additionally includes component ablations, evaluator validation, cost analysis, and an evaluation of surrogate RAG systems constructed from the extracted knowledge.
Critical strengths
-
A coherent framework for tracking and expanding knowledge acquisition in the implicit setting. Measuring extraction progress is difficult when responses do not reproduce retrieved passages verbatim. SILK addresses this through atomic-claim-based knowledge tracking and uses a semantic network to guide exploration. The combination is well motivated, and the ablations in Section 5.6 support the usefulness of its components. The results also help distinguish the contribution of anchor selection to retrieval coverage from that of query rewriting to leakage.
-
Consistent empirical improvements over the relevant implicit baseline. SILK outperforms IKEA on EE, LR, and SR across all four datasets under both input and output defenses in Table 1. For example, under input defense on FIQA, SR increases from 0.43 to 0.71. The additional evaluations with different attack models and RAG configurations provide further evidence that the improvements are not restricted to one configuration. Together with the surrogate-RAG evaluation and cost measurements, these results constitute a useful empirical contribution.
Critical weaknesses
-
The interpretation of semantic similarity as knowledge leakage needs qualification. Section 5.1 classifies a retrieved chunk as leaked when either ROUGE-L recall exceeds 0.5 or semantic similarity exceeds 0.65. The semantic criterion is useful for capturing paraphrases, but it may also count topical similarity without establishing recovery of distinctive facts. Appendix B provides helpful human validation, although the leakage assessment covers only 30 pairs. Moreover, the surrogate-RAG evaluation measures response similarity rather than factual correctness or the recovery of information unavailable without access to the target knowledge base. These limitations do not negate the reported comparative gains, but they make the stronger claims about proprietary knowledge recovery and near-equivalent service reconstruction less conclusive.
-
The defense evaluation and uncertainty reporting need clearer specification. Section 5.4 does not sufficiently describe the attack composition, train/test separation, sequence construction, and calibration protocol for the sequence detector. The model configurations and relationship between the shadow and target intent detectors also require clarification. In addition, the results do not report variability despite stochastic anchor sampling and LLM generation. These omissions limit reproducibility and make it difficult to determine how broadly the low blocking rates should generalize beyond the evaluated implementations.
Questions for authors
-
What performance is obtained when access to the target knowledge base is removed, both for the leakage metric and for the surrogate-system evaluation? Such a control would help distinguish knowledge-base-specific recovery from information already available in the underlying model. Can the authors also clarify how the semantic-similarity threshold was selected and how the 30 human-evaluated leakage samples were distributed across datasets and methods?
-
Please clarify the configurations of the shadow and target intent detectors, the three attacks used to train the sequence detector, the train/test separation, and the construction of the evaluated sequences. How was the 5% false-positive operating point calibrated? How many independent runs were conducted, and how variable were the principal results?
-
Section 5.1 defines SR = EE × LR, but the PoR result on HealthCareMagic without defense in Table 1 reports EE = 0.72, LR = 0.79, and SR = 0.64, whereas the product is approximately 0.57. Does this arise from separate averaging, another aggregation procedure, or a reporting error? Please also reconcile the discrepancies between Appendix A and Tables 6–7.
4 comments
Rebuttal by Authors [2/3]
Response to Weakness 2 / Question 2: Detector configuration, calibration, and variability
Regarding the target and shadow intent detectors
Under our threat model, the target detector’s prompt is unknown to the attacker. We therefore use different LLMs and prompts for the target and shadow intent detectors: the target detector uses DeepSeek-V3, and the shadow detector uses GLM-4.7. Their prompts are provided under RAG Intent Detection and Shadow Intent Detection in Appendix I. These configurations are now explicit in .
Regarding sequence construction and train/test separation
Our sequence-detector evaluation follows the sequence-detection setup in IKEA [7], with benign dialogue generation using DialogueForge [8]. The 2,200 benign dialogues use OASST1 and Chatbot Arena seeds, with one seed user query followed by five synthesized user queries. Each dialogue forms one six-query sequence. We select PoR, IKEA, and SILK as the three training attack sources to cover a strong explicit extraction attack and the two stealthy implicit attacks. We collect two 200-round trajectories per attack and dataset: 24 trajectories in total. Six-query windows with a stride of three produce 65 sequences per trajectory. We use a chronological split within each trajectory, retaining 51 earlier training sequences and 13 later test sequences, and discarding one boundary window. The additional attack test set contains 100 sequences each from TGTB, PIDE, DGEA, and RAG-Thief.
The data composition is provided in :
| Source | Training | Calibration | Test |
|---|---|---|---|
| PoR / IKEA / SILK | 1,224 | 0 | 312 |
| TGTB / PIDE / DGEA / RAG-Thief | 0 | 0 | 400 |
| Benign dialogues | 1,600 | 200 | 400 |
| Total sequences | 2,824 | 200 | 1,112 |
Regarding the 5% false-positive operating point
We calibrate the sequence detector on the 200 held-out benign dialogues, selecting the smallest evaluated threshold with empirical query-level FPR below 5%, then freezing it for testing. A query is blocked if any evaluated window containing it is flagged. At the selected threshold, nine calibration dialogues are flagged. Each is one six-query window, corresponding to 54/1,200 queries, or 4.5% FPR. The equations and aggregation procedure are given in .
Regarding independent runs and variability
Each reported extraction result comes from one complete attack trajectory ( rounds) per method, dataset, and setting. The paper evaluates SILK under 14 distinct settings per dataset: the three principal defense settings (Table 1) and eleven additional settings covering alternative attack LLMs, backend LLMs, retrievers, and rerankers (Tables 20–23). We summarize the 14 independently executed runs per dataset in : the standard deviations of EE, LR, and SR are all below 0.05 (maximum: 0.043), indicating SILK’s robustness.
Moreover, SILK exceeds IKEA in all 48 paired configuration–defense–dataset comparisons, with mean ΔSR +19.9%. We consider that these results account not only for the randomness of stochastic anchor sampling and LLM generation, but also for different RAG configurations and attack models; together they indicate that SILK is stable with respect to both randomness and configuration.
SILK’s EE, LR, and SR across the 14 settings per dataset are summarized below as mean ± sample SD ().
| Dataset | EE (mean ± SD) | LR (mean ± SD) | SR (mean ± SD) |
|---|---|---|---|
| HealthCareMagic | 0.813 ± 0.028 | 0.694 ± 0.029 | 0.564 ± 0.036 |
| FIQA | 0.854 ± 0.033 | 0.806 ± 0.030 | 0.689 ± 0.043 |
| SciFact | 0.644 ± 0.026 | 0.601 ± 0.019 | 0.386 ± 0.023 |
| NFCorpus | 0.577 ± 0.035 | 0.685 ± 0.025 | 0.395 ± 0.033 |
The paired comparison with IKEA is summarized below ().
| Metric | Pairs | Mean difference (SILK − IKEA) | SD | SILK higher |
|---|---|---|---|---|
| SR | 48 | +0.199 | 0.059 | 48/48 |
| LR | 48 | +0.141 | 0.060 | 48/48 |
Response to Question 3: SR consistency and Appendix A discrepancies
Comparison with the original records confirmed that the PoR SR discrepancy was a reporting error. We corrected the entry and carefully checked the other reported results for the same issue. We have also corrected the discrepancies between Appendix A and Tables 6–7 and checked the corresponding descriptions and tables throughout the manuscript for consistency.
Rebuttal by Authors [3/3]
References
[1] Lewis, Patrick, et al. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” NeurIPS (2020). Paper.
[2] Gao, Yunfan, et al. “Retrieval-Augmented Generation for Large Language Models: A Survey.” arXiv:2312.10997 (2023). Paper.
[3] Shuster, Kurt, et al. “Retrieval Augmentation Reduces Hallucination in Conversation.” Findings of EMNLP (2021). Paper.
[4] Leung, Kin Kwan, et al. “Classifying and Addressing the Diversity of Errors in Retrieval-Augmented Generation Systems.” EACL (2026). Paper.
[5] Zhang, Maosen, et al. “LeakDojo: Decoding the Leakage Threats of RAG Systems.” Findings of ACL (2026). Paper.
[6] Di Maio, Christian, et al. “Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases.” arXiv:2412.18295 (2024). Paper.
[7] Wang, Yuhao, et al. “Silent Leaks: Implicit Knowledge Extraction Attack on RAG Systems through Benign Queries.” ICLR (2026). Paper.
[8] Zhu, Ruizhe, et al. “DialogueForge: LLM Simulation of Human-Chatbot Dialogue.” arXiv:2507.15752 (2025). Paper.
Looking forward to further feedback
Dear Reviewer yb3r,
As the rebuttal deadline approaches, we would like to kindly follow up on our response and welcome any further feedback. If our clarifications address your concerns, we would greatly appreciate your reconsideration of the rating.
Thank you very much for your support!
Best,
The Authors
Rebuttal by Authors [1/3]
Response to Weakness 1 / Question 1: Leakage validity, surrogate controls, and human evaluation
Regarding the source of the extracted knowledge
A primary goal of RAG is to provide domain-specific knowledge while reducing hallucinations [1, 2, 3]. In line with this goal, our RAG setting follows a standard enterprise configuration used in prior work [4, 5, 6] (Section 5.1; see also Prompt 1, Appendix I): the model must answer strictly based on retrieved content, and respond “I don’t know” when the retrieved evidence is insufficient. Thus, in our experiments, the LLM is intended to reason over retrieved evidence rather than answer from parametric knowledge.
To verify this, we conduct two retrieval-control experiments using the actual query sets from each attack: (1) retrieval disabled; and (2) retrieved content unrelated to the question. We have added the results of these retrieval-control experiments in Tables 9–10, Appendix B.1. We test DeepSeek-V3 and GLM-4.7. Under both conditions, both models consistently answer “I don’t know”, with a leakage rate of 0, rather than relying on pretrained knowledge. This shows that the extracted content comes from the knowledge base, not the model’s parametric knowledge. Therefore, Table 1 measures knowledge base leakage, supporting our claim that SILK is highly effective at extracting RAG knowledge bases.
Regarding the surrogate-system evaluation
To isolate the effect of the knowledge base on RAG performance, we keep all other components fixed when comparing the surrogate and original RAG systems. They use the same backend, retrieval configuration, and answering prompt; only the knowledge base changes. We additionally disable retrieval for the surrogate while retaining the evaluation queries. As shown in Table 17, Appendix B.4, the surrogate then consistently answers “I don’t know,” with ROUGE-L and semantic similarity both below 0.01 across all four datasets. With retrieval enabled, the SILK surrogate exceeds the IKEA surrogate on both measures. This control supports the contribution of the extracted KB to surrogate performance.
Regarding the semantic threshold
We add a semantic branch because implicit extraction can preserve information while changing its wording. We now evaluate five cutoffs on the same 30 human-annotated pairs, keeping the ROUGE-L recall threshold fixed at 0.5. Among the tested cutoffs, 0.65 gives the highest observed accuracy, F1, and κ (Table 18, Appendix B.5).
Regarding annotation coverage
Table 19, Appendix B.5 provides the sample counts for every method–dataset combination. Under No Defense, we randomly sample 30 retrieved chunks from the pooled retrievals of all seven methods across all four datasets, pairing each chunk with its corresponding response. The dataset totals are 7/8/6/9 for HealthCareMagic/FIQA/SciFact/NFCorpus. The method totals are 6/3/3/2/4/6/6 for TGTB/PIDE/DGEA/RAG-Thief/PoR/IKEA/SILK, respectively.