- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
The paper proposes a light method to realize both lexical and semantic conflation over user query and server data corpus in a RAG system, to provide the sensitive information to some extent.
Critical strengths
The paper investigates an important and practically relevant research question: how to protect both client-query privacy and server-owned data confidentiality in RAG systems while improving retrieval efficiency.
Critical weaknesses
- Insufficient explanation of retrieval-quality improvements. The paper reports improved retrieval quality compared with existing methods, particularly Pisces. However, the source of these improvements is not sufficiently analyzed. Since the proposed method changes both candidate selection and relevance scoring, the reported gains may arise from differences in retrieval strategies rather than the proposed privacy-preserving mechanism itself.
- Limited methodological novelty and concerns regarding baseline comparability. The proposed method largely replaces Pisces's computationally expensive, full-corpus PSI-based BM25 retrieval with lightweight keyword-based candidate filtering and binary lexical scoring. Specifically, GateRAG selects up to five query keywords and uses their posting lists to restrict subsequent semantic retrieval. The authors should clarify the methodological contribution beyond simplifying Pisces's retrieval mechanism and provide comparisons under more closely matched retrieval functionality and privacy guarantees.
Questions for authors
- Related to weakness point two. Can authors explain how can current methods improve the privacy guarantee over lightweight baselines in Pisces (would be better if it is supported with any empirical result)?
5 comments
Response to Reviewer XPRi (2/3)
2. Weakness 2 & Question 1: Methodological Novelty and Comparison with Lightweight Pisces
We thank the reviewer for raising these important questions regarding the methodological contribution of GateRAG and the fairness of its comparison with Pisces. We agree that replacing BM25 with a simpler lexical scoring function would not, by itself, constitute a sufficient methodological contribution.
Our main contribution is a new protocol for query-private candidate reduction that acquires lexical evidence substantially more efficiently than Pisces's MultLPSI. GateRAG further reuses this evidence for secure hybrid score fusion, eliminating the need for a separate full-corpus lexical retrieval path. To substantiate these contributions, we have extended our evaluation with a query-privacy leakage experiment for lightweight Pisces, a protocol-level efficiency comparison under a matched keyword budget, and an ablation study isolating the retrieval-quality benefits of hybrid ranking.
(1) Privacy leakage in lightweight Pisces and baseline comparability
Pisces reduces semantic retrieval costs through coarse-to-fine filtering. However, its coarse filtering exposes the resulting query-dependent candidate set to the server, creating a potential channel for inferring information about the client's query.
To quantify this risk, we have added a query-representation recovery experiment in Appendix K. We consider a semi-honest server that attempts to reconstruct the query's SimHash representation from the exposed candidate identities, without observing the plaintext query or its embedding. We also evaluate how well the recovered representation can be used to estimate the original query embedding.
Table R2: Query-representation recovery from server-visible candidate sets in Pisces on PubMedQA.
| Candidates (% of Corpus) | SimHash Bit Accuracy (%) | Exact SimHash Recovery (%) | Embedding Cosine Similarity | |
|---|---|---|---|---|
| 8 | 0.1126 | 100.00 | 100.00 | 0.6002 |
| 16 | 10.2380 | 100.00 | 100.00 | 0.6002 |
| 24 | 93.9154 | 100.00 | 100.00 | 0.6002 |
| 28 | 99.9989 | 80.35 | 31.90 | 0.4371 |
| 32 (prior) | 100.0000 | 59.92 | 0.00 | 0.3607 |
Pisces retains documents whose SimHash representations lie within distance of the query. At , the full corpus is retained, and recovery uses prior-only predictions.
The results demonstrate that Pisces's query-dependent candidate filtering can reveal substantial information about the client's query representation, even when most documents remain in the candidate set. At , where 93.92% of the corpus is retained, the evaluated attack recovers all test-query SimHashes exactly. Even at , where filtering removes only 0.0011% of the corpus, the attacker achieves 80.35% bit accuracy and 31.90% exact SimHash recovery.
The recovered representations also provide information about the original query embeddings. At , the estimated embeddings achieve an average cosine similarity of 0.6002, compared with 0.3607 for the prior-only reference. These results provide empirical evidence that exposing query-dependent candidate identities can weaken query privacy, even without directly revealing query contents.
This finding explains why our main comparison in Section 5.3 uses the full-corpus, fine-only configuration of Pisces. Its lightweight coarse-to-fine configuration reveals query-dependent candidate information to the server, whereas the full-corpus configuration avoids this particular leakage. We therefore use the latter to match GateRAG's principal privacy requirements, including protection of query contents and access patterns from the server and protection of plaintext documents from the client.
Importantly, GateRAG achieves candidate reduction without this privacy compromise. Through OPRF-protected header lookup, PIR over fixed-size posting blocks, and query-independent padding, the server cannot observe the selected keywords or candidate access patterns. Therefore, GateRAG provides a distinct architectural approach to reducing retrieval costs while preserving the server-side query-privacy requirements.
Response to Reviewer XPRi (3/3)
(2) Efficiency beyond lightweight lexical scoring
We also investigate whether Pisces could achieve comparable efficiency simply by adopting the same keyword budget and lightweight lexical scoring as GateRAG.
Beyond the scoring functions, an important difference lies in how the two systems privately obtain lexical information. Pisces employs multi-instance labeled PSI (MultLPSI) to retrieve per-document term frequencies, whereas GateRAG uses OPRF-based header lookup and blocked PIR to acquire the posting information needed for candidate construction and lexical scoring.
To isolate the impact of this design difference, we have extended Appendix D with a comparison that restricts Pisces's MultLPSI to the same five query keywords used by GateRAG. Under this matched keyword budget, GateRAG's lexical information acquisition reduces latency by 94.7% and communication by 99.6% on HotpotQA.
These results demonstrate that reducing the number of query keywords alone does not eliminate the substantial cost difference between the two lexical information acquisition protocols.
To assess the end-to-end impact, we model a lightweight Pisces variant by replacing GateRAG's lexical access cost with the measured cost of five-keyword MultLPSI, while keeping the candidate set, lexical scoring, and all other pipeline stages unchanged. This isolates the efficiency difference between the two lexical evidence acquisition protocols.
Table R3: Protocol-level and estimated end-to-end retrieval costs on HotpotQA.
| Configuration | Lexical Access Latency (s) | Lexical Access Comm. (MB) | End-to-End Latency (s) | End-to-End Comm. (GB) |
|---|---|---|---|---|
| MultLPSI-based variant | 10.47 | 358.69 | 23.43 | 0.80 |
| GateRAG (Blocked PIR) | 0.55 | 1.31 | 13.51 | 0.44 |
| Reduction | 94.7% | 99.6% | 42.3% | 45.0% |
Under this controlled analysis, GateRAG achieves 42.3% lower end-to-end retrieval latency and 45.0% lower communication. These findings show that GateRAG's efficiency gains cannot be explained solely by using fewer query keywords, smaller candidates, or simpler lexical scores. The private postings retrieval design itself contributes substantially to reducing cryptographic overhead.
(3) Retrieval quality through secure hybrid fusion
Finally, GateRAG also differs from Pisces in how the acquired relevance information is used. Pisces independently selects documents from its semantic and lexical retrieval paths, while GateRAG securely fuses both relevance scores over the reduced candidate set before final top- selection. This allows documents with complementary semantic and lexical evidence to be ranked jointly, rather than relying on either signal independently.
As demonstrated in our new ablation study in Section 5.2 (Table R1 above), replacing independent semantic and lexical selection with hybrid score fusion improves Recall@10 by 1.50, 2.29, and 13.57 percentage points on PubMedQA, HotpotQA, and FEVER, respectively. Furthermore, the privacy-matched comparison in Section 5.3 shows that GateRAG's quality-oriented configuration achieves 1.70–20.64 percentage points higher recall than Pisces, while substantially reducing latency and communication.
Taken together, these results address the reviewer's central concern that GateRAG merely simplifies Pisces's expensive lexical retrieval. The fundamental distinction is not simply the use of fewer keywords or a lightweight scoring function, but how candidate reduction is achieved under strict privacy requirements. Pisces's lightweight coarse-to-fine retrieval reduces computation by exposing query-dependent candidate identities to the server, while GateRAG addresses this by enabling candidate reduction without revealing query-dependent access patterns to the server.
Beyond this privacy advantage over lightweight Pisces, GateRAG's lexical information acquisition protocol is substantially more efficient than Pisces's MultLPSI, even under the same keyword budget. Furthermore, its secure hybrid score fusion achieves additional retrieval-quality improvements beyond independent semantic and lexical selection. These results demonstrate that GateRAG introduces a distinct architecture for efficient private hybrid retrieval. We sincerely appreciate the reviewer for encouraging us to clarify these contributions and substantiate them with additional empirical evidence.
We sincerely appreciate the reviewer's constructive feedback and careful consideration of our work. The concerns raised have led us to strengthen the analysis of retrieval quality, investigate the query-privacy risks of Pisces's lightweight candidate filtering, and clarify the methodological contributions of GateRAG. We hope that these additional experiments and revisions address the reviewer's concerns and provide a clearer understanding of our approach. We would be happy to address any remaining questions.
Response to Reviewer XPRi (1/3)
Dear Reviewer XPRi,
We sincerely thank the reviewer for the careful evaluation of our work and for recognizing the practical importance of protecting both client queries and server-owned documents in RAG systems. We also greatly appreciate the reviewer's concerns regarding the sources of retrieval-quality improvements, methodological novelty, and baseline comparability. These comments have encouraged us to examine the contributions of individual retrieval components more carefully and to clarify the distinctions between GateRAG and existing approaches. We provide detailed responses below.
1. Weakness 1: Analysis of Retrieval-Quality Improvements
We thank the reviewer for highlighting the need to better explain the sources of GateRAG's retrieval-quality improvements. We agree that our original evaluation did not sufficiently distinguish the effects of candidate selection and relevance scoring. In particular, improvements over Pisces alone do not establish which components are responsible for the higher recall.
To address this concern, we have added an incremental ablation study, evaluating the individual contributions of keyword gating, lexical evidence reuse, and hybrid score fusion.
Incremental ablation study. Starting from full-corpus semantic retrieval, we progressively introduce the three components of GateRAG. First, keyword gating restricts semantic ranking to a reduced candidate set. Next, the lexical scores obtained during gating are reused to independently select k/2 documents from each of the semantic and lexical rankings. Finally, GateRAG replaces this independent selection with unified hybrid ranking, combining semantic and lexical relevance scores before selecting the final top-k documents.
Importantly, the last two configurations use the same candidate set and lexical scores. Their comparison therefore isolates the benefit of hybrid score fusion from the effects of candidate selection and lexical evidence acquisition.
Table R1: Incremental ablation of GateRAG components. Recall@10 (%) is reported under P0.995 configurations.
Parenthesized values indicate improvements over the preceding stage in percentage points.
Sources of retrieval-quality improvements. The results show that keyword gating alone yields modest recall improvements of 1.10–2.99 percentage points, suggesting that lexical filtering can help exclude semantically similar but irrelevant documents. Reusing the gating-derived lexical evidence provides further improvements, particularly on HotpotQA and FEVER.
Most importantly, hybrid score fusion consistently outperforms independent semantic and lexical selection, improving Recall@10 by an additional 1.50, 2.29, and 13.57 percentage points on PubMedQA, HotpotQA, and FEVER, respectively. Since these two configurations share the same candidates and lexical evidence, the improvements can be attributed to how the relevance signals are combined rather than differences in candidate selection.
Relation to Pisces. This analysis also helps explain the retrieval-quality difference between GateRAG and Pisces. Pisces uses separate semantic and lexical retrieval paths, whereas GateRAG combines both relevance signals before final top-k selection. This allows documents with complementary semantic and lexical evidence to be ranked jointly, including those that may not rank sufficiently highly in either individual retrieval path. Our additional analysis of retrieved gold-document distributions in Appendix I.2 further examines this distinction.
We would also like to clarify that we do not attribute the recall improvements to the cryptographic privacy mechanisms themselves. Rather, the improvements arise from GateRAG's retrieval strategy, particularly its candidate selection and integration of lexical and semantic relevance. Our privacy-preserving protocol makes it possible to execute this strategy efficiently while protecting query-dependent access patterns from the server and keeping plaintext documents hidden from the client.
We have incorporated this ablation study into Section 5.2 to more clearly distinguish the contributions of individual retrieval components. We appreciate the reviewer's suggestion, which has helped us provide a more precise explanation of why GateRAG achieves higher retrieval quality.