acceptodds
Under review as a conference paper at ICLR 2027

GateRAG: Keyword-Gated Hybrid Retrieval for Efficient Private RAG

Abstract

While retrieval-augmented generation (RAG) significantly enhances the capabilities of large language models, protecting both sensitive user queries and server-owned corpora remains challenging. Existing private RAG systems either incur prohibitive cryptographic overheads for strict privacy, or sacrifice substantial confidentiality to achieve practical efficiency. In this paper, we present GateRAG, a private RAG retrieval framework that considerably reduces cryptographic cost without revealing query-dependent information to the server or server-owned plaintext documents to the client, while also improving retrieval quality. GateRAG achieves this through private keyword gating, which prunes the search space before expensive secure computation. The keyword matches obtained during gating are reused as lexical relevance signals and securely fused with semantic similarity over the reduced candidate set. This hybrid fusion exploits complementary semantic and lexical evidence to improve retrieval quality, while reusing the gating-derived lexical signals avoids a separate costly lexical computation. Experiments on three standard benchmarks (PubMedQA, HotpotQA, and FEVER) demonstrate that GateRAG substantially improves the retrieval quality–efficiency tradeoff over prior private RAG systems. A single parameter controls this tradeoff, enabling different operating points along the quality–efficiency spectrum. At a quality-oriented operating point, GateRAG improves recall by 1.70–20.64 percentage points while reducing latency by 53.3–96.0% and communication by 89.5–98.6% compared with Pisces (Liang et al., 2026), the only prior system matching our privacy guarantees. At a more efficiency-oriented operating point, the latency and communication reductions reach 91.9–97.8% and 97.3–99.2%, respectively. GateRAG therefore provides a promising foundation for efficient private RAG.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
privacy-preserving machine learning, retrieval-augmented generation, cryptography

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_XPRiOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper proposes a light method to realize both lexical and semantic conflation over user query and server data corpus in a RAG system, to provide the sensitive information to some extent.

Critical strengths

The paper investigates an important and practically relevant research question: how to protect both client-query privacy and server-owned data confidentiality in RAG systems while improving retrieval efficiency.

Critical weaknesses
  1. Insufficient explanation of retrieval-quality improvements. The paper reports improved retrieval quality compared with existing methods, particularly Pisces. However, the source of these improvements is not sufficiently analyzed. Since the proposed method changes both candidate selection and relevance scoring, the reported gains may arise from differences in retrieval strategies rather than the proposed privacy-preserving mechanism itself.
  2. Limited methodological novelty and concerns regarding baseline comparability. The proposed method largely replaces Pisces's computationally expensive, full-corpus PSI-based BM25 retrieval with lightweight keyword-based candidate filtering and binary lexical scoring. Specifically, GateRAG selects up to five query keywords and uses their posting lists to restrict subsequent semantic retrieval. The authors should clarify the methodological contribution beyond simplifying Pisces's retrieval mechanism and provide comparisons under more closely matched retrieval functionality and privacy guarantees.
Questions for authors
  1. Related to weakness point two. Can authors explain how can current methods improve the privacy guarantee over lightweight baselines in Pisces (would be better if it is supported with any empirical result)?
5 comments
AuthorsOpenReview ↗

Response to Reviewer XPRi (1/3)

Dear Reviewer XPRi,

We sincerely thank the reviewer for the careful evaluation of our work and for recognizing the practical importance of protecting both client queries and server-owned documents in RAG systems. We also greatly appreciate the reviewer's concerns regarding the sources of retrieval-quality improvements, methodological novelty, and baseline comparability. These comments have encouraged us to examine the contributions of individual retrieval components more carefully and to clarify the distinctions between GateRAG and existing approaches. We provide detailed responses below.

1. Weakness 1: Analysis of Retrieval-Quality Improvements

We thank the reviewer for highlighting the need to better explain the sources of GateRAG's retrieval-quality improvements. We agree that our original evaluation did not sufficiently distinguish the effects of candidate selection and relevance scoring. In particular, improvements over Pisces alone do not establish which components are responsible for the higher recall.

To address this concern, we have added an incremental ablation study, evaluating the individual contributions of keyword gating, lexical evidence reuse, and hybrid score fusion.

Incremental ablation study. Starting from full-corpus semantic retrieval, we progressively introduce the three components of GateRAG. First, keyword gating restricts semantic ranking to a reduced candidate set. Next, the lexical scores obtained during gating are reused to independently select k/2k/2 documents from each of the semantic and lexical rankings. Finally, GateRAG replaces this independent selection with unified hybrid ranking, combining semantic and lexical relevance scores before selecting the final top-kk documents.

Importantly, the last two configurations use the same candidate set and lexical scores. Their comparison therefore isolates the benefit of hybrid score fusion from the effects of candidate selection and lexical evidence acquisition.

Table R1: Incremental ablation of GateRAG components. Recall@10 (%) is reported under P0.995P_{0.995} configurations.

MethodPubMedQAHotpotQAFEVER
Baseline (Semantic-only)93.1042.5863.35
+ Keyword Gating94.20 (+1.10)45.13 (+2.55)66.34 (+2.99)
+ Lexical Score Reuse96.50 (+2.30)51.16 (+6.03)76.10 (+9.76)
+ Hybrid Score Fusion (Ours)98.00 (+1.50)53.45 (+2.29)89.67 (+13.57)

Parenthesized values indicate improvements over the preceding stage in percentage points.

Sources of retrieval-quality improvements. The results show that keyword gating alone yields modest recall improvements of 1.10–2.99 percentage points, suggesting that lexical filtering can help exclude semantically similar but irrelevant documents. Reusing the gating-derived lexical evidence provides further improvements, particularly on HotpotQA and FEVER.

Most importantly, hybrid score fusion consistently outperforms independent semantic and lexical selection, improving Recall@10 by an additional 1.50, 2.29, and 13.57 percentage points on PubMedQA, HotpotQA, and FEVER, respectively. Since these two configurations share the same candidates and lexical evidence, the improvements can be attributed to how the relevance signals are combined rather than differences in candidate selection.

Relation to Pisces. This analysis also helps explain the retrieval-quality difference between GateRAG and Pisces. Pisces uses separate semantic and lexical retrieval paths, whereas GateRAG combines both relevance signals before final top-k selection. This allows documents with complementary semantic and lexical evidence to be ranked jointly, including those that may not rank sufficiently highly in either individual retrieval path. Our additional analysis of retrieved gold-document distributions in Appendix I.2 further examines this distinction.

We would also like to clarify that we do not attribute the recall improvements to the cryptographic privacy mechanisms themselves. Rather, the improvements arise from GateRAG's retrieval strategy, particularly its candidate selection and integration of lexical and semantic relevance. Our privacy-preserving protocol makes it possible to execute this strategy efficiently while protecting query-dependent access patterns from the server and keeping plaintext documents hidden from the client.

We have incorporated this ablation study into Section 5.2 to more clearly distinguish the contributions of individual retrieval components. We appreciate the reviewer's suggestion, which has helped us provide a more precise explanation of why GateRAG achieves higher retrieval quality.

AuthorsOpenReview ↗

Response to Reviewer XPRi (2/3)

2. Weakness 2 & Question 1: Methodological Novelty and Comparison with Lightweight Pisces

We thank the reviewer for raising these important questions regarding the methodological contribution of GateRAG and the fairness of its comparison with Pisces. We agree that replacing BM25 with a simpler lexical scoring function would not, by itself, constitute a sufficient methodological contribution.

Our main contribution is a new protocol for query-private candidate reduction that acquires lexical evidence substantially more efficiently than Pisces's MultLPSI. GateRAG further reuses this evidence for secure hybrid score fusion, eliminating the need for a separate full-corpus lexical retrieval path. To substantiate these contributions, we have extended our evaluation with a query-privacy leakage experiment for lightweight Pisces, a protocol-level efficiency comparison under a matched keyword budget, and an ablation study isolating the retrieval-quality benefits of hybrid ranking.

(1) Privacy leakage in lightweight Pisces and baseline comparability

Pisces reduces semantic retrieval costs through coarse-to-fine filtering. However, its coarse filtering exposes the resulting query-dependent candidate set to the server, creating a potential channel for inferring information about the client's query.

To quantify this risk, we have added a query-representation recovery experiment in Appendix K. We consider a semi-honest server that attempts to reconstruct the query's SimHash representation from the exposed candidate identities, without observing the plaintext query or its embedding. We also evaluate how well the recovered representation can be used to estimate the original query embedding.

Table R2: Query-representation recovery from server-visible candidate sets in Pisces on PubMedQA.

ttCandidates (% of Corpus)SimHash Bit Accuracy (%)Exact SimHash Recovery (%)Embedding Cosine Similarity
80.1126100.00100.000.6002
1610.2380100.00100.000.6002
2493.9154100.00100.000.6002
2899.998980.3531.900.4371
32 (prior)100.000059.920.000.3607

Pisces retains documents whose SimHash representations lie within distance tt of the query. At t=32t=32, the full corpus is retained, and recovery uses prior-only predictions.

The results demonstrate that Pisces's query-dependent candidate filtering can reveal substantial information about the client's query representation, even when most documents remain in the candidate set. At t=24t=24, where 93.92% of the corpus is retained, the evaluated attack recovers all test-query SimHashes exactly. Even at t=28t=28, where filtering removes only 0.0011% of the corpus, the attacker achieves 80.35% bit accuracy and 31.90% exact SimHash recovery.

The recovered representations also provide information about the original query embeddings. At t=24t=24, the estimated embeddings achieve an average cosine similarity of 0.6002, compared with 0.3607 for the prior-only reference. These results provide empirical evidence that exposing query-dependent candidate identities can weaken query privacy, even without directly revealing query contents.

This finding explains why our main comparison in Section 5.3 uses the full-corpus, fine-only configuration of Pisces. Its lightweight coarse-to-fine configuration reveals query-dependent candidate information to the server, whereas the full-corpus configuration avoids this particular leakage. We therefore use the latter to match GateRAG's principal privacy requirements, including protection of query contents and access patterns from the server and protection of plaintext documents from the client.

Importantly, GateRAG achieves candidate reduction without this privacy compromise. Through OPRF-protected header lookup, PIR over fixed-size posting blocks, and query-independent padding, the server cannot observe the selected keywords or candidate access patterns. Therefore, GateRAG provides a distinct architectural approach to reducing retrieval costs while preserving the server-side query-privacy requirements.

AuthorsOpenReview ↗

Response to Reviewer XPRi (3/3)

(2) Efficiency beyond lightweight lexical scoring

We also investigate whether Pisces could achieve comparable efficiency simply by adopting the same keyword budget and lightweight lexical scoring as GateRAG.

Beyond the scoring functions, an important difference lies in how the two systems privately obtain lexical information. Pisces employs multi-instance labeled PSI (MultLPSI) to retrieve per-document term frequencies, whereas GateRAG uses OPRF-based header lookup and blocked PIR to acquire the posting information needed for candidate construction and lexical scoring.

To isolate the impact of this design difference, we have extended Appendix D with a comparison that restricts Pisces's MultLPSI to the same five query keywords used by GateRAG. Under this matched keyword budget, GateRAG's lexical information acquisition reduces latency by 94.7% and communication by 99.6% on HotpotQA.

These results demonstrate that reducing the number of query keywords alone does not eliminate the substantial cost difference between the two lexical information acquisition protocols.

To assess the end-to-end impact, we model a lightweight Pisces variant by replacing GateRAG's lexical access cost with the measured cost of five-keyword MultLPSI, while keeping the candidate set, lexical scoring, and all other pipeline stages unchanged. This isolates the efficiency difference between the two lexical evidence acquisition protocols.

Table R3: Protocol-level and estimated end-to-end retrieval costs on HotpotQA.

ConfigurationLexical Access Latency (s)Lexical Access Comm. (MB)End-to-End Latency (s)End-to-End Comm. (GB)
MultLPSI-based variant10.47358.6923.430.80
GateRAG (Blocked PIR)0.551.3113.510.44
Reduction94.7%99.6%42.3%45.0%

Under this controlled analysis, GateRAG achieves 42.3% lower end-to-end retrieval latency and 45.0% lower communication. These findings show that GateRAG's efficiency gains cannot be explained solely by using fewer query keywords, smaller candidates, or simpler lexical scores. The private postings retrieval design itself contributes substantially to reducing cryptographic overhead.

(3) Retrieval quality through secure hybrid fusion

Finally, GateRAG also differs from Pisces in how the acquired relevance information is used. Pisces independently selects documents from its semantic and lexical retrieval paths, while GateRAG securely fuses both relevance scores over the reduced candidate set before final top-kk selection. This allows documents with complementary semantic and lexical evidence to be ranked jointly, rather than relying on either signal independently.

As demonstrated in our new ablation study in Section 5.2 (Table R1 above), replacing independent semantic and lexical selection with hybrid score fusion improves Recall@10 by 1.50, 2.29, and 13.57 percentage points on PubMedQA, HotpotQA, and FEVER, respectively. Furthermore, the privacy-matched comparison in Section 5.3 shows that GateRAG's quality-oriented configuration achieves 1.70–20.64 percentage points higher recall than Pisces, while substantially reducing latency and communication.

Taken together, these results address the reviewer's central concern that GateRAG merely simplifies Pisces's expensive lexical retrieval. The fundamental distinction is not simply the use of fewer keywords or a lightweight scoring function, but how candidate reduction is achieved under strict privacy requirements. Pisces's lightweight coarse-to-fine retrieval reduces computation by exposing query-dependent candidate identities to the server, while GateRAG addresses this by enabling candidate reduction without revealing query-dependent access patterns to the server.

Beyond this privacy advantage over lightweight Pisces, GateRAG's lexical information acquisition protocol is substantially more efficient than Pisces's MultLPSI, even under the same keyword budget. Furthermore, its secure hybrid score fusion achieves additional retrieval-quality improvements beyond independent semantic and lexical selection. These results demonstrate that GateRAG introduces a distinct architecture for efficient private hybrid retrieval. We sincerely appreciate the reviewer for encouraging us to clarify these contributions and substantiate them with additional empirical evidence.


We sincerely appreciate the reviewer's constructive feedback and careful consideration of our work. The concerns raised have led us to strengthen the analysis of retrieval quality, investigate the query-privacy risks of Pisces's lightweight candidate filtering, and clarify the methodological contributions of GateRAG. We hope that these additional experiments and revisions address the reviewer's concerns and provide a clearer understanding of our approach. We would be happy to address any remaining questions.

Reviewer_G6qWOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents GateRAG, a privacy-preserving hybrid retrieval framework designed to reduce the computational and communication overhead of private retrieval-augmented generation. The proposed method introduces private keyword gating to construct a reduced candidate set before performing expensive cryptographic retrieval and ranking operations. Specifically, GateRAG employs OPRF-protected keyword lookup, private information retrieval (PIR), and secure multiparty computation (MPC) to protect client queries and server-owned documents. The keyword matches obtained during candidate selection are further reused as lexical relevance signals and securely combined with semantic similarity scores for hybrid ranking.

Experiments on PubMedQA, HotpotQA, and FEVER demonstrate substantial improvements in retrieval efficiency and recall compared with existing private RAG systems. In particular, GateRAG achieves significant reductions in latency and communication overhead relative to Pisces, while maintaining or improving retrieval quality. The paper also investigates the accuracy-efficiency tradeoff controlled by the fixed posting-block retrieval budget.

Critical strengths
  1. The paper addresses an important practical challenge in private RAG: reducing the substantial overhead of cryptographic retrieval while protecting both user queries and server-owned documents. The proposed keyword-gating mechanism reduces the candidate search space before expensive secure computations, thereby addressing a major computational bottleneck in existing approaches. Unlike methods that achieve efficiency by revealing query-dependent access patterns or plaintext documents, GateRAG provides stronger privacy protection under its specified semi-honest threat model.

  2. A useful aspect of the proposed design is that keyword matches serve two purposes: reducing the candidate set and providing lexical relevance signals for subsequent hybrid ranking. This avoids a separate full-corpus lexical scoring procedure, which can be particularly expensive in privacy-preserving retrieval. The combination of OPRF, PIR, and MPC provides a practical architectural solution for integrating lexical and semantic retrieval under cryptographic protection.

Critical weaknesses
  1. Although GateRAG protects plaintext documents, its keyword-gating mechanism reveals posting lists to the client. Appendix F analyzes the cost of reconstructing the entire inverted index, but this is a much stronger objective than targeted information inference. An adversarial client may infer sensitive keyword co-occurrence or document properties through a small number of adaptive queries. The authors should evaluate such targeted inference risks to better characterize the practical limits of their server-data confidentiality guarantees.

  2. GateRAG performs semantic ranking only on documents selected through keyword matching. Consequently, semantically relevant documents without matching query keywords are permanently excluded, regardless of their semantic similarity. This limitation may be particularly significant for paraphrased queries, terminology mismatches, and multi-hop reasoning. Although the paper reports strong aggregate Recall@10, it lacks a detailed analysis of candidate-recall failures. The authors should evaluate retrieval performance under low lexical overlap and quantify how often relevant documents are eliminated during keyword gating.

Questions for authors

see the weakness

2 comments
AuthorsOpenReview ↗

Response to Reviewer G6qW (1/2)

Dear Reviewer G6qW,

We sincerely thank the reviewer for the thoughtful assessment of our work. We are particularly grateful for the positive evaluation of our keyword gating mechanism and the reuse of lexical evidence for private hybrid ranking. We also greatly appreciate the reviewer’s important concerns regarding potential information leakage through posting lists and the limitations of keyword gating under low lexical overlap. These comments have helped us identify aspects of GateRAG that deserved more thorough investigation. We provide our responses below.

1. Weakness 1: Targeted Inference from Posting-List Leakage

We thank the reviewer for highlighting an important limitation of our original security analysis. We agree that reconstructing the entire inverted index is a substantially stronger objective than inferring sensitive information about individual documents. A client may learn sensitive document properties from a small number of keyword observations without reconstructing the complete index. Therefore, the high extraction costs reported in our original Appendix F should not be interpreted as evidence that targeted inference is infeasible.

To address this concern, we have added a targeted single-document keyword-recovery experiment in Appendix F.4. We consider an adaptive client that selects a target document identified by an opaque ID from an observed posting list and attempts to discover additional keywords associated with that document. The experiment follows the protocol's posting-budget constraints and compares two probing strategies. The first uses Qwen2.5-7B-Instruct to adaptively propose keywords based on previous observations, while the second selects keywords randomly.

Table R1: Single-document keyword recovery (%) on PubMedQA.

SettingQ=1Q=1Q=10Q=10Q=20Q=20Q=200Q=200
P=16P=16, Random1.061.091.171.85
P=16P=16, LLM-adaptive1.061.381.633.01
P=68P=68, Random1.141.201.322.63
P=68P=68, LLM-adaptive1.142.453.417.45
QQ denotes the number of protocol executions.

The results confirm that targeted information leakage can occur even with relatively small query budgets. At P=68P=68, LLM-adaptive probing recovers an average of 3.41% of each target's indexed keywords within 20 protocol executions. Increasing the budget to 200 executions raises recovery to 7.45%, compared with 2.63% for random probing. Although adaptive probing is more effective than random selection, it still recovers only a limited fraction of the indexed keywords under the evaluated conditions. Reducing the posting budget to P=16P=16 further lowers LLM-adaptive recovery to 3.01% at 200 executions. These results suggest that a tighter posting budget can reduce client-visible keyword information, although this comes with a trade-off in retrieval quality.

Importantly, we agree that even partial keyword leakage may be sufficient to infer sensitive document properties. A small number of keywords associated with the same document could reveal meaningful relationships regardless of the overall recovery percentage. We recognize this as an important limitation of GateRAG's keyword-gating mechanism.

For applications requiring stronger protection against such inference, additional safeguards may be necessary. Authentication and fine-grained access control could provide complementary protection by restricting which users are permitted to access sensitive documents or perform certain keyword searches. For example, a medical RAG service could restrict access to sensitive records to authorized medical professionals. Prior work has explored combining encrypted keyword search with attribute-based access control for electronic health records [1], suggesting a relevant direction for extending GateRAG. Incorporating such mechanisms would require an authorization design that preserves query privacy and is beyond the scope of our current protocol. We also discuss periodic identifier-permutation refresh and rate limiting in Appendix F.5 as possible measures to constrain information accumulation across repeated executions.

Following the reviewer's suggestion, we have revised Appendix F to clearly distinguish exhaustive index extraction from targeted keyword inference. The new analysis provides a more realistic characterization of the information exposed through posting-list observations while explicitly acknowledging the remaining risks of sensitive property inference. We sincerely appreciate the reviewer for encouraging us to investigate this distinction more carefully.

Reference

[1] W. Li, L. Xu, Y. Wen, and F. Zhang. Conjunctive Multi-Key Searchable Encryption with Attribute-Based Access Control for EHR Systems. Computer Standards & Interfaces, vol. 82, 2022.

AuthorsOpenReview ↗

Response to Reviewer G6qW (2/2)

2. Weakness 2: Candidate-Recall Limitations under Low Lexical Overlap

We thank the reviewer for identifying an important limitation of keyword-based candidate selection. We agree that keyword gating can exclude relevant documents that do not contain any of the selected query keywords, regardless of their semantic similarity. This is an inherent limitation of the gating mechanism, and our original evaluation did not sufficiently characterize how often it occurs or how much it affects retrieval quality. We appreciate the reviewer's suggestion to examine these failures more closely.

To address this concern, we have added a detailed analysis of candidate coverage and retrieval outcomes in Appendix I.1 of the revised manuscript. The analysis separates documents lost during keyword gating from those retained as candidates but missed during ranking. We also investigate the relationship between lexical overlap and candidate coverage, particularly for multi-hop retrieval.

Candidate coverage and retrieval outcomes. We classify gold documents into four categories based on whether they are successfully retrieved (Retrieved), retained as candidates but not selected during ranking (Ranking Failure), excluded due to the fixed posting-block budget (Budget Truncation), or excluded because none of the selected query keywords match (Lexical Mismatch). The following table reports the results, where all percentages are averaged across queries.

Table R2: Distribution of gold-document retrieval outcomes (%).

OutcomePubMedQAHotpotQAFEVER
Candidate Recall99.6080.4598.47
Retrieved98.0053.4589.67
Ranking Failure1.6027.008.80
Budget Truncation0.206.401.08
Lexical Mismatch0.2013.150.45

Candidate recall measures the fraction of gold documents retained after keyword gating, corresponding to the sum of Retrieved and Ranking Failure. Keyword gating retains 99.60% and 98.47% of gold documents on PubMedQA and FEVER, respectively, with lexical mismatch accounting for only 0.20% and 0.45%. In contrast, HotpotQA exhibits a more pronounced limitation, with 13.15% of gold documents excluded due to lexical mismatch.

Impact of low lexical overlap and multi-hop retrieval. HotpotQA is particularly relevant to the reviewer's concern because its questions require retrieving supporting evidence for multi-hop reasoning. Its average query–gold lexical overlap is only 36.23%, compared with 75.54% for PubMedQA and 79.14% for FEVER. This substantially lower overlap is accompanied by a higher lexical-mismatch rate, indicating that keyword gating can indeed limit candidate coverage when relevant documents share relatively few terms with the query. We acknowledge that this limitation is particularly important for multi-hop retrieval and may also arise with paraphrased queries or terminology mismatches.

Recoverability through full-corpus semantic retrieval. To further understand the impact of lexical mismatch, we examine whether the excluded gold documents would have been retrieved by full-corpus semantic Top-10 search. Among gold documents excluded specifically due to lexical mismatch, 50.00%, 4.50%, and 0.00% appear in the semantic Top-10 on PubMedQA, HotpotQA, and FEVER, respectively. In particular, although lexical mismatch excludes 13.15% of gold documents on HotpotQA, only 4.50% of those excluded documents would have appeared in the evaluated full-corpus semantic Top-10 results. These findings show that some relevant documents excluded by keyword gating could have been recovered through unrestricted semantic search, while others would not have appeared in the semantic Top-10 even without gating. This does not eliminate the underlying limitation, but it helps characterize its practical impact under the evaluated settings.

We have incorporated these analyses into Appendix I.1 to provide a clearer understanding of how keyword gating affects candidate coverage and retrieval accuracy. We thank the reviewer for suggesting this additional evaluation.


We sincerely appreciate the reviewer's constructive feedback and encouraging assessment of our work. We hope the additional experiments and manuscript revisions address the reviewer's concerns and provide a more complete understanding of the proposed framework. We would be happy to clarify any remaining questions.

Reviewer_BxJpOpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents GateRAG, a private retrieval-augmented generation protocol with dual privacy goals: hide the contents of the client's query from a server hosting a search corpus and hide the clients of the plaintext documents from the client. The primary contribution of the paper is the insight that existing private hybrid search protocols (which combine lexical and semantic matching) are inefficient. Thus, GateRAG proposes a novel keyword gating mechanism to prune the search space considerably. GateRAG then uses a secret sharing protocol between the client and server to perform private search and then multi-party computation (MPC) LLM inference to compute the final RAG output. The authors validate GateRAG on three standard retrieval benchmarks where they report up to 90% latency and communication reductions over previously proposed private RAG systems.

Critical strengths
  1. Keyword gating for reducing the search space is, to my knowledge, a novel technique and a powerful insight that could influence future work in the domain of private search.
Critical weaknesses
  1. The most critical weakness of the paper in its current form, in my opinion, is that it positions itself as a private RAG solution but does not evaluate the end-to-end performance of private RAG. The authors note in the paper that the evaluation of the LLM generation component of RAG is out of scope, but I do not find this explanation satisfactory because of the security model used in the paper. The authors use multi-party computation (MPC) so private LLM inference would require MPC LLM inference or homomorphic encryption, both of which can be extremely slow to my understanding. Thus, even if GateRAG saves a considerable amount of latency for the private search step, these efficiency gains may be completely lost in the LLM inference step because of this MPC model. Other systems that do not require hiding the documents from the client simply host the LLM locally on the client and do not run into this problem. In short, without measuring the LLM inference cost, we cannot understand the performance of GateRAG as an end-to-end solution. I think this is a critical issue.

The authors could alternatively position their work as a private search protocol instead of private RAG and thus avoid having to measure the LLM inference component, but this alternative may not make sense since one of the goals of GateRAG is hiding the plaintext documents from the client.

  1. Secondly, I think the presentation of the paper is a bit confusing because several of the baseline private RAG systems that the authors evaluate against do not adopt the same privacy model of hiding both the query contents from the server and the documents from the client. It is unclear from reading the paper what the cost of this more stringent privacy model is, and I think it would be useful for the authors to better distinguish the settings of each system including GateRAG.
Questions for authors
  1. Can you provide an intuitive explanation of why GateRAG guarantees that clients do not learn the contents of the server's plaintext documents. If a client issues a query such as "Golf courses in California" and receives a relevant response from the LLM, then the client knows that the server contains documents including content on golf and California. Does this not leak information about the documents to the client? Or is the privacy guarantee that the client does not learn which specific document contains the relevant information? If that is the case, why is this an interesting or relevant privacy model to study or require?

  2. What is the inference cost of MPC LLM inference? Is it practical to run or would it be prohibitive relative to the cost of performing the search?

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to Reviewer BxJp (2/3)

2. Weakness 2: Clarifying Privacy Requirements and Baseline Comparisons

We sincerely appreciate the reviewer for pointing out that our original presentation did not sufficiently distinguish the privacy requirements of the evaluated systems. We agree that presenting baselines with different document-protection guarantees together made it difficult to interpret the comparisons and understand the implications of GateRAG's stronger privacy requirements.

To address this concern, we have clarified the evaluation configuration of each baseline in Section 5.1 and added a column that explains each setting to the comparison tables (Table 3, 4). More importantly, we have reorganized Section 5.3 into two explicitly separated comparisons, with distinct tables and discussions, based on whether plaintext documents are protected from the client.

In both comparisons, query contents and access patterns are required to remain hidden from the server. The distinction is whether the baseline additionally protects plaintext documents from the client. GateRAG maintains both protections in every comparison.

(1) Comparison with a document-protecting baseline

Table R2: Retrieval efficiency and recall comparison with Pisces.

DatasetMethodτ\boldsymbol{\tau}LatencyComm.Recall
PubMedQA (0.274M)Pisces–125.58 s9.50 GB96.30%
GateRAG0.955.81 s0.26 GB93.50%
GateRAG0.99518.38 s1.00 GB98.00%
HotpotQA (1.795M)Pisces–628.14 s57.82 GB51.01%
GateRAG0.9513.51 s0.44 GB51.88%
GateRAG0.99525.35 s0.81 GB53.45%
FEVER (5.396M)Pisces–0.75 h195.74 GB69.03%
GateRAG0.95218.32 s3.29 GB85.54%
GateRAG0.9950.35 h15.57 GB89.67%

τ\tau denotes the quality-retention target for operating-point selection.

This is our primary privacy-matched comparison, where both GateRAG and Pisces are required to protect query contents, access patterns, and plaintext documents. We evaluate Pisces using its full-corpus configuration, which preserves these protections without revealing candidate identities to the server.

As shown in Table R2, GateRAG achieves substantially lower retrieval latency and communication across all three datasets and both operating points. The two gating thresholds illustrate the efficiency–recall trade-off: although τ=0.95\tau=0.95 yields slightly lower recall than Pisces on PubMedQA, τ=0.995\tau=0.995 achieves higher recall than Pisces across all three benchmarks while retaining substantial efficiency improvements. This comparison therefore directly evaluates GateRAG's efficiency under the same document-confidentiality requirements as the baseline.

(2) Comparison with document-revealing baselines

Table R3: Retrieval efficiency and recall comparison with document-revealing baselines on FEVER.

DocsMethodτ\boldsymbol{\tau}LatencyComm.Recall
ExposedRemoteRAG–590.84 h10.87 GB63.35%
ExposedShieldRAG–534.32 h815.03 GB63.35%
ExposedPIR-RAG0.95/0.9950.39 h52.02 GB63.35%
HiddenGateRAG0.95218.32 s3.29 GB85.54%
HiddenGateRAG0.9950.35 h15.57 GB89.67%

We separately compare GateRAG with RemoteRAG, ShieldRAG, and PIR-RAG on FEVER, our largest benchmark. Unlike Pisces, these baselines reveal retrieved plaintext documents to the client. This comparison therefore serves a different purpose: it examines whether GateRAG can remain efficient relative to systems with less restrictive document-privacy requirements, while itself continuing to protect plaintext documents.

To maintain the same query-privacy and access-pattern requirements, RemoteRAG and ShieldRAG are evaluated using their full-corpus configurations, while PIR-RAG retains its private cluster-based candidate reduction. At τ=0.95\tau=0.95, GateRAG achieves lower latency and communication, as well as higher recall, than all three document-revealing baselines. At τ=0.995\tau=0.995, GateRAG further improves recall and continues to achieve lower retrieval latency than all three baselines, although its communication cost exceeds that of RemoteRAG.

These results show that GateRAG remains competitive even relative to systems that do not enforce plaintext-document protection. By privately reducing the candidate set and reusing the lexical signal obtained through keyword gating, GateRAG reduces the amount of expensive secure computation, achieving lower retrieval latency than all three baselines at both operating points despite its additional document-confidentiality requirement.

We believe that separating these two comparisons makes the privacy assumptions and the purpose of each evaluation substantially clearer.

AuthorsOpenReview ↗

Gentle Reminder Before the Discussion Deadline

Dear Reviewers,

We sincerely appreciate your thoughtful reviews and constructive feedback. With the discussion period ending in two days, we would appreciate your feedback on our responses and revisions, so that we could address any remaining concerns before the deadline.

If our additional evidence and clarifications have resolved your concerns, we would be grateful if you could consider updating your evaluation accordingly.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.