GateRAG: Keyword-Gated Hybrid Retrieval for Efficient Private RAG

While retrieval-augmented generation (RAG) significantly enhances the capabilities of large language models, protecting both sensitive user queries and server-owned corpora remains challenging. Existing private RAG systems either incur prohibitive cryptographic overheads for strict privacy, or sacrifice substantial confidentiality to achieve practical efficiency. In this paper, we present GateRAG, a private RAG retrieval framework that considerably reduces cryptographic cost without revealing query-dependent information to the server or server-owned plaintext documents to the client, while also improving retrieval quality. GateRAG achieves this through private keyword gating, which prunes the search space before expensive secure computation. The keyword matches obtained during gating are reused as lexical relevance signals and securely fused with semantic similarity over the reduced candidate set. This hybrid fusion exploits complementary semantic and lexical evidence to improve retrieval quality, while reusing the gating-derived lexical signals avoids a separate costly lexical computation. Experiments on three standard benchmarks (PubMedQA, HotpotQA, and FEVER) demonstrate that GateRAG substantially improves the retrieval quality–efficiency tradeoff over prior private RAG systems. A single parameter $P$ controls this tradeoff, enabling different operating points along the quality–efficiency spectrum. At a quality-oriented operating point, GateRAG improves recall by 1.70–20.64 percentage points while reducing latency by 53.3–96.0% and communication by 89.5–98.6% compared with Pisces (Liang et al., 2026), the only prior system matching our privacy guarantees. At a more efficiency-oriented operating point, the latency and communication reductions reach 91.9–97.8% and 97.3–99.2%, respectively. GateRAG therefore provides a promising foundation for efficient private RAG.