- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes FedGRAG, a federated GraphRAG framework for retrieval and generation over knowledge graphs distributed across multiple clients. The method consists of two main components: (1) a federated knowledge routing mechanism that represents each client using multiple graph-aware prototypes and constructs a global prototype graph for query-dependent client selection, and (2) an evidence abstraction mechanism that retrieves local evidence subgraphs, encodes them into compact representations, and reconstructs them at the server for cross-client evidence integration and generation. Experiments on HotpotQA, MuSiQue, and 2Wiki show improvements over a range of centralized GraphRAG and federated RAG baselines.
Critical strengths
- Federated GraphRAG is an interesting and practically motivated extension of GraphRAG, particularly when graph knowledge is distributed across organizations and cannot simply be centralized.
- The combination of prototype-based client routing and local evidence abstraction provides a reasonably well-integrated pipeline from distributed graph indexing to cross-client evidence aggregation.
- Representing each client with multiple prototypes is a reasonable way to accommodate heterogeneous local knowledge rather than forcing each client into a single representation.
Critical weaknesses
- The paper repeatedly motivates the method using privacy and describes the evidence construction as “controlled disclosure.” However, the server receives query-dependent evidence information, including the vocabulary associated with retrieved entities/relations, and subsequently reconstructs evidence through a decoder. Not transmitting the entire local KG is not by itself sufficient to establish privacy preservation. The paper does not specify a formal threat model or provide leakage evaluations such as reconstruction, entity inference, membership inference, or privacy-utility analysis. I therefore view the current evidence as demonstrating limited/controlled disclosure rather than a demonstrated privacy guarantee.
- The experiments construct federated scenarios by distributing the knowledge from HotpotQA, MuSiQue, and 2Wiki across clients. This is a reasonable proof-of-concept, but the paper provides limited characterization of the resulting federation. In particular, it is unclear how client heterogeneity, client size imbalance, entity overlap, and cross-client evidence dependencies are controlled. These factors are central to the claimed federated setting and could substantially affect the routing results.
Questions for authors
Clarification
- What is the assumed adversary model for the server? Is the server trusted, honest-but-curious, or potentially malicious? What specific privacy notion does “controlled disclosure” provide?
- Could the authors explicitly enumerate all information transmitted from a client to the server during inference, including prototypes, client identifiers, and any provenance information?
- How accurately can the server-side decoder reconstruct the original evidence attributes or entities. If reconstruction is highly accurate, what prevents the transmitted representation from leaking the original local knowledge? Suggestions
- A formal privacy analysis, including reconstruction/entity-inference attacks and privacy-utility tradeoffs, would substantially strengthen the privacy-related claims.
- Larger federated experiments with substantially more clients would be useful for validating the scalability of the prototype graph and the claimed communication advantages.
Thank you for recognizing the practical motivation of FedGRAG, our integrated multi-prototype pipeline, and the clarity of our presentation. Below we address your questions and suggestions concisely.
1. Server Model and Controlled Disclosure Scope We agree with your assessment that our framework demonstrates architectural controlled disclosure (Abstract and Section 3.1) rather than a formal cryptographic privacy guarantee, and we have calibrated broad privacy phrasing in the manuscript accordingly. Regarding your first question, the server in our setting operates as an honest-but-curious RAG coordinator rather than a malicious adversary or a fully trusted central repository. Controlled disclosure enforces two concrete data-minimization boundaries for collaborative RAG: first, clients never pool full local KGs, exposing under 0.05% of local triples per query; second, raw source documents remain strictly on clients, while the decoder reconstructs only concise facts needed to ground the LLM. Because RAG inherently requires surfacing query-relevant facts to prevent hallucination, formal membership or inversion attack benchmarks fall outside our collaborative RAG scope, though orthogonal cryptographic layers (such as FRAG or C-FedRAG) can wrap our vector transmissions when stricter security is needed.
2. Complete Enumeration of Transmitted Information We have added an explicit transmission summary in Section 3.3 covering both stages:
3. Reconstruction Accuracy vs. Local Knowledge Protection On HotpotQA (10 clients), our server decoder achieves 91.8% entity F1 and 86.3% relation F1 at the word level on retrieved subgraphs (yielding 52.5 QA F1, close to ground-truth triples at 53.4). High fidelity on the active subgraph is intentional and essential for RAG to ground generation accurately. What prevents leaking a client's broader local knowledge is pre-encoding scope truncation and source-text stripping, rather than noisy decoding: per query, a client's retrieved subgraph averages only 0.04% of its local triples, so over 99.95% of local entities and relations never enter the uploaded embeddings or subgraph word list. Moreover, all raw documents stay local, and answer-aligned training (Equations 11 and 12) strips away verbose surrounding sentences from original passages, which also improves QA over exposing raw text (w/o ES in Figure 3).
4. Characterization of the Federated Benchmarks We partition knowledge across clients in a non-IID manner and have added a quantitative characterization of our 10-client federations in Appendix B.1, reporting max-to-min client triple ratio, Jensen-Shannon divergence (JSD) of relation distributions, cross-client entity overlap, and the share of test queries whose supporting facts span at least two clients:
Across all three benchmarks, most multi-hop queries require connecting facts across multiple clients despite low entity overlap and skewed local relation distributions, explaining why multi-prototype routing and our Evidence Interaction Graph outperform single-vector matching.
5. Scalability Across 10, 20, 30, and 50 Clients Because online routing only computes lightweight 64-dimensional inner products on the prototype graph and only the fixed top B=5 clients transmit evidence packages per query, online evidence communication does not grow with the total number of clients. Combining our 10- and 20-client HotpotQA benchmarks with extended 30- and 50-client evaluations (under fixed B=5) against the strongest federated baseline FedMosaic confirms this scalability:
From 10 to 30 clients, although fixed B=5 narrows client coverage from 50% to 16.7%, FedGRAG peaks at 53.9 F1 (20 clients), holds 51.4 F1 at 30 clients, and widens its F1 lead over FedMosaic from 4.9 to 6.7 and 6.0 points as finer partitions yield cleaner local prototypes. Even at 50 clients (selecting only 10% of clients across heavily fragmented graphs), FedGRAG still outperforms FedMosaic by 2.8 R@5 and 4.2 F1, nearly matching FedMosaic's 20-client F1 (46.8 vs. 47.2).