acceptodds
Under review as a conference paper at ICLR 2027

SynFedRAG: Federated GraphRAG with Persistent Synthetic Knowledge Graph

Abstract

Federated learning (FL) enables LLM adaptation over decentralized private data, primarily by transferring knowledge through model parameters or adapters. However, for knowledge-intensive question answering (QA), relying solely on local information can be insufficient when relevant evidence is distributed across multiple clients. Retrieval-augmented generation (RAG) provides a natural mechanism for accessing such non-parametric knowledge rather than relying solely on knowledge encoded in model parameters. Existing RAG and GraphRAG systems typically assume centralized access to the underlying corpus or retrieval knowledge, making them difficult to apply when evidence is distributed across private clients. Recent federated RAG methods relax this by constructing shared retrieval memories from client-derived information. However, cross-client evidence may still need to be accessed remotely at query time, rather than being available as a persistent retrieval resource at each client. To address this gap, we propose SYNFEDRAG, which learns a shared synthetic graph that can be queried locally by every client without contacting other clients at inference time. The server constructs a shared synthetic knowledge graph from condensed graph optimized to approximate the retrieval behavior of clients’ private graphs capturing distributed retrieval knowledge without centralizing raw documents or full local graphs. Clients then jointly read from their private and the shared synthetic knowledge graph, allowing them to access complementary retrieval knowledge beyond what is available in their local corpus. A lightweight graph-guided prompt module maps the retrieved subgraphs into soft prompts for a frozen LLM. During federated refinement, private QA supervision jointly adapts the graph-guided prompt module and continuously refines the shared synthetic knowledge graph over successive federated rounds. Experiments on non-IID decentralized multi-hop QA benchmarks show that SYNFEDRAG consistently improves answer quality and cross-client QA performance over local-only GraphRAG and federated LoRA baselines.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Federated Retrieval-Augmented Generation, GraphRAG, Synthetic Knowledge Graph, Graph Condensation.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dtrEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SynFedRAG, a federated GraphRAG framework for knowledge-intensive QA over decentralized private corpora. The main idea is to construct a persistent synthetic knowledge graph that captures retrieval knowledge distributed across clients and can be queried locally without contacting other clients at inference time. SynFedRAG first condenses each client’s private entity-sentence-passage graph into a compact summary, aggregates these summaries to initialize a shared synthetic graph, and then jointly refines the synthetic graph and a lightweight graph-guided prompt module through federated QA supervision while keeping the LLM frozen.

Critical strengths
  1. The paper identifies a clear and practically relevant gap in federated RAG. The distinction between transferring model parameters and transferring retrieval knowledge is well motivated. In particular, the goal of learning a persistent retrieval resource that remains locally queryable at inference time gives SynFedRAG a clear positioning relative to federated approaches that still require query-time access to remote repositories.
  2. The two-stage design is reasonably well aligned with the motivation. Phase I addresses communication and decentralization through client-side graph condensation and synthetic-graph initialization, while Phase II makes the resulting graph task-aware through federated QA supervision. This is a more coherent design than simply aggregating compressed client graphs once and treating the resulting representation as fixed.
Critical weaknesses

If the authors adequately address the above concerns, I would be willing to raise my score.

  1. The robustness and scalability evaluation is still limited for a federated system. The main experiments consider three clients, and the evaluated heterogeneity is primarily induced through random and topic-skewed corpus partitions. I would like to see stronger stress tests involving different numbers of clients, partial client participation, client dropout, highly imbalanced corpus sizes, noisy or corrupted local graphs, and potentially low-quality client summaries. These settings are particularly relevant here because a persistent synthetic graph aggregates information from multiple clients and may accumulate errors from unreliable participants.
  2. The efficiency analysis is useful but not yet sufficient to establish the practical advantage of the full pipeline. The paper reports compression-versus-QA trade-offs and shows that one-time graph communication can be reduced substantially. However, SynFedRAG contains several potentially expensive stages, including private graph construction, graph condensation, synthetic-graph optimization, repeated graph refinement, and prompt-module synchronization. A more convincing evaluation would compare end-to-end training time, inference latency, peak GPU memory, preprocessing/graph-construction cost, and total communication against DGRAG, FD-RAG, and local GraphRAG baselines.
  3. The paper insufficiently discusses recent federated graph condensation and compression methods. This is important because one of the central methodological components of SynFedRAG is precisely client-side graph condensation followed by server-side construction of a shared synthetic graph. Recent methods such as FedGVD, FedC4, FedGC, and FedGM address closely related problems of communicating, aggregating, or learning compact graph knowledge in federated settings. Although these methods do not necessarily target GraphRAG or QA directly, the conceptual overlap with Phase I is substantial enough that they should at least be carefully discussed. Ideally, some of them should be adapted as alternative shared-graph construction strategies while keeping the downstream SynFedRAG retrieval/LLM pipeline fixed.
Questions for authors

See Weakness.

8 comments
AuthorsOpenReview ↗

Response to W1 (1/2): Number of clients, partial participation, and dropout

W1(1/2): We thank the reviewer for this suggestion. We added stress tests covering each of the requested settings. Following the protocol of the main experiments, all results are reported as mean ±\pm standard deviation over three runs with seeds 42, 43, and 44, where each seed controls both the corpus partition and training.

Number of clients (Table R3.1). As the number of clients grows, each client holds a smaller share of the corpus and almost all questions require evidence from other clients, so all methods degrade. Local RAG and GraphRAG baselines degrade the most, since each client can only retrieve from its own shrinking corpus. SynFedRAG remains the best method at every scale and maintains the highest F1 at every evaluated client count, because the shared synthetic graph keeps cross-client knowledge available locally.

Partial participation and client dropout (Table R3.2). Our main experiments used full participation; here, only part of the clients contributes in each round. F1 decreases gradually as fewer client updates reach the server, following the expected fraction of participating clients. At a similar expected fraction, random dropout performs slightly worse than client sampling, since the number of received updates varies from round to round. Cross-client F1 decreases by a similar amount as overall F1, suggesting that partial participation does not disproportionately affect questions that rely on other clients' knowledge.

Table R3.1: Question-answering F1 scores with different numbers of clients on HotpotQA, MuSiQue, and 2WikiMultiHopQA under random partition.

GroupMethodHotpotQAMuSiQue2WikiMultiHopQA
5 Clients10 Clients20 Clients5 Clients10 Clients20 Clients5 Clients10 Clients20 Clients
Federated LLMRoLoRA24.27 ± 0.4423.50 ± 0.5222.12 ± 0.6111.37 ± 0.4310.82 ± 0.4710.29 ± 0.5822.99 ± 0.4622.12 ± 0.5521.53 ± 0.63
FL + Local GraphRAGHippoRAG34.22 ± 0.6831.50 ± 0.7928.39 ± 0.9315.28 ± 0.5213.24 ± 0.6412.31 ± 0.7130.28 ± 0.5926.86 ± 0.7222.93 ± 0.88
LinearRAG31.70 ± 0.6228.47 ± 0.7423.34 ± 0.8613.76 ± 0.4811.52 ± 0.5710.38 ± 0.6926.95 ± 0.5724.31 ± 0.6621.17 ± 0.83
G-retriever33.49 ± 0.6630.89 ± 0.8126.53 ± 0.9717.03 ± 0.5515.51 ± 0.6213.52 ± 0.7629.81 ± 0.6127.21 ± 0.7024.23 ± 0.92
GRAG34.59 ± 0.7131.39 ± 0.7728.31 ± 0.9416.72 ± 0.5315.10 ± 0.6813.56 ± 0.6427.83 ± 0.5824.72 ± 0.7522.70 ± 0.86
FL + Local RAGFLARE36.23 ± 0.6732.75 ± 0.8328.17 ± 1.0216.28 ± 0.5414.88 ± 0.6312.16 ± 0.7834.63 ± 0.6431.48 ± 0.7927.68 ± 0.91
Dense RAG37.18 ± 0.6433.39 ± 0.7631.25 ± 0.8915.09 ± 1.2113.22 ± 1.3311.56 ± 1.4729.74 ± 0.9327.33 ± 1.0623.52 ± 1.18
Federated RAGDGRAG40.94 ± 0.3139.30 ± 0.3837.24 ± 0.5230.28 ± 0.2428.66 ± 0.3327.11 ± 0.4136.53 ± 0.2735.42 ± 0.3632.80 ± 0.49
FD-RAG40.73 ± 0.7138.40 ± 0.6935.94 ± 0.8428.82 ± 0.3827.14 ± 0.4624.58 ± 0.5735.84 ± 0.2233.98 ± 0.3432.09 ± 0.45
SynFedRAG (Ours)46.76 ± 0.1945.90 ± 0.2844.25 ± 0.4135.03 ± 0.2734.25 ± 0.3532.94 ± 0.5240.23 ± 0.5139.41 ± 0.5837.96 ± 0.69

Table R3.2: Robustness of SynFedRAG to partial participation and client dropout (M=10M=10 clients, T=10T=10 rounds, random partition). Reference: all clients participate in every round (q=1.0q=1.0); it corresponds to the 10-client setting in Table R3.1. Partial participation: in each round, only a fraction qq of the clients is sampled for training. Random dropout: each sampled client fails to send its update with probability pp, and the server aggregates the remaining updates. We report F1 on all test questions (Overall) and on questions whose supporting evidence is held by other clients (Cross), as mean ±\pm standard deviation over three runs.

HotpotQAMuSiQue2WikiMultiHopQA
ScenarioSettingOverallCrossOverallCrossOverallCross
Referenceq=1.0q=1.045.90 ± 0.2845.21 ± 0.3134.25 ± 0.3533.86 ± 0.3839.41 ± 0.5838.92 ± 0.63
Partial participationq=0.5q=0.545.08 ± 0.3744.27 ± 0.4133.61 ± 0.4433.12 ± 0.4938.70 ± 0.6638.06 ± 0.72
q=0.3q=0.343.96 ± 0.5242.95 ± 0.5832.58 ± 0.5731.97 ± 0.6337.62 ± 0.7736.81 ± 0.85
Random dropoutp=0.2p=0.245.31 ± 0.3444.52 ± 0.3933.79 ± 0.4133.30 ± 0.4538.95 ± 0.6338.34 ± 0.69
p=0.5p=0.544.22 ± 0.4943.30 ± 0.5532.91 ± 0.5932.28 ± 0.6637.98 ± 0.8137.19 ± 0.88
AuthorsOpenReview ↗

Response to W1 (2/2): Imbalanced corpora and corrupted clients

W1(2/2): Imbalanced corpus sizes (Table R3.3). As one client's share of the corpus grows, more of its questions become answerable locally, consistent with Section 4.1. The smaller clients' F1 under SynFedRAG decreases only moderately and remains clearly above the federated RAG baselines at every ratio, showing that the shared graph compensates for a limited local corpus.

Table R3.3: Imbalanced client corpus sizes (M=3M=3, random partition). Articles are allocated with size ratio r:1:1r{:}1{:}1, so the largest client holds r/(r+2)r/(r{+}2) of the corpus. Questions keep round-robin ownership, so every client owns the same number of questions. Large reports F1 on the questions of the largest client, and Small the average F1 on the questions of the two smaller clients.

HotpotQAMuSiQue2WikiMultiHopQA
Size ratioMethodOverallSmallLargeOverallSmallLargeOverallSmallLarge
1:1:1 (33%)DGRAG42.17 ± 0.2642.11 ± 0.3142.29 ± 0.4231.38 ± 0.1731.32 ± 0.2131.50 ± 0.2937.68 ± 0.1937.62 ± 0.2337.80 ± 0.32
FD-RAG42.30 ± 0.6742.24 ± 0.7442.42 ± 0.9630.04 ± 0.3229.98 ± 0.3730.16 ± 0.5137.04 ± 0.1336.98 ± 0.1737.16 ± 0.24
SynFedRAG47.43 ± 0.1347.37 ± 0.1647.55 ± 0.2335.61 ± 0.2135.55 ± 0.2535.73 ± 0.3440.85 ± 0.4740.79 ± 0.5340.97 ± 0.71
2:1:1 (50%)DGRAG42.41 ± 0.2941.91 ± 0.3643.43 ± 0.4431.02 ± 0.2231.80 ± 0.2729.47 ± 0.3337.83 ± 0.2437.53 ± 0.2938.45 ± 0.35
FD-RAG42.70 ± 0.7141.86 ± 0.8244.38 ± 0.9330.24 ± 0.3529.81 ± 0.4331.09 ± 0.4937.21 ± 0.1836.87 ± 0.2337.88 ± 0.27
SynFedRAG48.00 ± 0.1846.81 ± 0.2250.40 ± 0.2735.66 ± 0.2435.55 ± 0.3135.89 ± 0.3640.95 ± 0.5140.75 ± 0.5841.36 ± 0.69
4:1:1 (67%)DGRAG42.75 ± 0.3441.62 ± 0.4545.02 ± 0.4130.50 ± 0.2632.23 ± 0.3427.06 ± 0.3138.04 ± 0.2737.35 ± 0.3739.40 ± 0.34
FD-RAG43.26 ± 0.7641.40 ± 0.9347.00 ± 0.8830.52 ± 0.4129.57 ± 0.5232.42 ± 0.4737.43 ± 0.2236.68 ± 0.3138.94 ± 0.26
SynFedRAG48.80 ± 0.2146.14 ± 0.2954.13 ± 0.2535.74 ± 0.2935.49 ± 0.3736.24 ± 0.3341.09 ± 0.5540.63 ± 0.6641.99 ± 0.72
8:1:1 (80%)DGRAG43.19 ± 0.3841.48 ± 0.5446.61 ± 0.4729.84 ± 0.3132.45 ± 0.4224.64 ± 0.3738.30 ± 0.3337.27 ± 0.4640.36 ± 0.39
FD-RAG43.98 ± 0.8441.16 ± 1.0749.63 ± 0.9430.89 ± 0.4629.45 ± 0.6333.76 ± 0.5537.72 ± 0.2736.59 ± 0.3839.99 ± 0.33
SynFedRAG49.83 ± 0.2745.80 ± 0.3657.87 ± 0.3135.83 ± 0.3335.45 ± 0.4536.59 ± 0.3941.26 ± 0.6140.58 ± 0.7742.63 ± 0.74

Noisy or corrupted clients and low-quality summaries (Table R3.4). We corrupt one client (C2). The clean clients' F1 decreases with corruption severity, confirming that the shared graph carries C2's knowledge, but it remains above the setting where C2 is removed, so an unreliable client does not harm others more than losing its contribution. Most of the damage stays with the corrupted client itself. We will discuss robust aggregation in the revision.

Table R3.4: Robustness to an unreliable client (M=3M=3, random partition). C2 is corrupted; C1 and C3 are clean. Clean / C2: F1 on the clean clients' / C2's own questions. C2 removed: C2 does not participate. Noisy graph: a fraction of C2's graph edges is randomly rewired. Wrong facts: in a fraction of C2's passages, the answer entity is replaced with a wrong one of the same type. Low-quality summary: C2 uploads a condensed graph built from random anchors.

HotpotQAMuSiQue2WikiMultiHopQA
SettingLevelCleanC2CleanC2CleanC2
No corruption–47.52 ± 0.1747.25 ± 0.2935.70 ± 0.2635.43 ± 0.3840.94 ± 0.5340.67 ± 0.71
C2 removed–39.52 ± 0.62n/a22.60 ± 0.84n/a34.24 ± 0.77n/a
Noisy graph30% edges46.62 ± 0.2444.15 ± 0.4634.48 ± 0.3332.95 ± 0.5740.22 ± 0.5838.00 ± 0.83
60% edges45.12 ± 0.3640.25 ± 0.7132.58 ± 0.4729.83 ± 0.8839.02 ± 0.6434.65 ± 1.02
Wrong facts20% passages45.82 ± 0.3143.55 ± 0.5833.49 ± 0.4232.47 ± 0.6639.58 ± 0.6137.49 ± 0.91
50% passages42.72 ± 0.4937.65 ± 0.9329.46 ± 0.6827.75 ± 1.0737.10 ± 0.7932.41 ± 1.18
Low-quality summaryrandom anchors45.72 ± 0.4346.35 ± 0.3733.70 ± 0.5934.61 ± 0.4939.54 ± 0.7239.87 ± 0.76
AuthorsOpenReview ↗

Response to W2: End-to-end cost and inference latency

W2. We thank the reviewer for this suggestion. Table R3.5 reports the end-to-end offline cost of SynFedRAG, DGRAG, FD-RAG, and G-Retriever under the same hardware and backbone.

Offline computation. The stages listed by the reviewer that precede training, namely private graph construction, graph condensation, and synthetic-graph initialization, together cost less than the preprocessing of DGRAG and FD-RAG, since SynFedRAG builds its graphs without LLM generation, whereas both baselines use the LLM to produce summaries or QA memories. Most of SynFedRAG's cost lies in training, which consists of two parts. Training the graph-guided prompt module costs about the same as G-Retriever's training, since both train a lightweight graph encoder and projector through the frozen LLM. The additional cost comes from refining the shared synthetic graph, where gradient matching requires extra passes through the frozen LLM. As a result, its total offline cost is close to that of DGRAG and higher than that of FD-RAG and G-Retriever, and its peak memory remains within a single 24 GB GPU. This cost is paid once before deployment, in return for the accuracy gains reported in the main results.

Communication. Total communication is reported in Appendix D.1 (Tables 11 and 12). For retrieval knowledge, SynFedRAG transmits 40.22 MiB in total, compared with 62.37 MiB for DGRAG and 75.13 MiB for FD-RAG. Including prompt-module synchronization, its total offline communication is 701.0 MiB, which remains below the 770.0 MiB required by the RoLoRA reader adaptation used by DGRAG and FD-RAG alone.

Table R3.5: End-to-end offline cost on HotpotQA (M=3M=3, random partition, single RTX 4090, frozen Qwen2.5-7B). Preproc.: building the retrieval structure: local graph construction and text encoding for G-Retriever, knowledge-graph extraction and subgraph summarization for DGRAG, hypergraph learning and QA-memory generation for FD-RAG, and tri-graph construction, graph condensation, and synthetic-graph initialization for SynFedRAG. Training: all remaining offline training. G-Retriever keeps the LLM frozen and trains its graph encoder and projector with federated averaging. Time is summed over clients and the server; peak GPU memory is the maximum over all stages.

Time (min)
MethodPreproc.TrainingTotalPeak GPU mem. (GB)
G-Retriever27.8352.4380.221.7
DGRAG182.4301.5483.919.8
FD-RAG41.3301.5342.819.8
SynFedRAG (Ours)34.6458.0492.622.6

Table R3.6: Inference latency on HotpotQA (M=3M=3, random partition, single RTX 4090, frozen Qwen2.5-7B, batch size 1). Latency is the mean time per test question, including retrieval and generation and excluding network delay. Query-time comm.: whether answering a question may require contacting the server or other clients.

MethodLatency (s/query)Query-time comm.
LinearRAG0.92No
G-Retriever4.87No
DGRAG2.63Yes
FD-RAG1.17No
SynFedRAG (Ours)0.94No

Inference latency. Table R3.6 reports inference latency. After deployment, SynFedRAG answers each query locally, without communicating with the server or other clients. Its latency is close to that of LinearRAG, which uses the same local retrieval: the additional components operate only on a small evidence subgraph and the compact synthetic graph, and contribute a soft token rather than additional retrieved text, so access to cross-client knowledge adds little inference cost. FD-RAG is slightly slower, since it adds retrieved QA memories to the prompt. DGRAG is slower still, since it generates several local answers to decide whether to escalate a query and retrieves from remote clients for escalated queries. G-Retriever has the highest latency, since it solves a subgraph optimization over the local graph for each query.

Reviewer_UES7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses federated GraphRAG, where QA evidence is distributed across private client corpora that cannot be pooled. SynFedRAG learns a persistent shared synthetic knowledge graph that every client can query locally at inference time. In Phase I, each client condenses its private entity-sentence-passage graph into a compact summary (Eq. 1), and the server initializes a synthetic graph with a parameterized graph estimator by aligning pooled graph-level representations plus a node-diversity regularizer (Eq. 2 to 5). In Phase II, clients jointly refine the synthetic graph and a graph-guided prompt module (GNN encoder plus projector producing soft graph tokens for a frozen LLM) using private QA supervision, with weighted server aggregation (Eq. 9 to 13). Experiments on HotpotQA, MuSiQue, and 2WikiMultiHopQA under random and topic-skewed partitions show the best F1 among federated LLM, local RAG, and federated RAG baselines (Tables 1 and 2), with ablations on components and objectives (Table 4), a cross-client question split (Table 5), and a membership leakage study (Table 6).

Critical strengths
  1. The problem formulation is well chosen and clearly articulated. Decoupling knowledge sharing from raw corpus sharing, and requiring the shared retrieval structure to be persistent and locally queryable rather than remotely accessed at query time, is a clean conceptual contribution that distinguishes the work from prior federated RAG systems.
  2. The evaluation is broader and more honest than typical for this niche. It covers random and topic-skewed partitions, separates local-answerable from cross-client questions (Table 5), measures condensation fidelity against final QA sensitivity (Table 3), and reports a membership leakage analysis that openly shows what the protocol still exposes (Table 6).
Critical weaknesses
  1. The privacy premise is undercut by the paper's own measurements. The uploaded condensed graphs expose passage membership at 67.1 to 72.9% AUC to the server, and accumulated synthetic-graph snapshots raise other-client membership AUC to 59.3% on 2WikiMultiHopQA (Table 6). Since avoiding data exposure is the stated motivation, the absence of any formal guarantee or of mechanisms such as secure aggregation or differential privacy makes the practical privacy gain over simply sharing dense retrieval indices unclear.
  2. The experimental setting is far from deployment scale. Only three clients are used, and each corpus is built from the candidate passages of the very question sets under evaluation (for example 19,583 passages for HotpotQA), so retrieval operates over small, self-contained collections rather than realistic client-held corpora. The two federated RAG baselines are recent preprints adapted to this setting, so the strength of the empirical comparison is hard to judge.
Questions for authors
  1. The privacy premise is undercut by the paper's own measurements. The uploaded condensed graphs expose passage membership at 67.1 to 72.9% AUC to the server, and accumulated synthetic-graph snapshots raise other-client membership AUC to 59.3% on 2WikiMultiHopQA (Table 6). Since avoiding data exposure is the stated motivation, the absence of any formal guarantee or of mechanisms such as secure aggregation or differential privacy makes the practical privacy gain over simply sharing dense retrieval indices unclear.
  2. The experimental setting is far from deployment scale. Only three clients are used, and each corpus is built from the candidate passages of the very question sets under evaluation (for example 19,583 passages for HotpotQA), so retrieval operates over small, self-contained collections rather than realistic client-held corpora. The two federated RAG baselines are recent preprints adapted to this setting, so the strength of the empirical comparison is hard to judge.
6 comments
AuthorsOpenReview ↗

Response to Q1

Q1 We thank the reviewer for the careful reading of our leakage analysis and for raising the comparison with sharing dense retrieval indices, which lets us quantify the practical privacy gain directly.

SynFedRAG follows the data-locality premise of federated learning: raw documents, complete private graphs, QA pairs, and query-specific retrieval traces never leave the client. We measure what the shared representations reveal (Sec. 4.5) so that this premise can be compared quantitatively with alternatives.

Comparison with sharing dense retrieval indices.

A dense retrieval index contains one encoder embedding per passage. These are exactly the passage-node features of the full private graph, which is the reference row of Table 7 of the revised paper. Under the same attack, this exposure yields 100.0%, 99.9%, and 100.0% AUC and 100% TPR at 1% FPR on HotpotQA, MuSiQue, and 2WikiMultiHopQA, and it applies to every party that receives the index. SynFedRAG reduces this exposure for both parties:

  • Other clients receive only the synthetic graph, from which target-client membership is at chance level (49.5–51.1% AUC). A stronger readout-aware attack does not increase it (Table 16). Accumulating all snapshots changes other-client membership AUC on 2WikiMultiHopQA from 55.8±4.855.8\pm4.8 to 59.3±3.659.3\pm3.6, and leaves the other two datasets unchanged (53.3 to 53.1 and 51.7 to 52.6).
  • The server receives each condensed graph once, at 67.1–72.9% AUC and 20.6–25.9% TPR, compared with 100% for a dense index. The condensed graph retains at most 20% of the nodes, and its features fuse neighborhood information rather than store one embedding per passage (Appendix B.2.2).

Compatibility with secure aggregation.

The server uses client information only through sums over clients, in both phases. In Phase II, it requires only the weighted sums of prompt-module parameters and graph updates (Eq. 12). In Phase I, the condensed graphs enter only through their graph-level embeddings zmz_m (Eq. 2), and the alignment objective (Eq. 3) depends on them only through their normalized sum:

Laligninit=1−⟨1M∑m=1Mzm∥zm∥,zsyn∥zsyn∥⟩.\mathcal{L}^{\mathrm{init}}_{\mathrm{align}} = 1 - \left\langle \frac{1}{M}\sum_{m=1}^{M}\frac{z_m}{\lVert z_m\rVert}, \frac{z_{\mathrm{syn}}}{\lVert z_{\mathrm{syn}}\rVert} \right\rangle. Each client can therefore compute zmz_m locally with the shared frozen module W(0)W^{(0)} and submit it through secure aggregation. This yields the same synthetic graph, while the server observes neither the condensed graphs nor any individual client update. Differential privacy can be applied to the same aggregated quantities.

Revision.

We will state the data-locality premise precisely, add the comparison with dense retrieval indices to the leakage analysis, and describe the secure-aggregation protocol above in the revised paper.

AuthorsOpenReview ↗

Response to Q2 (1/2): Evaluation with more clients

Q2(1/2) We thank the reviewer for highlighting these limitations. We agree that the original three-client evaluation on benchmark-derived corpora provides limited evidence of scalability. We have therefore added experiments with more clients and substantially larger retrieval corpora, and clarify the breadth of our baseline comparison below.

Larger client populations.

Table 19 extends the evaluation to 5, 10, and 20 clients under random partition. With the total corpus fixed, increasing the number of clients distributes evidence across more private collections, making cross-client knowledge access more challenging. Although all decentralized methods lose accuracy as the number of clients increases, SynFedRAG achieves the highest F1 among them across all client counts and datasets. At 20 clients, it achieves 44.25, 32.94, and 37.96 F1 on HotpotQA, MuSiQue, and 2WikiMultiHopQA, respectively, exceeding the strongest baseline by 7.01, 5.83, and 5.16 points.

Table 19. Question-answering F1 scores with different numbers of clients on HotpotQA, MuSiQue, and 2WikiMultiHopQA under random partition.

GroupMethodHotpotQA: 5 clientsHotpotQA: 10 clientsHotpotQA: 20 clientsMuSiQue: 5 clientsMuSiQue: 10 clientsMuSiQue: 20 clients2WikiMultiHopQA: 5 clients2WikiMultiHopQA: 10 clients2WikiMultiHopQA: 20 clients
Federated LLMRoLoRA24.27 ± 0.4423.50 ± 0.5222.12 ± 0.6111.37 ± 0.4310.82 ± 0.4710.29 ± 0.5822.99 ± 0.4622.12 ± 0.5521.53 ± 0.63
FL + Local GraphRAGHippoRAG34.22 ± 0.6831.50 ± 0.7928.39 ± 0.9315.28 ± 0.5213.24 ± 0.6412.31 ± 0.7130.28 ± 0.5926.86 ± 0.7222.93 ± 0.88
FL + Local GraphRAGLinearRAG31.70 ± 0.6228.47 ± 0.7423.34 ± 0.8613.76 ± 0.4811.52 ± 0.5710.38 ± 0.6926.95 ± 0.5724.31 ± 0.6621.17 ± 0.83
FL + Local GraphRAGG-retriever33.49 ± 0.6630.89 ± 0.8126.53 ± 0.9717.03 ± 0.5515.51 ± 0.6213.52 ± 0.7629.81 ± 0.6127.21 ± 0.7024.23 ± 0.92
FL + Local GraphRAGGRAG34.59 ± 0.7131.39 ± 0.7728.31 ± 0.9416.72 ± 0.5315.10 ± 0.6813.56 ± 0.6427.83 ± 0.5824.72 ± 0.7522.70 ± 0.86
FL + Local RAGFLARE36.23 ± 0.6732.75 ± 0.8328.17 ± 1.0216.28 ± 0.5414.88 ± 0.6312.16 ± 0.7834.63 ± 0.6431.48 ± 0.7927.68 ± 0.91
FL + Local RAGDense RAG37.18 ± 0.6433.39 ± 0.7631.25 ± 0.8915.09 ± 1.2113.22 ± 1.3311.56 ± 1.4729.74 ± 0.9327.33 ± 1.0623.52 ± 1.18
Federated RAGDGRAG40.94 ± 0.3139.30 ± 0.3837.24 ± 0.5230.28 ± 0.2428.66 ± 0.3327.11 ± 0.4136.53 ± 0.2735.42 ± 0.3632.80 ± 0.49
Federated RAGFD-RAG40.73 ± 0.7138.40 ± 0.6935.94 ± 0.8428.82 ± 0.3827.14 ± 0.4624.58 ± 0.5735.84 ± 0.2233.98 ± 0.3432.09 ± 0.45
Federated RAGSynFedRAG (Ours)46.76 ± 0.1945.90 ± 0.2844.25 ± 0.4135.03 ± 0.2734.25 ± 0.3532.94 ± 0.5240.23 ± 0.5139.41 ± 0.5837.96 ± 0.69
CentralizedDense RAG55.60 ± 0.2455.60 ± 0.2455.60 ± 0.2443.18 ± 0.3143.18 ± 0.3143.18 ± 0.3152.54 ± 0.3752.54 ± 0.3752.54 ± 0.37

The centralized Dense RAG reference uses the complete corpus and is independent of the number of clients; its values are repeated across columns for readability.

AuthorsOpenReview ↗

Response to Q2 (2/2): Full open-domain corpora and baseline comparison

Q2(2/2)

Larger retrieval corpora (Tables 29 and 30). We enlarge the retrieval corpora while keeping the same 1,000 test questions per dataset and the three-client random partition. HotpotQA grows from 19.6K passages to the official Wikipedia corpus of 5,233,329 introductory paragraphs. For MuSiQue and 2WikiMultiHopQA, we pool supporting and distractor paragraphs over the train, development, and test sets (139,416 and 430,225 paragraphs). F1 decreases for every method. SynFedRAG reaches 39.08, 32.97, and 36.40 F1, exceeding the strongest decentralized baseline by 4.22, 4.23, and 3.09 points. The shared graph does not need to grow with the corpus: the default budget of 128 synthetic nodes (0.25 MiB) is the best or within 0.23 F1 points of the best on all three full corpora (Table 30).

Table 29. F1 on the benchmark corpora used in the paper and on the full open-domain corpora (M=3M=3, random partition, same 1,000 test questions per dataset). Full corpora follow the standard open-domain setting of IRCoT: the official HotpotQA Wikipedia corpus (5,233,329 introductory paragraphs), and, for MuSiQue and 2WikiMultiHopQA, which ship no open-domain corpus, all supporting and distractor paragraphs pooled over their train, dev, and test sets (139,416 and 430,225 paragraphs). The full corpus is partitioned across clients in the same way as the benchmark corpus.

MethodHotpotQA: 19.6K (bench.)HotpotQA: 5.23M (full)MuSiQue: 25.9K (bench.)MuSiQue: 139K (full)2WikiMultiHopQA: 11.3K (bench.)2WikiMultiHopQA: 430K (full)
LinearRAG34.77 ± 0.5527.04 ± 0.6815.06 ± 0.4313.48 ± 0.5129.18 ± 0.4925.04 ± 0.62
Dense RAG39.87 ± 0.5731.18 ± 0.7216.67 ± 1.1514.84 ± 1.2432.52 ± 0.8227.71 ± 0.93
DGRAG42.17 ± 0.2634.31 ± 0.3831.38 ± 0.1728.74 ± 0.2937.68 ± 0.1933.31 ± 0.32
FD-RAG42.30 ± 0.6734.86 ± 0.7930.04 ± 0.3227.37 ± 0.4137.04 ± 0.1332.45 ± 0.27
SynFedRAG (Ours)47.43 ± 0.1339.08 ± 0.2735.61 ± 0.2132.97 ± 0.3040.85 ± 0.4736.40 ± 0.58
Centralized Dense RAG55.60 ± 0.2445.73 ± 0.3343.18 ± 0.3139.77 ± 0.3852.54 ± 0.3746.60 ± 0.46

Table 30. Contribution and size of the shared synthetic graph on the full open-domain corpora (M=3M=3, random partition, node ratio 0.2). Mean ± std over three seeds.

SettingHotpotQA (5.23M)MuSiQue (139K)2Wiki (430K)Graph state (MiB)
w/o synthetic graph28.62 ± 0.3614.57 ± 0.4826.93 ± 0.55–
Kg=32K_g=3237.92 ± 0.3430.86 ± 0.4132.11 ± 0.660.05
Kg=128K_g=128 (default)39.08 ± 0.2732.97 ± 0.3036.40 ± 0.580.25
Kg=256K_g=25639.31 ± 0.2932.74 ± 0.3335.93 ± 0.610.63
Kg=512K_g=51238.86 ± 0.3132.21 ± 0.3735.37 ± 0.641.71

Baseline comparison.

We appreciate the concern about relying on recently proposed federated RAG methods. We include DGRAG and FD-RAG because they explicitly address access to knowledge distributed across clients, complementing our comparisons with federated LLM adaptation and local RAG/GraphRAG methods. To provide an additional reference beyond these federated approaches, we have added centralized Dense RAG to assess how closely SynFedRAG approaches retrieval with unrestricted access to all clients' passages. Together with local Dense RAG, this comparison shows that SynFedRAG narrows the performance gap between local and full-corpus retrieval while keeping private corpora local. Its benefit is therefore supported by comparison with standard retrieval settings, in addition to the two federated RAG baselines.

Revision and scope. We have added these results and the corpus-construction details to the revised paper (Appendix D.7 and D.12). They provide evidence for larger client populations and larger retrieval collections, but they do not establish deployment-scale performance: the two dimensions are tested separately, and the corpora remain public benchmarks rather than naturally occurring client-held collections. Appendix D.12 states this limitation.

Reviewer_xRKQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper addresses the problem that current federated RAG methods still require remote access to cross-client evidence. It proposes SynFedRAG, a federated RAG framework based on a persistent synthetic knowledge graph. The method mainly constructs a shared synthetic graph, allowing clients to obtain cross-client knowledge from this graph during inference without remotely accessing other clients, and then use this knowledge to support their current reasoning. The overall method consists of two stages.

  • In the first stage, each client compresses its local graph and uploads the compressed graph to the server. The server aggregates these compressed graphs to construct an initial synthetic knowledge graph.
  • In the second stage, the synthetic knowledge graph generated by the server is optimized through federated learning, jointly updating the synthetic graph and the graph-guided prompt module.

During inference, each client uses both local private graph and the shared synthetic graph. The retrieved shared graph representation is converted into the soft prompt and combined with locally retrieved data to enhance the QA ability and enable cross-client knowledge sharing.

Critical strengths
  • The paper explores a federated RAG framework that uses a shared synthetic graph as a medium for cross-client knowledge transfer. This allows clients to utilize knowledge contributed by other clients during inference without relying on remote retrieval, which has certain research significance.
  • The method does not require directly sharing raw documents or complete private graphs. Instead, it mainly achieves knowledge sharing through a shared knowledge graph, reducing direct exposure of raw data to some extent. Moreover, no additional cross-client communication is required during inference.
Critical weaknesses
  • The authors identify the problem that existing methods rely on remote access to other clients during inference to achieve cross-client knowledge transfer. However, the paper does not clearly explain why it is necessary to construct a persistent shared graph retrieval resource. It does not explain why this approach should be adopted instead of other knowledge-sharing methods, or what advantages it offers over these alternatives. Furthermore, FD-RAG also employs persistent shared retrieval memory and does not require remote access during inference under its evaluated configuration. Therefore, what advantages does SynFedRAG offer compared with FD-RAG?
  • The shared synthetic graph mainly integrates client knowledge through graph-level representation alignment and federated parameter aggregation. However, these mechanisms do not explicitly constrain the preservation of client-specific knowledge. Could such knowledge gradually be weakened during repeated aggregation? When other clients need to retrieve this type of knowledge, can the shared synthetic graph still provide effective information support?
Questions for authors
  • The paper does not clearly highlight its main innovations. Using graphs as a medium for cross-client knowledge sharing is a valuable idea, but the paper does not sufficiently emphasize or explain this contribution. Furthermore, the organization of Figure 1 is rather confusing. On the one hand, the figure contains numerous fine-grained operations without highlighting the key components of the framework. On the other hand, the internal logic of the two training stages is not clearly presented.
  • The experiments are conducted with only three clients, and the paper lacks generalization experiments with varying numbers of clients. Moreover, the method is evaluated only on Qwen2.5-7B. How well does it adapt to other LLMs?
  • When constructing the compact graph, the clients first inherit connections from the original graph and then add edges based on semantic similarity. Are these additional edges necessary? What is the motivation for introducing them?
  • The writing and organization of the paper are relatively poor, making it difficult to understand. The contribution section summarizes five contributions but does not clearly highlight the core innovation and its role. The writing is verbose, and the main text provides insufficient discussion of related work, failing to clearly explain the differences between the proposed method and recent approaches.
  • The paper has certain limitations but does not provide corresponding discussions. For example, if a client contains incorrect facts or noisy local evidence, could this information affect the quality of the global shared graph?
11 comments
AuthorsOpenReview ↗

Response to W1 – Why a persistent shared graph, and comparison with FD-RAG

W1 We thank the reviewer. FD-RAG does support persistent shared memory and local inference. The difference lies in the shared memory itself: SynFedRAG learns a graph whose node representations and connectivity are optimized for downstream QA, whereas FD-RAG aggregates memories constructed separately at each client.

Why a persistent shared graph. When a question needs knowledge held by other clients, local retrieval cannot supply it. Aggregating parameters or adapters moves knowledge into the model but provides no memory to query for evidence. Distributed search retrieves the evidence, but requires remote clients to stay available and adds communication to every query. A persistent shared memory avoids both problems: knowledge is transferred before deployment, and each client queries the memory locally alongside its private corpus. We make this memory a graph so that related knowledge can be combined before it reaches the LLM. In a flat collection of memory items, each item is scored against the query on its own. In a graph, connected nodes first exchange information. We learn this connectivity through QA supervision rather than fixing it by similarity, so the way knowledge is combined is optimized for answering questions.

Evidence.

  1. Need for shared knowledge: 82–91% of test questions require evidence from another client under round-robin ownership, and 53–84% under topic-matched ownership (Table 9, Appendix D.3).

  2. Role of the graph: Removing the shared graph lowers F1 by 11.18–20.15 points, while a shared memory without connectivity recovers at most 3.99 of them (Table 5, α = 1.0).

  3. Alternatives: On the random partition, SynFedRAG exceeds federated reader adaptation with local retrieval by 4.01–17.02 F1 points and DGRAG by 3.17–5.26, without DGRAG’s per-query communication (Tables 2 and 12).

  4. FD-RAG: With fixed kNN connectivity, SynFedRAG falls below FD-RAG on all three datasets, and learned topology puts it ahead (Tables 3 and 5). Overall, SynFedRAG exceeds FD-RAG by 0.30–6.93 F1 across partitions and by 1.71–4.44 on cross-client questions (Tables 2, 3 and 6). Its retrieval-knowledge communication is also lower: 40.22 vs. 75.13 MiB on HotpotQA, excluding prompt-module exchanges (Table 12).

Revision. We have stated this motivation and distinction in the introduction, and added a table comparing SynFedRAG with existing approaches by shared medium, access to cross-client evidence, inference-time communication, and whether the shared structure is learned.

AuthorsOpenReview ↗

Response to W2 – Preservation of client-specific knowledge

W2

We thank the reviewer for this question and agree with its premise: neither Laligninit\mathcal{L}^{\text{init}}_{\text{align}} nor local Lalign\mathcal{L}_{\text{align}} explicitly preserves client-specific knowledge. Both constrain pooled graph-level representations, and Phase I aligns the synthetic graph to an equally weighted combination of client summaries.

Where client-specific signal enters. In Phase II, each client computes LQA\mathcal{L}_{\text{QA}} and LGM\mathcal{L}_{\text{GM}} on its own questions and local evidence, so Δm\Delta_m reflects that client's data rather than a pooled target. Because the synthetic readout is query-conditioned (Eqs. 36–38), updates from different clients can concentrate on different nodes. Lreg\mathcal{L}_{\text{reg}} keeps nodes from collapsing together, which would otherwise merge client-specific content (App. D.5). This argument covers XsynX_{\text{syn}}; θPGE\theta_{\text{PGE}} is shared by all nodes, so its updates are fully pooled. Because this is not a formal guarantee, we test preservation directly.

Experiment. Under topic-skewed partitions (α=1.0\alpha=1.0), we take cross-client test questions whose missing evidence is held by exactly one other client, and group them by that client. If aggregation erased a client's knowledge, F1 in its column would stall or fall over rounds.

Table 24. Cross-client F1 by the client holding the missing evidence (α=1.0\alpha=1.0, mean ± std over 3 seeds). w/o synthetic graph and Graph frozen (graph fixed after Phase I, prompt module trained for 10 rounds) match Table 5; compare Graph frozen with Round 10. Ablations are after round 10.

SettingHotpotQA C1C2C3MuSiQue C1C2C32Wiki C1C2C3
# questions191239251226191137203211288
w/o synthetic graph31.24 ± 1.4630.17 ± 1.3430.48 ± 1.4213.62 ± 0.9214.35 ± 1.0114.11 ± 1.2229.37 ± 0.9630.29 ± 1.0328.41 ± 0.90
Graph frozen37.12 ± 2.1736.84 ± 1.6435.27 ± 1.6919.74 ± 0.8620.31 ± 1.1220.88 ± 1.2931.08 ± 1.0232.17 ± 1.2629.62 ± 1.07
Round 133.86 ± 2.0934.21 ± 1.8631.94 ± 1.6224.92 ± 1.4924.47 ± 1.8426.13 ± 1.9329.93 ± 1.9829.41 ± 1.9727.85 ± 1.60
Round 338.47 ± 1.8838.92 ± 1.6736.15 ± 1.7928.15 ± 1.5929.73 ± 1.7830.04 ± 2.0334.76 ± 2.0934.38 ± 1.9533.27 ± 1.60
Round 543.08 ± 1.9742.36 ± 1.7641.73 ± 1.6632.06 ± 1.5432.41 ± 1.5233.58 ± 1.7638.42 ± 1.7739.36 ± 1.6436.81 ± 1.66
Round 10 (full)46.12 ± 1.6345.57 ± 1.6243.98 ± 1.4233.87 ± 1.5435.26 ± 1.6435.94 ± 1.5742.31 ± 1.6542.05 ± 1.9140.47 ± 1.47
w/o local Lalign\mathcal{L}_{\text{align}}40.27 ± 2.0639.05 ± 1.6135.62 ± 1.4421.38 ± 1.5824.16 ± 1.7825.72 ± 1.8637.84 ± 1.7637.12 ± 1.8434.06 ± 1.82
w/o LGM\mathcal{L}_{\text{GM}}41.36 ± 1.9639.48 ± 1.5038.03 ± 1.5122.67 ± 1.3923.18 ± 1.4925.31 ± 2.1131.47 ± 1.8032.26 ± 1.9528.93 ± 1.62

Findings. The shared graph supports retrieval of every client’s knowledge. At Round 10, it raises cross-client F1 over w/o synthetic graph for every source client, by 13.50–15.40 points on HotpotQA, 20.25–21.83 on MuSiQue, and 11.76–12.94 on 2WikiMultiHopQA. Refinement does not erode any client’s contribution. Every column improves at each checkpoint, and Round 10 exceeds the frozen graph for every source client by similar margins within each dataset (8.71–9.00, 14.13–15.06, and 9.88–11.23 points). The gap between source clients also does not consistently widen. Removing local Lalign\mathcal{L}_{\mathrm{align}} or LGM\mathcal{L}_{\mathrm{GM}} lowers F1 for every source client, so both help retain client contributions. These results show no progressive loss of access to client-specific knowledge over the evaluated rounds, although they do not establish preservation of every individual fact.

Revision. We have added Table 24 and this statement to Appendix D.9 that client-specific preservation is an empirical property, not an explicit constraint.

AuthorsOpenReview ↗

Response to Q1

Q1 We thank the reviewer for recognizing graph-based cross-client knowledge sharing as a valuable idea, and agree that the submission did not foreground it enough. We have revised the paper accordingly.

Main innovation. SynFedRAG makes a learned graph the medium of cross-client knowledge transfer. Prior work transfers knowledge through model parameters, through remote retrieval at query time (e.g., DGRAG), or through aggregated memory items (e.g., FD-RAG). SynFedRAG instead learns a persistent synthetic graph, with node features and topology optimized jointly, that every client queries locally alongside its private graph.

Changes to the text.

  • The Abstract and Introduction now state this idea explicitly as the key idea of the paper.
  • The Introduction contrasts it with the three existing media of knowledge sharing, summarized in the new Table 1.
  • The five contributions are now three: the learned graph as the medium of transfer, the two-phase procedure that learns it, and the evaluation.

Changes to Figure 1.

  • Key component. The shared synthetic graph Msyn\mathcal{M}_{\text{syn}} is now highlighted.
  • Two training stages. In the Phase-II local optimization block, Stage 1 (prompt-module adaptation, LQA\mathcal{L}_{\text{QA}}) and Stage 2 (synthetic-graph refinement, Lmem\mathcal{L}_{\text{mem}}) are now drawn as separate paths, each marking what is trained and what is frozen.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.