acceptodds
Under review as a conference paper at ICLR 2027

FedDiG: Federated Continual Graph Learning with Disentangled Generative Replay

Abstract

Federated Continual Graph Learning (FCGL) is essential for real-world applications where graph data is both decentralized across clients and evolving over time. The primary challenge in this setting is catastrophic forgetting without centralizing raw client graphs. This forgetting manifests in two coupled aspects: the erosion of semantic decision boundaries and the degradation of structural relational patterns. Graph representations entangle node-level semantics with topology-dependent message passing, so a shift in either aspect can silently corrupt the other. In the federated continual setting this coupling is amplified: raw graphs are not centralized, precluding a shared raw-graph replay buffer, while heterogeneous topologies fragment old knowledge. Existing methods generally do not explicitly disentangle semantic and structural factors at the representation level, limiting factor-specific retention under drift. We propose FedDiG (Federated Disentangled continual Graph learning), which mitigates forgetting through post-hoc dual-gradient feature masking and generative replay. FedDiG derives complementary semantic and structural channels, trains a client-side generator on the current local graph, and applies server-side distillation that aligns both channels. Extensive experiments on seven datasets show that FedDiG achieves the highest average accuracy and lowest average forgetting among the compared baselines.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Federated Continual Learning, Federated Graph Learning, Continual Graph Learning, Catastrophic Forgetting, Generative Replay, Representation Disentanglement, Graph Neural Networks

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_bUgmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies federated continual graph learning (FCGL), where clients train a shared GNN over sequential tasks without centralizing raw graphs. It identifies two failure modes, semantic overlap and structural forgetting, and proposes FedDiG. After each task, a post-hoc dual-gradient scoring of hidden dimensions (cross-entropy gradient versus adjacency-reconstruction probe gradient, Eq. 2 to 4) splits representations into a semantic channel and a structural channel. Each task trains a client-side generator and link predictor on the current local graph, and the server aggregates them, synthesizes per-task subgraphs calibrated by a high-frequency roughness score, and performs dual-channel distillation of a frozen teacher on the disjoint union of past synthetic graphs (Eq. 13 to 17). Experiments on seven datasets with sixteen baselines from four families report the highest mean average accuracy and lowest mean average forgetting, with ablations on the four loss terms, the mask ratio, and the channel split.

Critical strengths
  1. The empirical study is broad and well controlled. Table 1 covers seven datasets including heterophilic graphs, sixteen baselines from four families, and five seeds with standard deviations, all under a unified federated protocol, which is more careful than most FCGL papers.
  2. The ablations genuinely isolate design choices. Table S4 separates the topology loss from link-predictor training, Table 3 shows a random mask collapses to 54.3 AA on Cora while the semantic-gradient mask alone recovers only part of the gap, and the channel probes in Figure 4 plus the retention analysis in Figure 5 connect the design to old-task retention (72.5% old-class accuracy versus 11.4% for POWER).
Critical weaknesses
  1. The claimed disentanglement is only weakly supported. The channel split is a post-hoc top-r mask on two gradient scores with a fixed ratio r of 50%, and Figure 4 shows the split is far from clean, since HS keeps 70.58% class accuracy against 74.85% for the full representation while HR still retains 65.0% structure AP. It is therefore unclear how much of the gain comes from channel separation rather than from generative replay itself, especially since Fed-DSLR with real-node replay stays competitive in AA on several datasets in Table 1.
  2. The evaluation protocol simplifies both the federated and the continual aspects. Main results use only K = 3 clients with full participation, contiguous class blocks as tasks with leftover class IDs discarded, and inter-task edges excluded for all methods. Partial participation and client-scale sweeps are deferred to the appendix, so the method's behavior under more realistic federation regimes remains largely unverified in the main evidence.
Questions for authors

Refer to weaknesses.

1 comment
AuthorsOpenReview ↗

Follow-up to Reviewer bUgm: How Much of the Gain Comes from the Channel Split?

Thank you again for your review. We have not yet heard back during the discussion, so we would like to share one additional controlled experiment that directly addresses your main question in W1: how much of the gain comes from channel separation rather than from generative replay itself.

In short: rows A1 and A3 below use identical generative replay, identical distillation losses and identical budgets; the only difference is whether the distillation losses read the split channels. This isolated change gives +7.1 AA / −13.8 AF on Cora and +1.8 AA / −10.6 AF on CiteSeer, so a substantial part of the gain is attributable to the split and not to replay alone.

Stage-wise and loss-wise isolation of the split (AA ↑ / AF ↓, %)

RowSplit in generator objectivesSplit in server KDKD lossesCoraCiteSeer
A1✗✗CE + contrast + sim66.8 / 35.268.5 / 27.4
A2✓✗CE + contrast + sim71.1 / 32.473.3 / 26.1
A3✗✓CE + contrast + sim73.9 / 21.470.3 / 16.8
A4 (FedDiG)✓✓CE + contrast + sim75.6 / 20.375.6 / 14.7
B1✗✗CE only62.3 / 42.661.4 / 31.2
B2✓✓CE only65.8 / 38.161.8 / 23.1

Every row uses generative replay with the same generator, replay budget, communication rounds, local epochs and distillation epochs; where the split is not used, the contrast and sim losses act on the full representation H. The split helps both when used only in the generator (A2) and only in distillation (A3), the two uses are complementary (A4), and even with CE alone the split lowers forgetting (B1 → B2).

On the split not being perfectly clean. We agree that H_S and H_R are not fully disentangled (70.6% class accuracy in H_S, 65.0% structure AP in H_R). The split is not intended as a hard disentanglement; its role is to route semantic and relational supervision to the dimensions most responsive to each. The A1 vs. A3 contrast shows that this routing yields a clear gain even though the channels overlap.

On W2. In our earlier response we added results with inter-task edges retained in both training and evaluation, longer task sequences (7-task Cora, 6-task CiteSeer), and random class orders; FedDiG achieves the highest AA and lowest AF in all of these settings.

We would sincerely appreciate it if you could take a look and share any remaining concerns, and we would be grateful if you could take these results into account in your final assessment.

Reviewer_1DXdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

FedDiG stores a synthetic subgraph for each task and replays these graphs to reduce forgetting in a federated graph model. After local training, classification and adjacency-reconstruction gradients divide hidden features into semantic and structural channels; clients train a generator for the current task, and the server distills both channels using the stored synthetic graphs. Experiments compare sixteen baselines on seven datasets, reporting an average accuracy of 54.7% and average forgetting of 14.8% across datasets, the best aggregate results among the compared methods.

Critical strengths

1.The comparison covers seven datasets, sixteen baselines, and five random seeds across different graph structures, with Table 1 showing a 5.3 percentage-point gain in mean accuracy over FedFST and a reduction in mean forgetting from 22.7% to 14.8%. 2.The full method outperforms the unsplit, random-mask, and semantic-gradient variants in Table 3, while Figure 4 shows that the two channels retain different amounts of class and structural information, supporting the proposed split.

Critical weaknesses

1.Table S5 uses only two to four tasks per dataset in a fixed class order and removes inter-task edges during training and evaluation, so the results mainly support short class-incremental sequences and provide limited evidence for changing connections between old and new nodes. 2.FedDiG stores 80 synthetic nodes per class and adds generator training and server distillation, but baseline replay-memory budgets are not listed and runtime comparisons are limited to Cora, leaving its advantage under equal resource budgets uncertain.

Questions for authors

1.Does FedDiG retain its advantage over FedFST and MOTION when inter-task edges between already observed nodes are preserved and evaluation uses several random class orders? 2.Could the authors report the actual memory use of each replay method and compare results under equal memory and training-time budgets to assess how much the gains depend on additional computation? 3.Could the authors report current- and previous-task accuracy before and after server distillation and compare historical-only replay with replay that also includes the current task's synthetic graph? 4.How do accuracy, forgetting, and server distillation time change over longer task sequences with a fixed total memory budget, and does the method still need to retain a synthetic subgraph for every previous task?

Reviewer_QKYrOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FedDiG, a federated continual graph learning framework that addresses catastrophic forgetting from both semantic and structural perspectives. The method consists of three key components: dual-gradient feature disentanglement to separate semantic and structural representations, generative graph replay to preserve historical knowledge without storing raw graphs, and dual-channel knowledge distillation to maintain semantic consistency and structural relationships. Experiments on seven graph datasets against sixteen baselines demonstrate improvements in average accuracy and average forgetting, supported by ablation studies and additional analyses.

Critical strengths

S1. FedDiG demonstrates strong empirical performance across seven graph datasets, outperforming sixteen baselines in terms of average accuracy and average forgetting. The extensive experiments, including ablation studies and robustness analyses, provide evidence of its effectiveness. S2. the proposed framework offers a meaningful approach to addressing semantic and structural forgetting separately. The dual-gradient feature disentanglement mechanism, combined with generative replay and dual-channel distillation, provides a coherent solution for preserving historical knowledge in federated continual graph learning.

Critical weaknesses

W1. the experimental protocol only considers relatively short task sequences and excludes inter-task edges, which limits its ability to demonstrate the effectiveness of FedDiG under realistic graph evolution and long-term continual learning. W2. although the ablation studies support the effectiveness of individual components, the contribution of dual-gradient feature disentanglement is not sufficiently isolated from the benefits of generative replay and knowledge distillation. Furthermore, the substantial additional computational overhead raises concerns about the practicality and efficiency of the proposed framework.

Questions for authors

Q1. Could the authors evaluate FedDiG on longer task sequences with evolving graph topologies and inter-task edges to better demonstrate its effectiveness in realistic continual learning scenarios? Q2. Could the authors provide a more controlled comparison to isolate the contribution of dual-gradient feature disentanglement from generative replay and knowledge distillation, ideally under matched computational budgets? Q3. How well do the generated graphs preserve the structural properties of the original graphs? Could the authors provide quantitative comparisons using metrics such as degree distributions, clustering coefficients, or spectral properties?

Comments on the paper

2 comments
Area_Chair_2HbPOpenReview ↗

Please engage in the rebuttal

Dear Reviewers,

As we enter the final stage of the author-reviewer discussion, I would like to encourage everyone to actively engage with the authors’ responses and any additional evidence they have provided.

Please take this opportunity to reconsider your initial assessments with an open mind, particularly where the authors have offered substantive clarifications, detailed arguments, or additional experimental results. The goal of the discussion is not to defend initial positions, but to reach the fairest and most informed evaluation of the submission.

I encourage you to engage directly with the authors on any remaining concerns, clearly distinguishing between issues that have been satisfactorily addressed and those that remain unresolved. Where appropriate, please update your reviews and scores accordingly, or explain why your original assessment still stands.

Your active and constructive participation in these final exchanges is essential to reaching a fair, evidence-based decision, and so far only Reviewer QKYr has begun to engage in the discussion.

Your AC

Summary of Contributions and Discussion

Dear Area Chair and Reviewers,

Thank you for your time during the review and discussion. As the discussion closes, we summarize the paper's contributions, its findings, and what the revision added.

Core contributions

  1. A two-factor view of forgetting in federated continual graph learning. Forgetting on graphs has a semantic part, where class decision boundaries erode, and a structural part, where relational patterns degrade. Message passing entangles the two, and raw client graphs cannot be pooled into a shared replay buffer. We retain the two parts separately.
  2. Gradient-based disentanglement that costs nothing to train. After each task, a dual-gradient mask compares the classification gradient with an adjacency-reconstruction gradient and splits the hidden dimensions into a semantic channel HSH_S and a structural channel HRH_R. It adds no trainable parameters and leaves local training as plain cross-entropy. The structural score uses edges, not labels, so it does not assume homophily.
  3. Replay without raw data. Each client trains a generator and a link predictor on its current graph only. The server aggregates them and synthesizes one compact subgraph per task, calibrated to the clients' high-frequency roughness. No real node is stored for replay, and earlier generators are never retrained.
  4. Dual-channel distillation. On the past synthetic subgraphs, the server aligns HSH_S with the previous model's class prototypes and aligns pairwise similarities in HRH_R. This runs only at the server and takes seconds.

Main findings

  1. Best accuracy on every dataset. Against sixteen baselines over five seeds, FedDiG has the highest AA on all seven datasets (tied on one), including the heterophilic ones, and the best mean AA and AF (Table 1).
  2. The two channels specialize. HSH_S keeps most of the class information and HRH_R most of the structural information, and HSH_S is far more consistent across clients than HRH_R (Figure 4).
  3. The disentanglement accounts for a large part of the gain, separately from replay. With replay, distillation losses and budgets held identical, letting the distillation losses read the two channels gives +7.1 AA and −13.8 AF on Cora. A random mask is worse than no split, so the gain comes from which dimensions the gradients select.
  4. Old knowledge is retained in both channels. FedDiG keeps far higher old-class accuracy than POWER and FedAvg (Figure 5), and gives the lowest semantic and structural drift on held-out historical nodes.
  5. The advantage holds beyond the standard protocol. FedDiG has the highest AA and the lowest AF against FedFST and MOTION with inter-task edges kept, on longer sequences, and under random class orders. It also stays ahead of Fed-DSLR under partial participation with 10 clients.
  6. The cost is modest. FedDiG uses less replay storage and training time than FedFST and MOTION, and stays accurate at one eighth of the replay budget, with a moderate rise in forgetting.

What the discussion added

Reviewer QKYr confirmed that the protocol concern was resolved and, after the stage-wise isolation experiment, raised the rating from 2 to 3 (weak acceptance). Reviewer 1DXd (3) wrote that our "explanations and results address my questions and strengthen the paper." Reviewer bUgm (2, weak rejection) has not replied. The two weaknesses in that review are close to points the other reviewers raised and now consider addressed: W1, how much of the gain comes from the disentanglement and how much from replay, is Reviewer QKYr's W2; the continual-learning part of W2 is W1 of both other reviewers. We answered both in our response and follow-up to Reviewer bUgm. The new evidence, marked in blue in Appendix J of the revised PDF except for the stage-wise table:

  • Disentanglement versus replay (bUgm W1, QKYr W2): the stage-wise isolation table, posted in our follow-ups and to be added to the paper, and the representation-drift measurements (Table S11). The mask is meant to route semantic and relational supervision to the dimensions most responsive to each, not to make the channels independent.
  • Protocol (QKYr W1, 1DXd W1, bUgm W2): inter-task edges, longer sequences and random class orders (Tables S8–S10); client scale and partial participation (Figure 6, Table S3).
  • Cost (QKYr W2, 1DXd W2): replay storage, training time, a replay-budget sweep and distillation time (Tables S12–S14).
  • Server distillation (1DXd Q3): it brings historical-task accuracy back from zero, at a cost of a few points on the current task (Table S15).
  • Generated subgraphs (QKYr Q3): roughness and homophily stay close to the real graphs; average degree and clustering shift under compression, as Table S17 reports.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.