acceptodds
Desk rejected at ICLR 2027

MemEvo-FCGL: Structural Memory Evolution for Federated Continual Graph Learning

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Federated continual graph learning (FCGL) enables distributed clients to collaboratively learn from sequential graph-task views without sharing local observations. However, clients must preserve previous knowledge while adapting to new graph patterns, and the server must coordinate structurally heterogeneous clients. Existing methods often describe task evolution using coarse task- or label-level statistics and preserve historical knowledge through raw-sample replay, limiting their ability to capture structural changes. We propose MemEvo-FCGL, a structural memory evolution framework for local knowledge retention and cross-client coordination. Each client learns memory keys that distill node representations into a compact state describing the current task's structural patterns. The server compares current states along client memory-state trajectories to adapt model aggregation and prototype-based knowledge transfer, reducing interference from structurally incompatible updates. Meanwhile, a bounded abstract key-value memory merges similar historical patterns, dynamically reorganizes novel patterns under a fixed capacity, and retrieves relevant knowledge through attention. Across six protocol-matched node-classification benchmarks, MemEvo-FCGL attains the highest final average performance on five while retaining a bounded number of abstract memory slots. These results demonstrate the effectiveness of structural memory evolution for continual learning over decentralized class-incremental graph streams.The code is available at https://anonymous.4open.science/r/anonymous_accept.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Federated Continual Graph Learning, Graph Neural Networks, Structural Memory Evolution

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_DJR1OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

MemEvo-FCGL addresses federated class-incremental node classification through shared structural keys, bounded local key–value memory, and historical-memory retrieval. Client memory states guide server aggregation and knowledge transfer. The method reports the highest final average accuracy on five of six datasets against the protocol-matched baseline subset.

Critical strengths

Connects local knowledge retention and server coordination through a common representation.

Shows substantial gains on several datasets, particularly Coauthor CS, with five-run results and paired-seed comparisons.

Includes component ablations and task-wise accuracy/forgetting analyses.

Clearly discloses transductive access, unmatched comparisons, and the absence of formal privacy guarantees.

Critical weaknesses

MOTION, a directly relevant FCGL baseline, is not uniformly protocol-matched. Cora and CiteSeer also have task or training-budget mismatches. These disclosures are helpful, but controlled comparisons with the closest competitors remain necessary.

States summarize assignments of graph-conditioned embeddings, which reflect both features and topology. The experiments do not establish whether they capture structural compatibility beyond feature or class distributions, or whether state similarity predicts update compatibility.

Coordination combines aggregation and teacher selection, while memory evolution combines allocation, merging, and compression. Removing entire modules cannot identify which mechanism produces the gains. Comparisons against simpler fixed-memory alternatives are missing.

All experiments use three clients with full participation and short task streams. Robustness to more clients, partial participation, different task orders, and stronger client heterogeneity is unclear.

A fixed slot count does not establish storage or runtime efficiency. The paper lacks measured memory bytes, communication costs, and runtime comparisons, including server reference-graph reconstruction.

Questions for authors

Match MOTION and POWER under identical task splits, training budgets, and comparable memory budgets.

Compare structural-state coordination against label histograms and simple embedding summaries; measure its relationship with gradient conflict or transfer benefit.

Separately ablate state-aware aggregation, teacher weighting, slot merging, and capacity-triggered compression.

Evaluate memory-capacity sensitivity, longer streams, more clients, and partial participation.

Explain how historical keys and values remain compatible with the evolving encoder, and report sensitivity to representation drift.

9 comments

1. Controlled comparison with MOTION and POWER

We retained the baseline settings to document performance under the configurations used to reproduce or adapt each method. On Cora and CiteSeer, our MOTION reproduction used two clients, one new class per task, and one communication round with 50 local epochs per task. Keeping these settings preserved the graph partitions and task boundaries that determine what MOTION can coarsen and replay, as well as the synchronization schedule that shapes its conflict-aware updates. For FedAvg, FedDyn, and FedSSP, we retained the optimization schedules of our existing FCGL adaptations. This preserved both the amount of local training between synchronizations and the frequency with which the server combined updates, which together determine the optimization regime of each adaptation.

The original Cora and CiteSeer results for MemEvo-FCGL and POWER also came from different benchmark configurations. On Cora, the MemEvo-FCGL task builder kept only complete two-class groups, producing three tasks with exactly two new classes each. The POWER configuration retained the seventh class as a fourth task. That extra task expands the final label space and adds a training stage over which earlier classes must be retained. On CiteSeer, all six classes fit into three two-class tasks. The saved MemEvo-FCGL configuration used 10 communication rounds with three local epochs per round, and the POWER configuration used 15 rounds with five local epochs per round. This difference affects both the amount of local training and the number of server aggregations per task. The submitted paper disclosed these settings and used the unmatched results as references for their respective protocols, restricting its common-protocol conclusions to the matched subset.

Experiments under a shared protocol. We conducted a controlled comparison of MemEvo-FCGL, MOTION, and POWER on Cora and CiteSeer. For each of five paired seeds, the three methods shared the same three Louvain client partitions, class order, and 20/40/40 train/validation/test masks. All three used a GAT backbone and 15 communication rounds per task with five local epochs per round. The Cora experiment covers all seven classes across four tasks (2, 2, 2, and 1 classes), and the CiteSeer experiment covers six classes across three two-class tasks. Each method used its own memory configuration. A comparison under a common persistent-memory budget remains open.

We report final average accuracy (AM) and forgetting (FM) as mean ± standard deviation over the five seeds.

DatasetMethodAM (%) ↑FM (%) ↓
CoraMemEvo-FCGL68.35 ± 3.345.77 ± 4.81
CoraMOTION64.73 ± 1.257.50 ± 2.98
CoraPOWER40.28 ± 10.5511.74 ± 10.87
CiteSeerMemEvo-FCGL67.49 ± 1.2112.42 ± 1.14
CiteSeerMOTION55.64 ± 1.357.90 ± 1.68
CiteSeerPOWER22.72 ± 5.2535.15 ± 15.24

The main accuracy conclusion holds under the shared protocol. MemEvo-FCGL achieves the highest mean AM among the three methods on both datasets, exceeding MOTION by 3.62 percentage points on Cora and 11.85 on CiteSeer. MOTION has lower mean forgetting on CiteSeer. On the other six datasets, the reported results use common-protocol experiments for MOTION and archived experiments for MemEvo-FCGL and POWER.

Thank you for emphasizing the need for a controlled comparison. We have included the results obtained under the shared protocol in the revised appendix.

2. Structural states as an aggregation signal

The structural state compares clients by the distribution of their node embeddings over aligned keys. In Eq. (2), the base GNN embedding hk,i\mathbf{h}_{k,i} of node ii is softly assigned to the PP structural keys at client kk:

u_{k,i,p}=\left(1+\\|\mathbf{h}_{k,i}-\mathbf{k}_{k,p}\\|_2^2/\tau_s\right)^{-(\tau_s+1)/2}, \qquad q_{k,i,p}=\frac{u_{k,i,p}}{\sum_{p'=1}^{P}u_{k,i,p'}}. \tag{2}

Equation (4) averages these assignments over the task-visible nodes Vkt\mathcal{V}_k^t:

\mathbf{s}_k^{t,r}=\frac{1}{|\mathcal{V}_k^t|}\sum_{i\in\mathcal{V}_k^t}\mathbf{q}_{k,i}\in\Delta^{P-1}. \tag{4}

The server broadcasts the key bank so the same coordinates can be compared across clients. Equations (2) and (4) explain what this state can capture beyond the two controls. A change in neighbors can change the messages entering hk,i\mathbf{h}_{k,i} and hence the state, even with node features and class frequencies fixed. A label histogram stays the same under that change. The assignments also depend on the full embedding vector, so two clients with the same per-dimension means and standard deviations can occupy different keys if their embedding dimensions co-vary differently. The state depends jointly on features and connectivity through the GNN.

For aggregation, Eq. (5) combines the sample-count prior πk=nkt/∑ℓnℓt\pi_k=n_k^t/\sum_\ell n_\ell^t, where nkt=∣Vkt∣n_k^t=|\mathcal{V}_k^t|, with agreement ak=1K−1∑ℓ≠k(1+cos⁡(sk,sℓ))/2a_k=\frac{1}{K-1}\sum_{\ell\ne k}(1+\cos(\mathbf{s}_k,\mathbf{s}_\ell))/2 for K>1K>1 clients (a1=1a_1=1 for a single client):

\omega_k=(1-\lambda_a)\pi_k+\lambda_a\frac{a_k}{\sum_\ell a_\ell}, \qquad (\boldsymbol{\theta}_g,\mathbf{K}_g)=\sum_k\omega_k(\boldsymbol{\theta}_k,\mathbf{K}_k). \tag{5}

The normalized agreement term gives more weight to clients whose current states are closer to those of their peers, while the sample-count term preserves the contribution of data volume. The same state also informs teacher weighting in Eq. (6). For a reference node uu, the overlap between its assignment q~k,u\widetilde{\mathbf{q}}_{k,u} and client kk’s state gives a node-specific weight:

b_{k,u}=\sum_p\sqrt{\widetilde{q}_{k,u,p}s_{k,p}}, \qquad \alpha_{k,u}=\frac{b_{k,u}}{\sum_\ell b_{\ell,u}}. \tag{6}

Equation (5) measures agreement in occupied modes across clients, and Eq. (6) matches reference nodes to those modes. Both use representation similarity as a proxy for where an update or teacher may be useful. Applying the source update Δθk\Delta\boldsymbol{\theta}_k to the global model gives the following first-order approximation to the validation gain:

−∇Lℓ(θg)⊤Δθk-\nabla L_\ell(\boldsymbol{\theta}_g)^\top\Delta\boldsymbol{\theta}_k

where LℓL_\ell is the target validation loss. State similarity measures overlap in occupied modes, so whether it tracks this gain is an empirical question.

Controlled aggregation experiments. We evaluated the three signals over the original full task streams on Cora and Chameleon, using three clients and five paired seeds per dataset. We compared structural states with normalized histograms of current-task training labels and summaries formed from the per-dimension means and standard deviations of current-task node embeddings. We changed the similarity input to Eq. (5) and used the same mixing coefficient, λa=0.5\lambda_a=0.5, across the three variants.

The table reports the full-stream summary for final average accuracy (AP) and forgetting (FM), shown as mean ± standard deviation.

DatasetAggregation signalAP (%) ↑FM (pp) ↓
CoraLabel histogram70.09 ± 5.2514.06 ± 5.91
CoraEmbedding mean and standard deviation60.71 ± 4.1910.95 ± 1.46
CoraMemEvo-FCGL structural state79.65 ± 2.957.09 ± 2.20
ChameleonLabel histogram45.98 ± 1.3913.82 ± 0.90
ChameleonEmbedding mean and standard deviation48.79 ± 2.5111.74 ± 1.62
ChameleonMemEvo-FCGL structural state57.63 ± 0.966.42 ± 0.77

Relationship to transfer benefit. At rounds 1, 8, and 15 of each task, we computed all three similarities on the same client updates from the structural-state runs. For each directed source–target pair, we measured transfer gain as the reduction in the target client’s validation cross-entropy when using the source client’s locally updated model in place of the pre-aggregation global model. We computed Spearman correlations within each sampled round and averaged them over rounds and then over the five seeds.

DatasetTarget taskLabel histogram ρEmbedding summary ρStructural state ρ
CoraCurrent0.3930.4670.415 ± 0.052
CoraHistorical−0.0520.0880.035 ± 0.194
ChameleonCurrent−0.2990.114−0.011 ± 0.119
ChameleonHistorical0.375−0.1470.131 ± 0.143

On Cora’s current task, structural-state similarity was positively associated with transfer gain within rounds for all five seeds. The mean correlation was −0.024 when pooling observations across rounds within each seed. On Chameleon’s current task, the within-round association was near zero, and negative transfer occurred in 55.7% of pairs in the upper similarity quartile versus 10.0% in the lower quartile. For historical tasks on Chameleon, the corresponding rates were 39.3% and 72.0%. The signal with the highest within-round correlation depended on the dataset and task. Before server distillation, the mean validation cross-entropy of the three weighted aggregates differed by at most 0.0023 within any dataset and task group. Isolating the contribution of topology would require varying connectivity while holding features and labels fixed. Directly measuring gradient conflict would require gradient or update-vector alignment measurements.

Thank you for highlighting the distinction between structural similarity and transfer benefit. We will include the control settings and transfer analysis in the revised appendix.

3. Ablations of aggregation, teacher weighting, slot merging, and compression

The original ablations in Table 2 assess three contributions: coordinating clients through their structural states when local memory is available, retaining prior knowledge in a bounded buffer, and using the stored values in the current forward pass. We grouped aggregation and teacher weighting into the coordination module (A) because both use client states at the server. We grouped writing, merging, and compression into the memory evolution module (B) because these operations jointly determine what the buffer retains. Historical memory interaction (C) uses the retained values to augment current node representations. This decomposition also accounts for how the modules depend on each other. Removing A while retaining B and C tests server coordination with local retention and retrieval still available. Removing C while retaining B tests the added value of reading from a buffer that the client continues to maintain. Because C needs the slots produced by B, we compare the variants without C and without B and C to assess the effect of removing the evolving buffer when feature-level retrieval is already disabled.

The single-factor comparisons address the distinction you raised by isolating individual operations within A and B while keeping the rest of the method fixed. In A, state agreement can change the averaged model and key bank, and node-specific teacher weights can change the distillation signal. In B, a compatible new slot can update a historical slot even when there is room in the buffer, and compression determines how to fit the accumulated slots once capacity is exceeded. The task-boundary logs show when merging and compression were triggered, which determines when replacing either operation can affect the results. Single-factor experiments. We evaluated the four controls on the original task streams with five paired seeds per dataset.

  1. State-aware aggregation. Equation (5) mixes the sample-count prior πk=nkt/∑ℓnℓt\pi_k=n_k^t/\sum_\ell n_\ell^t with normalized state agreement. Here nktn_k^t counts task-visible nodes, and aka_k is the mean of (1+cos⁡(sk,sℓ))/2(1+\cos(\mathbf{s}_k,\mathbf{s}_\ell))/2 over the other clients. Setting λa=0\lambda_a=0 tests the role of state agreement in aggregation with the teacher weighting still active.

    \omega_k=(1-\lambda_a)\pi_k+\lambda_a\frac{a_k}{\sum_\ell a_\ell}, \qquad (\boldsymbol{\theta}_g,\mathbf{K}_g)=\sum_k\omega_k(\boldsymbol{\theta}_k,\mathbf{K}_k). \tag{5}

  2. Teacher weighting. Equation (6) weights each teacher by the overlap between its assignment for reference node uu and its current state. Uniform weights test whether this node-specific weighting adds value with state-aware aggregation and server distillation still active. b_{k,u}=\sum_p\sqrt{\widetilde q_{k,u,p}s_{k,p}}, \qquad \alpha_{k,u}=\frac{b_{k,u}}{\sum_\ell b_{\ell,u}}. \tag{6}

Reviewer_r53UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies federated class-incremental node classification where each client holds a private graph and tasks reveal new classes over time. MemEvo-FCGL organizes graph-conditioned embeddings around a shared set of learnable structural keys using Student-t assignments (Eq. 2 to 4). The same key space serves three purposes: state-aware aggregation that blends sample counts with cross-client structural agreement (Eq. 5), structure-conditioned knowledge transfer on a server reference graph with node-specific teacher weights (Eq. 6), and a bounded key-value memory that merges compatible slots and reclusters under capacity (Eq. 7 to 10), with top-K retrieval injecting stored values into current-node embeddings (Eq. 11). Training combines cross-entropy with state learning, task-balanced distillation on abstract slots, an old-new margin loss, key separation, and an EWC-style anchor (Eq. 12 to 14). On eight graphs and ten baselines, the method reports the highest final average accuracy on five of six protocol-matched datasets, including the heterophilous Squirrel.

Critical strengths
  1. Using one structural key space for both local retention and server coordination is a well-executed idea. The read-before-write cycle in Eq. 11 makes history part of current representation learning rather than only a regularizer on past examples, and the ablation in Table 2 shows each module is load-bearing, with removal of the coordination module dropping Cora AM from 79.65 to 31.25.
  2. The evaluation covers more structural variety than most FCGL papers. Eight graphs including the heterophilous Chameleon and Squirrel, five runs with standard deviations, and task-wise accuracy trajectories give a reasonably full picture of behavior, and the gains on Squirrel (53.14 AM against 19.92 for the next best) are large.
Critical weaknesses
  1. The headline comparison mixes protocols. Section 5.1 states that only six of the ten baselines share the controlled client, task, backbone, and training protocol, and MOTION, the strongest competitor and second-best overall in Table 1, is not among them, while some rows on Cora and CiteSeer are retained from uncontrolled runs. The central claim of ranking first on five of six datasets therefore partly reflects protocol differences rather than method quality.
  2. The setting is simplified in ways that favor the design. The graph topology is fixed across tasks, each task adds at most two new classes, there are only three Louvain-partitioned clients, and the coordination states are computed over all task-visible nodes including test-node features and edges. How the method behaves under edge evolution, larger class increments, or more clients remains untested, and the server-side reference graph reconstructed from prototype gradients inherits the privacy exposure noted in the POWER line of work.
Questions for authors

Refer to weaknesses.

4 comments

1. Protocol-matched comparison

On Cora and CiteSeer, our MOTION reproduction used two clients, one new class per task, and one communication round with 50 local epochs per task. Client partitions and task boundaries determine the neighborhoods and historical subgraphs available to MOTION's coarsening and replay. The local training schedule also shapes the direction and magnitude of the updates used in its conflict-aware aggregation. For FedAvg, FedDyn, and FedSSP, we retained the optimization schedules of our existing FCGL adaptations. On PubMed, for example, these were 12 rounds × 2 local epochs, 10 × 3, and 10 × 2, respectively, all with a learning rate of 0.003. The local epochs and learning rate affect how far clients train on their own subgraphs between synchronizations, while the number of rounds determines how often the server integrates those updates.

Existing results under a shared protocol. We had already completed 46 method–dataset comparisons with five paired seeds, covering all 11 methods on Cora and CiteSeer and FedAvg, FedDyn, FedSSP, and MOTION on the other six datasets. Within each dataset, the experiments used the same three-client Louvain partitions, class order, node splits, and GAT backbone, with 15 communication rounds per task and five local epochs per round. The Cora stream covers all seven classes (2, 2, 2, and 1 per task), and CiteSeer covers six classes in three two-class tasks. MemEvo-FCGL had the highest mean AM in both complete comparisons, ahead of MOTION by 3.62 pp on Cora and Fed-ERGNN by 8.82 pp on CiteSeer. MOTION had the lowest FM on CiteSeer.

Cells report AM (%) / FM (%), each as mean ± standard deviation over five seeds.

Cora and CiteSeer (all 11 methods)

MethodCoraCiteSeer
MemEvo-FCGL68.35 ± 3.34 / 5.77 ± 4.8167.49 ± 1.21 / 12.42 ± 1.14
FedAvg25.09 ± 0.13 / 95.05 ± 1.1430.01 ± 0.26 / 81.55 ± 1.64
FedDyn19.09 ± 6.23 / 66.59 ± 11.0317.86 ± 1.63 / 76.01 ± 5.04
FedSSP25.00 ± 0.00 / 92.57 ± 0.4829.41 ± 0.50 / 79.05 ± 1.71
MOTION64.73 ± 1.25 / 7.50 ± 2.9855.64 ± 1.35 / 7.90 ± 1.68
FedATH39.52 ± 1.55 / 73.16 ± 2.5631.45 ± 0.36 / 76.92 ± 1.68
Fed-ERGNN61.47 ± 5.43 / 34.79 ± 7.3258.67 ± 0.88 / 31.71 ± 2.45
Fed-TWP34.33 ± 4.49 / 81.85 ± 6.8648.02 ± 9.83 / 53.87 ± 15.60
FedAGC42.53 ± 2.92 / 70.20 ± 3.9932.65 ± 0.90 / 76.07 ± 1.80
Powder26.42 ± 6.46 / 53.54 ± 14.1431.48 ± 9.25 / 37.35 ± 13.00
POWER40.28 ± 10.55 / 11.74 ± 10.8722.72 ± 5.25 / 35.15 ± 15.24

Other six datasets (four baselines under the shared protocol)

MethodPubMedPhotoChameleonSquirrelCoauthor CSComputers
FedAvg50.00 ± 0.00 / 94.20 ± 0.1621.99 ± 0.97 / 98.17 ± 1.2434.16 ± 1.41 / 70.10 ± 2.8733.33 ± 0.00 / 63.42 ± 1.9221.87 ± 4.60 / 88.00 ± 5.2118.11 ± 0.67 / 98.04 ± 0.55
FedDyn50.00 ± 0.00 / 88.41 ± 3.9723.15 ± 1.98 / 73.99 ± 6.2341.25 ± 3.07 / 47.43 ± 7.6831.82 ± 7.69 / 46.13 ± 8.4918.10 ± 3.56 / 52.14 ± 16.3817.48 ± 9.50 / 66.90 ± 5.27
FedSSP50.00 ± 0.00 / 94.30 ± 0.0823.64 ± 0.16 / 97.87 ± 0.2133.33 ± 0.00 / 70.56 ± 0.8233.33 ± 0.00 / 65.36 ± 0.5112.50 ± 0.00 / 98.07 ± 0.0619.56 ± 0.29 / 97.43 ± 1.29
MOTION80.81 ± 3.23 / 24.22 ± 9.1081.63 ± 5.52 / 18.65 ± 8.2153.78 ± 2.60 / 8.92 ± 2.2848.19 ± 1.03 / 23.89 ± 1.3830.16 ± 4.64 / 9.36 ± 5.2071.64 ± 4.40 / 32.27 ± 5.34

We have included the complete protocol-matched results and evaluation settings in the revised appendix.

2. Experimental setting and evaluation scope

Original setup. We kept the underlying graph fixed to study forgetting driven by the arrival of new classes. Using at most two new classes per task created repeated opportunities to measure adaptation to new classes and retention of earlier ones. We used three Louvain clients because that is the smallest federation in which mean pairwise state agreement can differ across clients. With two clients, both agreement scores come from the same cosine similarity. A small number of Louvain partitions also preserved more within-client neighborhoods on the graphs used in our scale study. Full participation among eligible clients held the contributing set fixed within each task, making round-to-round changes in aggregation weights easier to interpret. Transductive access. Under the transductive protocol, clients compute structural states from GNN embeddings of task-visible nodes. The features and edges of validation and test nodes contribute to these embeddings, and their key assignments enter the state average. The assignments use no node labels, and supervised objectives use only training-node labels. Consequently, the results apply to settings where those features and edges are available at training time. A training-node-only state control and an inductive evaluation remain open.

The task-sequence, client-scale, and partial-participation experiments were already completed before the review. We report those results below to show how performance varies beyond the main benchmark setting.

Larger class increments and longer task streams. On Coauthor CS, we regrouped the same 15 classes into 3, 4, 5, and 8 tasks, with up to 5, 4, 3, and 2 new classes per task, respectively. For each seed, we held the class order, client partitions, and train/validation/test membership fixed. All three clients participated in every round. We evaluated two budgets: 15 rounds per task and 120 total rounds. The eight-task setting is shared by both budgets. We compared MemEvo-FCGL with POWER.

With five new classes per task, MemEvo-FCGL achieved 85.86% macro accuracy at 15 rounds per task and 85.56% at 120 total rounds. At the fixed 120-round budget, its accuracy ranged from 84.50% to 86.18% across the four streams. Its mean accuracy was higher and its mean forgetting was lower than POWER's in all seven settings. On the eight-task stream, MemEvo-FCGL reached 84.50% accuracy and 6.50 pp forgetting, compared with 57.25% and 33.43 pp for POWER.

Each cell reports final 15-class macro accuracy (%) / class-level forgetting (pp), as means ± sample standard deviations over five seeds.

TasksRounds/taskTotal roundsMemEvo-FCGLPOWER
3154585.86 ± 0.84 / 4.75 ± 1.5957.25 ± 9.36 / 14.05 ± 6.57
4156086.29 ± 0.24 / 4.33 ± 1.3466.04 ± 7.80 / 22.25 ± 4.76
5157585.78 ± 0.49 / 4.62 ± 1.0664.48 ± 2.65 / 19.01 ± 5.25
81512084.50 ± 2.60 / 6.50 ± 2.1257.25 ± 8.32 / 33.43 ± 6.27
34012085.56 ± 1.04 / 4.70 ± 1.3961.16 ± 6.68 / 20.47 ± 5.66
43012086.11 ± 0.87 / 4.32 ± 1.3667.87 ± 7.00 / 19.43 ± 7.87
52412086.18 ± 0.25 / 4.35 ± 0.9567.43 ± 7.68 / 22.13 ± 7.63

Client scale. We evaluated MemEvo-FCGL on Cora and Squirrel with 2, 3, 5, 15, 20, 30, and 50 Louvain-partitioned clients, using five seeds per setting. We retained each dataset’s original class schedule and training budget, including the three-task Cora stream. Only clients with labeled training nodes for the current task participated in local updates and aggregation.

At five clients, AM was within 0.4 percentage points of the three-client result on both datasets. At 15 clients, AM fell to 65.01% on Cora and 31.82% on Squirrel. At 50 clients, the corresponding values were 52.62% and 26.80%. The fraction of edges crossing client partitions also increased, from 9.47% to 56.27% on Cora and from 26.90% to 91.18% on Squirrel between three and 50 clients. The accuracy decline occurred alongside a reduction in the fraction of edges available within each client.

ClientsCora AM (%) ↑Cora FM (%) ↓Squirrel AM (%) ↑Squirrel FM (%) ↓
280.50 ± 1.566.58 ± 2.9745.99 ± 2.087.43 ± 3.10
379.65 ± 2.647.09 ± 1.9753.14 ± 4.2621.10 ± 3.16
579.77 ± 1.195.17 ± 1.3652.74 ± 1.4517.11 ± 1.69
1565.01 ± 2.4219.43 ± 2.5331.82 ± 1.6017.68 ± 1.53
2061.82 ± 5.3825.19 ± 6.3928.08 ± 1.7815.69 ± 0.79
3053.71 ± 1.3835.09 ± 3.7428.53 ± 0.7818.50 ± 0.94
5052.62 ± 1.5928.45 ± 0.9226.80 ± 1.2616.55 ± 0.31

Partial participation. At 15 and 30 clients, we also sampled all, two-thirds, one-half, or one-third of clients with current-task training labels per round. All methods shared the same sampling schedules within each seed. The table reports MemEvo-FCGL's mean final AP and the reduction in bidirectional tensor payload at one-third participation, measured against the full-participation run from the same experiment.

DatasetClientsFull AP (%)2/3 AP (%)1/2 AP (%)1/3 AP (%)Payload reduction at 1/3
Cora1564.9464.5460.4456.9156.84%
Cora3053.8646.0542.0640.8657.64%
Squirrel1531.8231.2331.3231.8858.33%
Squirrel3028.5328.3927.6427.8358.29%

At one-third participation, Cora's mean AP dropped by 8.02 pp at 15 clients and by 13.00 pp at 30 clients. On Squirrel, it stayed within 0.90 pp of full participation at all tested fractions. MemEvo-FCGL had higher mean AP than POWER in all 16 tested settings, and POWER had lower mean forgetting in all eight Squirrel settings. Client-level performance could still be low. With 30 Squirrel clients and one-third participation, the 10th-percentile client AP was 2.46% for MemEvo-FCGL and 13.71% for POWER.

Privacy. In federated class-incremental node classification, MemEvo-FCGL uses bounded structural key–value memory to retain and reuse knowledge from previous tasks, mitigating client-side forgetting. It also uses state-aware aggregation and teacher weighting to address conflicts among client updates and guide knowledge transfer. The aggregation in Eq. (4) limits direct disclosure during coordination. Client kk uploads the aggregate state. For structural key pp, the corresponding coordinate is

sk,pt,r=(qk,1,pt,r+qk,2,pt,r+⋯+qk,Nkt,pt,r)/Nkt, p=1,…,P.s_{k,p}^{t,r} = (q_{k,1,p}^{t,r} + q_{k,2,p}^{t,r} + ⋯ + q_{k,N_k^t,p}^{t,r}) / N_k^t,   p = 1, …, P.

Here, tt is the task, rr is the communication round, NktN_k^t is the number of task-visible nodes on client kk, and qk,i,pt,rq_{k,i,p}^{t,r} is node ii's soft assignment weight for key pp among the PP structural keys. Different sets of node-level assignments can produce the same mean, so the uploaded state alone generally cannot uniquely determine individual assignments. Clients also keep their memory values, masses, and semantic targets local.

Thank you for raising these points. We have added the task-sequence, client-scale, and partial-participation results and clarified the privacy implications of structural-state sharing in the revised appendix.

Dear reviewer r53U. Thank you for your time and effort to review this article. Your comments are valuable in improving the quality of this article. Since the author-reviewer discussion is coming to an end, we gently request that you read the rebuttal and whether if your concerns are addressed. Thanks again for your time and help in improving the quality of this article. Regards from the authors.

Reviewer_AVKSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies federated continual graph learning for transductive, semi-supervised class-incremental node classification, where each client owns a private graph and observes a sequence of task-induced views with disjoint class sets. The authors argue that label counts and raw-sample replay cannot describe structural task evolution, and propose MemEvo-FCGL, which couples local retention and server coordination through a shared structural key space. Clients learn DEC-style Student-t assignments of GNN node embeddings to P learnable structural keys, forming simplex-valued memory states that guide state-aware aggregation and structure-conditioned teacher selection on a server reference graph reconstructed from prototype gradients. At task boundaries, each client consolidates a bounded abstract key-value buffer through class-balanced writes, similarity-based merging, and capacity-triggered reclustering, and during the next task each node retrieves the Kh most similar slots through attention to obtain history-enhanced embeddings. On six protocol-matched benchmarks the method reports the highest final average performance on five and the second best on Chameleon, with ablations and task-wise analyses supporting the coupled design.

Critical strengths
  1. Motivation: The problem is significant and well positioned. Section 1 and Figure 1 articulate a coupled view of FCGL failure modes: equal label counts can hide different feature-neighborhood organization, a fixed memory budget must be reallocated as task views shift, and count-based aggregation cannot detect structurally incompatible updates. Treating local forgetting and cross-client interference as coupled rather than independent is a useful framing, and including heterophilous graphs makes the structural argument concrete, since label statistics are least informative exactly there.

  2. Method design: The design is coherent: the same structural keys serve server coordination in Section 4.1 and local retention in Sections 4.2 and 4.3, so no separate coordination signal is needed. The bounded abstract buffer in Eq. 7 to Eq. 10 stores summed values, masses, and semantic targets rather than raw nodes, which decouples retention from a fixed set of exemplars, and the read-before-write interaction in Eq. 11 injects history into current embeddings without labels or task identifiers at inference. Individual components are standard, namely DEC assignments, EWC-style encoder stability, and temperature-scaled distillation, but the integration is clean, and the privacy posture is stated explicitly: raw records, abstract values, and targets never leave the client.

  3. Experiments: The evaluation is broad and honest. Eight datasets span citation, product, coauthor, and heterophilous graphs; ten baselines cover FL, FGL, federated CGL adaptations, FCL, and FCGL; each cell reports mean and standard deviation over five runs. The authors explicitly separate protocol-matched from unmatched comparisons in Section 5.1 and Appendix B, openly discuss the failure case on Chameleon where Fed-TWP wins both metrics, and point out in Section 5.2 that low FM can merely reflect failure to learn new tasks, as seen for POWER and Fed-ERGNN on PubMed. The ablation in Table 2 uses paired seeds and correctly notes that module effects are not additive because C depends on B, and the task-wise curves in Figure 3 show when the endpoint advantage arises rather than only reporting final numbers.

Critical weaknesses
  1. The federation scale is very small. All experiments use three clients obtained by Louvain partitioning of a single graph. With K=3 the agreement term in Eq. 5 averages over only two other clients, and partitioning-induced heterogeneity is milder than naturally distributed client graphs. Claims about cross-client coordination are therefore only weakly tested, and it is unclear whether the method or its overheads scale to tens of clients.

  2. No efficiency analysis despite an efficiency-motivated design. The conclusion itself admits that a bound on memory slots alone does not establish lower runtime or storage cost. There is no measurement of communication bytes per round, wall-clock time, or memory footprint against FedAvg, POWER, or MOTION, even though the server additionally reconstructs a reference graph from prototype gradients, computes per-node teacher weights in Eq. 6, and refines the global model every round, and clients upload states and round-one prototype gradients. These costs are never quantified.

  3. Heavy hyperparameter burden with no sensitivity analysis. The method introduces many knobs, including lambda_a, tau_s, delta_key, delta_z, mu, lambda_z, Kh, tau_h, sigma, gamma, and several loss weights in Eq. 14, but the paper fixes them per Table 4 and Appendix B without any sensitivity study. The ablation removes whole modules but never varies B, P, or Kh, so it is impossible to tell how robust the gains are to a smaller memory budget or whether the reported settings are near a fragile operating point.

Questions for authors

Please refer to the weaknesses.

4 comments

1. Federation scale and cross-client coordination

We chose three Louvain-partitioned clients for the main benchmark after comparing different federation sizes on Cora and Squirrel. Three is the smallest number of clients for which the structural-agreement scores can differ across clients. With two clients, both scores are determined by the same cosine similarity. For K>1K>1, the agreement score is ak=1K−1∑ℓ≠k(1+cos⁡(sk,sℓ))/2a_k=\frac{1}{K-1}\sum_{\ell\ne k}(1+\cos(\mathbf s_k,\mathbf s_\ell))/2, where sk\mathbf s_k is the current structural state. Equation (5) combines this score with the sample-count prior πk=nkt/∑ℓnℓt\pi_k=n_k^t/\sum_\ell n_\ell^t.

\qquad (\boldsymbol{\theta}_g,\mathbf{K}_g)=\sum_k\omega_k(\boldsymbol{\theta}_k,\mathbf{K}_k). \tag{5}$$ Moving from three to five clients changed mean final accuracy by at most 0.4 percentage points on either dataset. At 15 clients, accuracy fell on both datasets and more edges crossed partition boundaries. The three-client setting let us examine how structural agreement affects aggregation while retaining more of the original neighborhoods within clients. Full participation kept the contributing clients fixed across rounds within each task, making it easier to relate changes in aggregation weights to changes in their structural states. **Client-scale experiments.** On Cora and Squirrel, we evaluated 2, 3, 5, 15, 20, 30, and 50 Louvain-partitioned clients on the original task streams, using 15 rounds per task, five local epochs per round, and five paired seeds. All clients with training labels for the current task participated. At $K=50$, each round included 30–43 clients on Cora and 50 on Squirrel. The table reports final average accuracy (AM, mean ± SD) and mean bidirectional tensor payload per run. | Nominal clients | Cora AM (%) | Cora payload (MiB/run) | Squirrel AM (%) | Squirrel payload (MiB/run) | |---:|---:|---:|---:|---:| | 2 | 80.50 ± 1.56 | 75.41 | 45.99 ± 2.08 | 106.44 | | 3 | 79.65 ± 2.64 | 113.12 | 53.14 ± 4.26 | 159.67 | | 5 | 79.77 ± 1.19 | 187.74 | 52.74 ± 1.45 | 266.11 | | 15 | 65.01 ± 2.42 | 488.96 | 31.82 ± 1.60 | 798.34 | | 20 | 61.82 ± 5.38 | 620.32 | 28.08 ± 1.78 | 1,064.45 | | 30 | 53.71 ± 1.38 | 830.64 | 28.53 ± 0.78 | 1,596.67 | | 50 | 52.62 ± 1.59 | 1,254.47 | 26.80 ± 1.26 | 2,661.12 | Between 3 and 50 clients, the share of edges crossing partitions rose from 9.47% to 56.27% on Cora and from 26.90% to 91.18% on Squirrel. Over the same range, mean server-update time in the single-machine simulator rose from 0.77 to 21.81 s per round on Cora and from 0.66 to 11.80 s on Squirrel. The agreement calculation itself averaged below 0.8 ms per round at every tested scale. **Partial participation.** At 15 and 30 clients, we kept the same round and local-epoch budget and sampled all, two-thirds, one-half, or one-third of clients with current-task training labels in each round, rounding the sample size up. All methods shared the same client schedule for a given seed and setting. The table gives MemEvo-FCGL's final average task accuracy (AP, mean ± SD over five seeds) and the reduction in total tensor payload at one-third participation relative to full participation. | Dataset | Nominal clients | Full AP (%) | 2/3 AP (%) | 1/2 AP (%) | 1/3 AP (%) | Payload reduction at 1/3 | |---|---:|---:|---:|---:|---:|---:| | Cora | 15 | 64.94 ± 2.81 | 64.54 ± 2.55 | 60.44 ± 3.63 | 56.91 ± 3.43 | 56.84% | | Cora | 30 | 53.86 ± 2.81 | 46.05 ± 6.98 | 42.06 ± 2.96 | 40.86 ± 5.11 | 57.64% | | Squirrel | 15 | 31.82 ± 1.79 | 31.23 ± 0.80 | 31.32 ± 0.59 | 31.88 ± 2.23 | 58.33% | | Squirrel | 30 | 28.53 ± 0.87 | 28.39 ± 1.01 | 27.64 ± 1.25 | 27.83 ± 3.10 | 58.29% | MemEvo-FCGL had higher mean AP than POWER in all 16 settings. POWER had lower mean forgetting in all eight Squirrel settings. MemEvo-FCGL-Sample, which changes Eq. (5) to sample-count weighting and retains structure-conditioned teacher weighting, had higher mean AP in all six reduced-participation settings on Cora, with its largest gain of 6.40 percentage points at 30 clients and one-half participation. On Squirrel, the largest difference in mean AP between the two aggregation rules was 0.73 points. **Task-stream length.** On Coauthor CS, we regrouped the same 15 classes into 3, 4, 5, or 8 tasks with three fully participating clients. MemEvo-FCGL's mean final 15-class macro accuracy ranged from 84.50% to 86.29% at 15 rounds per task and from 84.50% to 86.18% at 120 total rounds. On the eight-task stream, it achieved 84.50% accuracy and 6.50 points of class-level forgetting, versus 82.11% and 9.10 points for MemEvo-FCGL-Sample and 57.25% and 33.43 points for POWER. Thank you for raising the question of how the method behaves at larger scales. The revised appendix includes the full results and partitioning details.

2. Communication, runtime, and memory footprint

In our Cora and CiteSeer measurements, server updates accounted for most of MemEvo-FCGL's runtime, and communication peaked in the first round of each task.

Where the costs arise. Each client averages the key assignments qk,i\mathbf q_{k,i} of its task-visible nodes Vkt\mathcal V_k^t to obtain its structural state.

\in\Delta^{P-1}. \tag{4}$$ For $K>1$ participating clients, the server computes $a_k=\frac{1}{K-1}\sum_{\ell\ne k}(1+\cos(\mathbf s_k,\mathbf s_\ell))/2$ and uses the sample-count prior $\pi_k=n_k^t/\sum_\ell n_\ell^t$. For $K=1$, it sets $a_1=1$. $$\omega_k=(1-\lambda_a)\pi_k+\lambda_a\frac{a_k}{\sum_\ell a_\ell}, \qquad (\boldsymbol\theta_g,\mathbf K_g)=\sum_k\omega_k(\boldsymbol\theta_k,\mathbf K_k). \tag{5}$$ The pairwise state comparisons cost $O(K^2P)$ per round. Each client uploads its $P$-dimensional state and $P\times d$ key bank and receives the aggregated keys. With embedding dimension $d$ and FP32 tensors, this adds $4K(2Pd+P)$ bytes per round to the model exchange. Prototype-gradient uploads and any gradient-encoder downloads add traffic in the first round of each task. On the reconstructed reference graph, $\widetilde q_{k,u,p}$ is the assignment of reference node $u$ to key $p$ under client $k$. The server computes its teacher weights as $$b_{k,u}=\sum_p\sqrt{\widetilde q_{k,u,p}\\,s_{k,p}}, \qquad \alpha_{k,u}=\frac{b_{k,u}}{\sum_\ell b_{\ell,u}}. \tag{6}$$ For $R$ reference nodes, computing the assignments costs $O(KRPd)$ per distillation step, and computing the teacher weights from them costs $O(KRP)$. Each step also requires teacher GNN forward passes and a global-model update. The server reconstructs reference nodes from prototype gradients at the start of each task. Each client keeps a local buffer $\mathcal B_k^t=\\{(\mathbf k_j,\mathbf v_j,m_j,\mathbf z_j)\\}_{j=1}^{M_k^t}$ with $M_k^t\le B$. A slot has a $d$-dimensional key and value, a scalar mass, and a $C$-dimensional class target, giving a bound of $B(2d+1+C)$ scalar entries plus per-slot metadata. To read from this buffer, the client scores at most $B$ keys per query and uses the top-$K_h$ set $\mathcal N_i^{K_h}$ in the historical read. $$\begin{aligned} a_{i,j}&=\operatorname{softmax}_{j\in\mathcal N_i^{K_h}} \\!\left(\frac{\cos(\mathbf h_{k,i},\mathbf k_j)}{\tau_h}\right), \\\\ \widehat{\mathbf h}_{k,i}&=\mathbf h_{k,i}+\sigma \sum_{j\in\mathcal N_i^{K_h}}a_{i,j}\frac{\mathbf v_j}{m_j}. \end{aligned}\tag{11}$$ **Resource measurements.** We profiled MemEvo-FCGL, FedAvg, POWER, and MOTION on Cora and CiteSeer with three Louvain clients, the same task splits and GAT backbone, 15 rounds per task, five local epochs per round, and five paired seeds. We measured uplink and downlink tensor payload across all clients, excluding network framing, and divided by the total rounds (60 on Cora and 45 on CiteSeer). Wall-clock time covers the full run in the single-CPU-worker simulator. Peak RSS is the operating system's high-water mark for the whole process. Time and RSS are mean ± standard deviation across seeds. | Dataset | Method | Payload (MiB/round) | Wall-clock time (s) | Peak RSS (MiB) | |---|---|---:|---:|---:| | Cora | FedAvg | 2.114 | 7.87 ± 0.14 | 480.78 ± 0.59 | | Cora | POWER | 2.433 | 51.25 ± 10.20 | 512.49 ± 5.82 | | Cora | MOTION | 2.114 | 27.74 ± 0.79 | 767.27 ± 1.32 | | Cora | MemEvo-FCGL | 2.462 | 55.89 ± 1.49 | 523.76 ± 4.21 | | CiteSeer | FedAvg | 5.438 | 11.46 ± 0.24 | 540.35 ± 0.06 | | CiteSeer | POWER | 6.327 | 70.52 ± 12.90 | 614.20 ± 17.20 | | CiteSeer | MOTION | 5.438 | 28.48 ± 0.53 | 911.64 ± 15.12 | | CiteSeer | MemEvo-FCGL | 6.341 | 70.40 ± 2.86 | 619.90 ± 5.03 | **Communication.** Mean payload per round was about 17% higher than FedAvg and MOTION and within 1.3% of POWER on both datasets. The first versus subsequent rounds of each task averaged 6.92 versus 2.14 MiB on Cora and 18.78 versus 5.45 MiB on CiteSeer. In the separate trace on the original three-task Cora stream, first-round prototype-gradient uploads and gradient-encoder downloads added 4.78 and 0.80 MiB, respectively. With $K=3$, $P=20$, and $d=64$, the recurring structural payload was 240 bytes of client states and 30 KiB of bidirectional keys per round, matching the formula above.

Runtime. MemEvo-FCGL took longer than FedAvg and MOTION on both datasets, and its mean runtime was close to POWER's, especially on CiteSeer. The server update accounted for 38.85 s on Cora and 51.86 s on CiteSeer (70% and 74% of the full run). Gradient-based reconstruction and node-wise teacher weighting were timed together with aggregation and global-model refinement. In the original-stream Cora trace, the agreement calculation and the construction of nearest-neighbor edges took 0.009 and 0.238 s per run, respectively. Client-side abstract-memory updates took 0.17 s on Cora and 0.11 s on CiteSeer in the shared-protocol runs.

Memory. In the original-stream Cora profile at B=40B=40, abstract slots occupied 64.22 KiB across the three clients. The logged client buffers totaled 4.78 MiB, including raw replay and retained model snapshots in that configuration. Reducing BB to 10 cut slot storage to 16.05 KiB with similar mean accuracy (see Question 3). In the shared-protocol runs, MemEvo-FCGL's peak process RSS was higher than FedAvg's, close to POWER's, and lower than MOTION's on both datasets.

Thank you for raising this point. The revised appendix includes the full resource comparison and explains how each cost was measured.

3. Hyperparameter sensitivity and memory budget

Hyperparameter selection. We selected the settings reported in Table 4 and Appendix B after comparing candidate configurations on validation nodes, then fixed each dataset's settings across test seeds. The slot cap BB, number of structural keys PP, and retrieval depth KhK_h govern how much history the model can represent or access within its storage and computation budget. The retrieval temperature τh\tau_h and injection scale σ\sigma control the concentration of attention and the strength of the historical features, respectively. The aggregation coefficient λa\lambda_a balances sample counts and structural agreement, and the encoder stability weight balances adaptation to the current task against drift from the previous encoder. The key-similarity threshold δkey\delta_{\mathrm{key}} controls how closely two slots must match before they can merge.

Sensitivity experiments. On Cora, we varied one parameter at a time with the same five seeds per setting. The table reports ranges of changes in mean final average accuracy (AM) and nonnegative forgetting (FM) relative to each sweep's reference, in percentage points (pp). We used the original Cora stream with its archived client-local replay configuration and a separate class-revisit stream for the merge-threshold sweep.

RoleParameterReferenceTested valuesΔAM (pp)ΔFM (pp)
Memory budget and accessSlot cap BB4010, 20, 30, 40, 60, 80[−0.18, +0.30][−0.29, +0.17]
Structural keys PP205, 10, 20, 30, 40[−0.33, +2.09][−3.70, +1.23]
Retrieval depth KhK_h31, 2, 3, 5, 10[−1.46, +0.50][−1.70, 0.00]
Historical feature fusionRetrieval temperature τh\tau_h0.050.0125, 0.025, 0.05, 0.1, 0.2[−3.34, +0.52][−2.00, +0.23]
Memory injection scale σ\sigma0.50.125, 0.25, 0.5, 0.75, 1[−3.70, +1.09][−1.74, +2.25]
Server coordinationAggregation coefficient λa\lambda_a0.50, 0.25, 0.5, 0.75, 1[−0.46, +1.27][−2.33, 0.00]
Continual optimizationEncoder stability weight0.50.125, 0.25, 0.5, 0.75, 1[−0.22, +0.91][−0.96, 0.00]
Slot consolidationMerge threshold δkey\delta_{\mathrm{key}}0.95−1, 0, 0.25, 0.5, 0.75, 0.95, 0.98[−0.13, 0.00][−0.12, +0.04]

Reducing BB from 40 to 10 cut abstract-slot storage across the three clients from 64.22 to 16.05 KiB, with mean AM of 79.65% and 79.96%, respectively. Retrieving a single slot yielded 79.59% AM, and using five structural keys yielded 81.74% AM. The most pronounced declines occurred with very sharp attention (τh=0.0125\tau_h=0.0125, −3.34 pp AM) and the largest tested injection scale (σ=1\sigma=1, −3.70 pp AM and +2.25 pp FM). On the class-revisit stream, lowering δkey\delta_{\mathrm{key}} from 0.95 to −1 increased the mean merge count from 0 to 60 and changed AM by −0.11 pp.

Thank you for highlighting the importance of reporting the basis for these choices and their sensitivity. We have included the sensitivity results and experimental settings in the revised appendix.

Dear reviewer AVKS. Thank you for your time and effort to review this article. Your comments are valuable in improving the quality of this article. Since the author-reviewer discussion is coming to an end, we gently request that you read the rebuttal and whether if your concerns are addressed. Thanks again for your time and help in improving the quality of this article. Regards from the authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.