acceptodds
Under review as a conference paper at ICLR 2027

HETEROGENEOUS FEDERATED GRAPH LEARNING VIA OPTIMIZER SECOND-MOMENT COORDINATION

Abstract

Federated graph learning (FGL) has emerged as a promising approach to train graph neural networks on distributed graph data. However, existing methods often cause personalized learning degradation and convergence failure due to mismatch fixed second moments of gradients on different clients with heterogeneous graph dataset. To address this issue, we propose a novel federated graph learning framework, namely Fedseed, which leverage the second moment of Adam optimizer as statistical metric of multi-hop graphs feature. On the client side, Fedseed designs an adaptive gradient update mechanism to adapt local heterogeneity, which dynamically balance the local and global second moment on each client per round. On the server side, Fedseed proposes a similarity aggregation strategy to mitigate model drift by heterogeneity, which aggregates different local second moments by clustering. Extensive experiments demonstrate that Fedseed significantly outperforms state-of-the-art methods on accuracy over real-world graph datasets with statistical and topological heterogeneity settings.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Federated Graph Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QCeaOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses topological and statistical heterogeneity in Federated Graph Learning (FGL) by proposing FedSeed, a framework that coordinates the second moment of local Adam optimizers. The authors observe that under a decoupled GNN architecture with a linear readout, the hop-block energy share of the local second moment approximates the client's multi-hop feature energy profile. Based on this, FedSeed introduces: (1) a compressed communication channel transmitting block-level second-moment scalars; (2) a hyperparameter-free, closed-form per-parameter client mixing coefficient βk\beta_k^{} derived by minimizing an update-space risk trading off step distortion against gradient noise amplification; and (3) a server-side similarity aggregation strategy using a Gaussian kernel over client hop energy profiles. The method is evaluated on seven datasets using GCN and GAMLP backbones across different client scales and Dirichlet non-IID settings, comparing against generic federated learning baselines.

Critical strengths
  1. FedSeed treats Adam’s second moment as a signal for both personalization and client coordination. This is a concrete and potentially novel design direction for federated graph learning.
  2. By formulating second-moment mixing in update space (u=v(β)−1/2u = v(\beta)^{-1/2}) as a trade-off between step distortion and gradient noise floor, the paper derives an exact closed-form coefficient βk\beta_k^{} that requires no tuning and provably prevents step-size explosion by ensuring vk(β)≥vkv_k(\beta^{}) \ge v_k.
Critical weaknesses
  1. The empirical validation does not include the closest relevant baselines. The experiments omit FedAdamW and several graph-federated methods discussed in the related-work section, including FedGTA, FedTAD, FedPub, and GraphP-FL. Therefore, the reported gains only establish superiority over the selected baselines, not over the most relevant alternatives.
  2. Methodological Disconnect for Standard GNNs and Missing Reproducibility Details: The theoretical derivation of hop energy profiles ek(h)e_k(h) (Eq. 4) and hop-block compression (Section 3.3, Appendix A) explicitly requires a decoupled GNN where readout weights Wc,hW_{c,h} are indexed by propagation hop hh. However, half of the experiments (Table 1, Figure 2, Figure 4) use a standard GCN, where layers are coupled with non-linearities and no hop-indexed readout blocks exist. The paper fails to explain how hop profiles or block compression/decompression operate on GCN. Combined with unspecified formulas for profile dispersion σw\sigma_w and bandwidth hh (Algorithm 1), contradictory mixing schedules, and unverified partial-participation claims, the method is neither fully sound across backbones nor reproducible.
Questions for authors
  1. Why are FedAdamW and the graph-federated baselines discussed in Section 2 absent from the experiments? Under matched architectures, partitions, hyperparameter budgets, and communication budgets, does FedSeed still outperform them?
  2. How are "hop blocks," the hop energy profile pkp_k (Eq. 4), and the compression/decompression mapping between the 10 scalars and full dd-dimensional parameter vectors defined and implemented when the backbone is a standard multi-layer GCN rather than a decoupled GNN?
  3. Is evaluation performed using the global model, client-local models, or personalized models? Please specify the data splits, model-selection procedure, and the construction of the AGNews graph.
2 comments

Dear Reviewer QCea,

We sincerely appreciate the time and effort you have invested in reviewing our paper. We hope that our responses below will effectively address your concerns.

Q1: Why are FedAdamW and the graph baselines absent, and does our framework still win under matched conditions?

Re1: The earlier harness covered only generic FL methods; the current version runs all four, each following its reference implementation: FedAdamW (official: v reset to the uploaded mean, m zeroed, step pinned to the server counter, −γ·lr/K·m̄), FedGTA (PVLDB'24, gate α=0.65), FedTAD (IJCAI'24), FED-PUB (ICML'23, MaskedGCN + functional similarity), the latter two via OpenFGL. GraphP-FL lacks a public implementation, so we omit it rather than use an unvalidated re-implementation. Accuracy (GCN, K=10, Metis, 200 rounds, 3 seeds, val-tuned):

MethodCoraCiteseer
FedAvg78.54±1.8771.30±1.16
FedProx (µ=0.5)86.29±1.2573.19±0.65
FedAdamW84.74±1.4671.02±0.16
FedGTA55.46±10.9670.62±1.66
FedTAD78.54±1.9571.27±1.25
FED-PUB79.27±1.5871.40±1.11
Our framework84.46±1.0271.35±1.37

Personalized (node-weighted): ours 84.03±0.80 on Cora, FedAdamW 82.42±0.85, FedGTA 80.95±0.07, FED-PUB 79.30±1.59. Ours beats FedAvg, FedAdam, SCAFFOLD, FedTAD, FED-PUB, ties FedAdamW; tuned FedProx strongest. Compressed budget: 720.6 vs 720.5 KB/client/round, 0.4 KB server state.

Q2: How are hop blocks, the profile p_k, and the compression defined for a multi-layer GCN?

Re2: Two backbones. SIGN/GAMLP: the readout acts on [z⁰;…;zᴷ], so its columns split into K+1 blocks, p_k(h) being the hop-h energy share,

pₖ(h) = ‖AₖʰXₖ‖²_F / Σₕ′‖Aₖʰ′Xₖ‖²_F,

with no n_k. 2-layer GCN: with no hop-indexed readout, blocks follow receptive field — lin1 on 1-hop, lin2 on 2-hop inputs; {lin1.weight, lin1.bias}=block 0, {lin2.weight, lin2.bias}=block 1 (K=#layers−1), profile = shares of ‖X‖², ‖AX‖². A layer-equivalent, not W_{c,h}; v-side fidelity is 0.970–0.9998 on SIGN (exact blocks), ≈0.59 on GCN. The channel sends one scalar per hop slice (mean v) plus one per other tensor: (K+1)+#tensors — 12 for SIGN/Cora, 2+#tensors for GCN ("≈10 floats" = decoupled). Compression costs ~0.3 points on GCN (84.46±1.02→84.16±1.23), none on SIGN (83.67±0.15→84.16±1.23). Algorithm 1 (mix at round start) is implemented; Sec. 3.4 is aligned. Bandwidth: w_kj ∝ exp(−‖p_k−p_j‖²/2h²), h=σ_w (0.011 near-IID to 0.077 at α=0.1), λ=0.9, no tuning.

Q3: Which model is evaluated, and what are the splits, selection, and AGNews construction?

Re3: The global model is evaluated each round on the full graph with union masks; personalized methods additionally use node-weighted local test accuracy. Each subgraph splits 20/40/40 into train/val/test (per seed). Numbers are test accuracy at the validation-best round; baseline hyper-parameters (η_g∈{0.003,0.01,0.1}, µ∈{0.01,…,1.0}) are tuned on validation only; ours has none. AGNews graph (released code):

ItemSetting
Documents12,000, class-balanced from AG-News train CSV
FeaturesTF-IDF, 500 dims, stop-words, sublinear tf
Edgescosine kNN, k=10, symmetrized, deduplicated
Partitionsame protocol as all datasets, cached per seed

Two fixes: the current version broadcasts to all clients (the submitted one updated only active ones), anchors refresh for every known client (Corollary 2); ours stays at 84.5 on Cora at 50% and 25%. Scope: Cora/Citeseer (GCN), Cora/PubMed (SIGN).

Thank you to the authors for the detailed response and for providing additional clarifications on the GCN layer-wise block mapping, bandwidth calibration, AGNews graph construction, and supplementary baseline runs on Cora and CiteSeer.

I have carefully reviewed the rebuttal. While these clarifications improve transparency regarding the implementation details, my main concerns remain unresolved:

1.Empirical Validation & Baseline Comparisons: The newly added FGL baselines (FedGTA, FedTAD, FED-PUB) and FedAdamW are only evaluated on two small-scale datasets (Cora and CiteSeer, GCN, K=10K=10), where the proposed framework still underperforms tuned FedProx on both datasets (84.46 vs. 86.29 on Cora; 71.35 vs. 73.19 on CiteSeer) and ties with FedAdamW (84.46 vs. 84.74). Moreover, the severe data anomalies, truncated entries, and cross-backbone identical baseline values in the original main tables (Tables 1 and 2) were not directly addressed.

2.Methodological Fit on Standard GNNs: The authors' reported fidelity metric for GCN (≈0.59\approx 0.59, compared to 0.970–0.99980.970\text{--}0.9998 for decoupled SIGN) confirms that the hop-block factorization assumption does not hold well for standard message-passing GNNs.

Given the extent of the required revisions and re-evaluations across all datasets and backbones, I will maintain my current rating. I encourage the authors to incorporate these clarifications and thoroughly overhaul the experimental tables in a future revision.

Reviewer_cASGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FedSeed addresses heterogeneous federated graph learning by coordinating the second moments of local Adam optimizers. Each client mixes its local second moment with a server estimate using a coefficient computed for each parameter. The server aggregates compressed second moments according to the similarity of client hop profiles. The paper reports accuracy gains across several datasets, two backbones, and different client counts, supported by an ablation of the two main components.

Critical strengths
  1. A compact approach to optimizer coordination. The method reuses optimizer statistics and computes the mixing coefficients in closed form. It offers a practical way to control how much each client borrows from others without adding an auxiliary trainable model.
  2. Consistent gains in the reported comparisons. FedSeed has the highest mean accuracy among the evaluated federated baselines in all 18 configurations in Tables 1 and 2. Results on both GCN and GAMLP, with three seeds, provide useful evidence that the gains extend beyond a single setting.
Critical weaknesses
  1. The advance is modest and the choice of method is not sufficiently justified. FedAdamW already shares and compresses second moments, while topology-aware client aggregation is established in methods such as FedGTA. The contribution therefore rests on the proposed adaptive mixing and its connection to graph structure. The paper does not establish why second-moment mismatch is a major cause of failure here, or why similarity in these statistics identifies clients that benefit from sharing. Second moments also reflect labels, feature scales, and training history. The analysis relies on residual-feature assumptions whose relevance during training is not demonstrated, and the hop-based construction is not clearly specified for GCN. There are also gaps in the theoretical support. The factorization in Eq. (19) drops cross-node terms without a condition that makes them vanish. The distortion bound in Eq. (5) fails under its stated assumptions, as shown in Question 3. Since the compression argument relies on this bound, these issues affect the justification for the method.
  2. The experiments do not establish the value of the new components. The comparisons omit FedAdamW and relevant graph-specific methods such as FedGTA and FED-PUB. This leaves the advantage over the closest alternatives unresolved. The ablation is limited to one dataset and does not report the fixed mixing coefficient. Its claim of synergy also lacks the matching variant with both fixed mixing and uniform second-moment aggregation; FedAvg is not an equivalent substitute. More fundamentally, the mixing rule only maintains or increases the local second moment, thereby reducing update magnitudes. The experiments do not separate the benefit of topology-based sharing from that of more conservative local updates. The claims about practical applicability are also broader than the evaluation: partial participation, client-level personalization gains, and communication or runtime costs are not measured. The reported accuracy gains are encouraging, but the need for the proposed mechanism remains unproven.
Questions for authors

How does FedSeed compare with FedAdamW and graph-specific methods such as FedGTA under matched data splits, tuning budgets, and evaluation protocols? These comparisons would affect my assessment of the claimed advance. How are the hop blocks defined for GCN? What evidence distinguishes the benefit of topology-based sharing from that of simply reducing local update magnitudes? In Eq. (5), set the reference second moment to 1 and the mixed second moment to 0.5 for one coordinate with the same nonzero update numerator. Using the proof’s omission of epsilon, the relative distortion is about 0.414, while the claimed upper bound is 0.25. The stated condition is satisfied. How should this bound be corrected, and what changes are needed in the compression guarantee?

1 comment

Dear Reviewer cASG,

We sincerely appreciate the time and effort you have invested in reviewing our paper. We hope that our responses below will effectively address your concerns.

Q1: How does our framework compare with FedAdamW and graph methods under matched splits and budgets?

Re1: All baselines share one code path (FedAdamW official; FedGTA, FedTAD, FED-PUB OpenFGL); knobs tuned on validation, ours none. Table 1: global-graph accuracy (GCN, K=10, Metis, 200 rounds, 3 seeds).

MethodCoraCiteseer
FedAvg78.54±1.8771.30±1.16
FedProx (µ=0.5)86.29±1.2573.19±0.65
FedAdam77.72±2.4971.94±0.89
FedAdamW84.74±1.4671.02±0.16
SCAFFOLD78.91±2.0771.30±1.16
FedGTA55.46±10.9670.62±1.66
FedTAD78.54±1.9571.27±1.25
FED-PUB79.27±1.5871.40±1.11
Our framework84.46±1.0271.35±1.37

Ours is the strongest zero-tuned method: it ties FedAdamW on Cora, leads on personalized accuracy (84.03 vs 82.42); tuned FedProx ahead. Full ablation (fixed β, fixed+uniform, two datasets × two partitions):

Fixed βElasticUniform
085.47±1.1385.65±1.40
0.2585.80±1.4385.68±1.35
0.585.50±1.1485.78±1.23
0.785.38±1.2285.59±1.02
0.985.32±1.2985.20±1.13
Adaptive (ours)84.46±1.0284.56±1.05
FedAvg (β=1)78.54±1.8778.73±1.06

Any fixed β in [0,0.9] captures most of the gain; elastic vs uniform is within noise; the rule is tuning-free and keeps κ≥1 (Re4: c=1/2).

Q2: How are hop blocks and the compression channel defined for GCN?

Re2: SIGN/GAMLP: the readout acts on [z⁰;…;zᴷ], so its columns split into K+1 hop blocks. 2-layer GCN: blocks follow receptive field — lin1 on 1-hop, lin2 on 2-hop inputs; {lin1.weight, lin1.bias}=block 0, {lin2.weight, lin2.bias}=block 1; profile = shares of ‖X‖², ‖AX‖². The channel sends one scalar per hop slice (mean v) plus one per other tensor; v-side fidelity is 0.970–0.9998 on SIGN, ≈0.59 on GCN. The profile uses only (A_k, X_k), hence it is label- and history-independent.

Q3: What distinguishes topology sharing from smaller local updates?

Re3: Sharing v helps; only FedAdam's server moment underperforms (77.72 vs 78.54/84.74/84.46). Even β=0 is among the strongest: the gain comes from the channel. Ours inflates v by a measured 1.013, the control scaling v by γ without sharing:

SettingCoraCiteseer
γ=1.1, no sharing79.00±2.1271.25±1.12
γ=2.0, no sharing72.668.9

γ=1.1 (8× inflation) gives FedAvg level (78.54/71.30) exactly; γ=2.0 degrades, so conservative updates alone bring no gain: the benefit is which coordinates are raised (σ²/v noise floor), not magnitude.

Q4: How should Eq. (5) be corrected, and what changes in the compression guarantee?

Re4: Lemma 1 under min(v_s,v*)≥v*/2 does not cover your counterexample; the sharp statement is an exact identity (√2−1≈0.414 at v_s=0.5, v*=1; sharp constant 2(√2−1)):

D₀ = |vₛ − v∗| / v∗ · 1 / (√r (1 + √r)), r = vₛ / v∗,

D ≤ c(κ) · E[ |vₛ − v∗| / v∗ ], c(κ) = 1 / (√κ (1 + √κ)), κ = inf vₛ / v∗.

At r=1/2 it gives 2(√2−1)·½=√2−1; zero violations on 200,000 random coordinates. The compression guarantee changes only in the constant, D(v̂)≤c(κ)(κ_Z−1)/(κ_Z+1); the path is unchanged. Ours never lowers a coordinate below its own v, so κ≥1 and c=1/2 exactly, κ<1 being the avoided region. Eq. (19) now states its condition: cross-sample residual uncorrelatedness under mini-batch sampling.

Reviewer_h9nvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FedSeed, a federated graph learning framework that leverages the second moment of the Adam optimizer to address graph heterogeneity across clients. On the client side, it introduces an adaptive per-parameter mixing mechanism to balance local and server-side second moments. On the server side, it uses multi-hop graph energy profiles to estimate client similarity and aggregate optimizer statistics accordingly. The paper further provides theoretical analyses of moment estimation, compression, and adaptive mixing. Experiments on multiple graph datasets demonstrate improvements over the evaluated baselines.

Critical strengths

The motivation is clear and addresses an important challenge in federated graph learning. Using optimizer second moments as structural signals provides an interesting alternative to conventional aggregation methods based on model updates or handcrafted descriptors. The framework also has a relatively coherent design, combining client-side adaptation with server-side similarity-aware coordination.

The theoretical analysis follows the main components of the proposed method. The authors investigate the relationship between second moments and multi-hop graph features, the effects of compression, and the adaptive mixing mechanism. The ablation study also provides preliminary evidence that the two main components contribute to the overall performance.

The experiments cover multiple graph datasets, different numbers of clients, and two GNN backbones. FedSeed reports consistent improvements over the evaluated baselines, with a maximum reported accuracy improvement of 8.0%. The compressed second-moment representation is also potentially useful for reducing the communication overhead of federated optimization.

Critical weaknesses

see the questions for authors.

Questions for authors

Can the authors clarify the assumptions connecting optimizer second moments to graph structure? In particular, Equations 5 and 10 appear inconsistent with their subsequent derivations. Stronger justification for the reliability of hop energy profiles under statistical heterogeneity would improve the theoretical soundness.

Can the authors compare FedSeed with graph-specific baselines such as FedGTA and FedTAD under matched datasets, client partitions, and GNN backbones? It would also be useful to isolate the contributions of the client-side mixing mechanism and server-side similarity aggregation across more settings.

Can the authors provide an end-to-end comparison of communication costs, runtime, and memory consumption? Additionally, how does FedSeed perform under partial client participation, and to what extent does its client-specific optimizer state translate into genuine personalization at inference time?

1 comment

Dear Reviewer h9nv,

We sincerely appreciate the time and effort you have invested in reviewing our paper. We hope that our responses below will effectively address your concerns.

Q1: Which assumptions connect second moments to graph structure, how should Eq. (5) and Eq. (10) be read, and how reliable are hop profiles under heterogeneity?

Re1: The earlier derivation used Lemma 1 under v_s ≥ v*/2, which does not cover the reviewed regime; the sharp statement is an exact identity:

D₀ = |vₛ − v∗| / v∗ · 1 / (√r (1 + √r)), r = vₛ / v∗,

D ≤ c(κ) · E[ |vₛ − v∗| / v∗ ], c(κ) = 1 / (√κ (1 + √κ)), κ = inf vₛ / v∗.

Ours never lowers a coordinate below its own v (Corollary 1): κ≥1 and c=1/2; κ→0 diverges. Verified on 200,000 random coordinates (zero violations); Eq. (8) follows. In Eq. (10) the two v_k differ (population moment, Prop. 3; optimizer state, Eq. (9)); the coefficient solves only the update-space risk (Prop. 4). Eq. (19) states its condition: cross-sample residual uncorrelatedness under mini-batch sampling.

Hop-profile reliability during training (cosine of live v's hop shares with ‖A^h X‖²_F), on SIGN:

Settingα=0.1α=10MetisPubMed
cos0.978±0.0120.9998±0.00010.970±0.0080.973±0.004

It holds under skew; on GCN it is weaker (cos≈0.59). Measured σ_w: 0.077 at α=0.1 vs 0.011 at α=10, so the kernel concentrates under heterogeneity.

Q2: How does our framework compare with FedGTA and FedTAD, and can the two components be isolated more?

Re2: FedAdamW (official), FedGTA, FedTAD, FED-PUB (OpenFGL) share one code path; knobs tuned on validation, ours none. Global-graph accuracy (GCN, K=10, Metis, 200 rounds, 3 seeds):

MethodCoraCiteseer
Global86.78±1.3672.97±1.28
FedAvg78.54±1.8771.30±1.16
FedProx (µ=0.5)86.29±1.2573.19±0.65
FedAdam77.72±2.4971.94±0.89
FedAdamW84.74±1.4671.02±0.16
SCAFFOLD78.91±2.0771.30±1.16
FedGTA55.46±10.9670.62±1.66
FedTAD78.54±1.9571.27±1.25
FED-PUB79.27±1.5871.40±1.11
Our framework84.46±1.0271.35±1.37

Ours is the strongest zero-tuned method: ties FedAdamW on Cora, leads on personalized accuracy (84.03 vs 82.42); tuned FedProx ahead. The fixed-β × {elastic, uniform} grid shows elastic reduces to uniform within noise (Cora, α=0.1: 84.69±0.76 vs 84.48±0.51).

Q3: What are the end-to-end costs, and how does our framework behave under partial participation?

Re3: Costs per client over complete Cora rounds, peak GPU memory logged:

MethodUp/Down KBServer states/roundPeak GPU MB
FedAvg720.5—0.1868.6
FedAdamW1441.11441.1 KB0.2083.4
FedTAD720.510 client models2.0691.9
FED-PUB1570.1/7850.3K client models0.09140.7
Ours (full v)1441.17205.3 KB0.3592.8
Ours (compressed)720.60.4 KB0.3692.8

Compressed traffic matches FedAvg at ~0.3 points on GCN (84.46→84.16), zero on SIGN (83.67→84.16). The current version broadcasts to all clients; the submitted one only active ones (Corollary 2):

RatioFedAvgFedAdamWOurs
0.579.12±1.9284.74±1.1584.46±0.86
0.2578.05±1.5685.20±0.5784.53±1.28

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.