acceptodds
Under review as a conference paper at ICLR 2027

FedSuCo: Federated Graph Clustering via Riemannian Geometry-Guided Supernode Collaboration

Abstract

Federated graph clustering requires learning across fragmented graph structures while keeping individual-node information local. We propose FedSuCo, which uses supernodes guided by Riemannian geometry as the unit of collaboration. Specifically, in FedSuCo, clients learn Riemannian node representations and construct supernodes through geometric renormalization, preserving coarse structural organization while retaining clustering-relevant angular similarity. Supernode representations and securely aggregated connectivity statistics jointly form a weighted global quotient graph, on which the server learns structural guidance and transfers it back to local nodes. Final federated spherical clustering aggregates only cluster-level sufficient statistics, while node representations and assignments remain local. Experiments on five real-world graph datasets against thirteen federated baselines show consistent improvements across clustering metrics.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Federated Graph Clustering, Privacy-Preserving Graph Learning, Graph Coarsening, Secure Multi-Party Computation, Hyperspherical Representation Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ATb9OpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes FedSuCo to address the lack of global structural information in federated graph clustering. Each client first learns node representations in a Riemannian space and constructs supernodes through geometric renormalization. The framework then uses secure aggregation to collect cross-client supernode connectivity statistics and build a global weighted hypergraph. The resulting structural information is sent back to clients to support local clustering. Finally, clustering is performed using federated spherical K-means or a von Mises–Fisher mixture model. Experiments on five real-world graph datasets show that FedSuCo outperforms thirteen federated baselines in ACC, NMI, ARI, and Macro-F1.

Critical strengths
  1. This paper addresses the lack of cross-client structural information in federated graph clustering. Unlike existing methods that rely mainly on model parameter aggregation, it uses supernodes to capture cross-client structural relationships. This is a meaningful approach.

  2. The proposed method achieves good clustering performance on five graph datasets. The experiments also examine the effects of client number, supernode scale, and structural feedback. The paper further discusses the trade-off between communication overhead and privacy protection.

Critical weaknesses
  1. Riemannian geometry-guided supernode construction is a main contribution of this paper. However, the experiments only study the effect of supernode scale. It remains unclear whether Riemannian geometry itself brings performance gains. The authors should compare it with random grouping, Euclidean clustering, and embedding similarity-based construction under the same number of supernodes and training settings.

  2. FedPAM and FedNCN are discussed in the related work but are missing from the experiments. The current baselines do not sufficiently cover federated graph clustering methods. In addition, FedSuCo uses cross-client connectivity information, while most baselines do not. This makes it difficult to determine whether the gains come from the proposed method or the additional structural information. The authors should include FedPAM and FedNCN and compare methods under the same cross-client structural information.

Questions for authors
  1. How much performance gain does the proposed Riemannian geometry-guided construction strategy provide over random grouping, Euclidean clustering, and node-embedding-based supernode construction?

  2. Why were relevant federated graph clustering methods, such as FedPAM and FedNCN, not included in the comparison? Does FedSuCo still outperform existing methods when cross-client structural information is kept consistent?

  3. Why does CiteSeer perform better in Table 3 when global structural feedback is removed? Could the feedback strength be adjusted according to structural differences across clients?

  4. What are the total communication costs and end-to-end execution time of the secure computation protocols in a real multi-server setting? How do these results compare with those from trusted simulations?

  5. The privacy mechanism relies on strict assumptions. However, the paper provides little empirical evidence of its actual privacy protection. The risk of information leakage needs further investigation.

  6. The two-server protocol assumes that the servers do not collude and that at most one server is passively compromised. However, the protection is limited by the public information and authorized outputs. Servers can still access supernode representations, scale information, and aggregated connection statistics. These statistics may still expose sensitive information. In particular, attribute inference and membership inference attacks remain possible, even without direct access to individual node representations.

  7. The construction of cross-client edge statistics assumes that both endpoint clients hold the corresponding connection records. This assumption may not hold in all federated learning settings and could limit practical applicability.

  8. The authors should further evaluate privacy risks through attribute inference and membership inference attacks based on publicly available supernode statistics.

9 comments

Dear Reviewer ATb9,

We sincerely thank you for your careful review and insightful suggestions.

W1 & Q1: The contribution of Riemannian geometry has not been sufficiently validated against alternative supernode construction strategies.

We thank the reviewer for the suggestion. Following your recommendation, we have added comparisons with random grouping, Euclidean-distance grouping, and grouping based on the cosine similarity of learned embeddings.

Comparison settings. To evaluate FedSuCo’s overall geometric design, we construct three Euclidean baselines by replacing its coupled geometric representation learning, supernode assignment, aggregation, and feedback components with Euclidean counterparts, while retaining supernode compression and global structural collaboration. The three controls use the same encoder architecture and share the same pretrained checkpoint before grouping for each dataset and seed. They differ only in their supernode grouping strategies:

ControlGrouping strategy
RandomRandomly permute nodes within each client and divide them according to the target capacity vector.
DistancePerform capacity-constrained grouping using squared Euclidean distances between learned Euclidean node embeddings.
CosineUse the same capacity-constrained grouping procedure with cosine similarity between learned embeddings; normalization is used only to compute grouping similarities.

The three Euclidean variants reconstruct groups at the beginning of each outer iteration and use the within-group mean of unnormalized Euclidean node embeddings as the supernode representation. They replace hidden geometric statistics with observed coarse-graph degrees and use corresponding Euclidean server learning and feedback operations. Each variant reconstructs its coarse graph from the same original edge records using its own assignment matrix, preserving actual cross-client connectivity.

Results. Experiments cover Cora, CiteSeer, Amazon Photo, and Amazon Computers.

Final clustering uses K-means on original-node representations: Full uses unit-normalized node embeddings, while the Euclidean baselines use unnormalized Euclidean vectors. The table reports mean±sample standard deviation of ACC (%) across five training seeds.

DatasetFull FedSuCoRandomDistanceCosine
Cora73.73±1.2447.48±5.4041.88±3.7943.93±4.68
CiteSeer59.99±2.4737.48±8.8336.78±8.6237.02±8.75
Amazon Photo65.41±6.8237.97±2.5839.04±3.2937.64±5.47
Amazon Computers43.92±4.2828.96±2.4925.49±5.4128.61±1.82

Compared with the strongest Euclidean baselines on each dataset, FedSuCo improves ACC by 26.25, 22.51, 26.37, and 14.96 percentage points, respectively. The advantage is consistent across all five paired training seeds. FedSuCo also achieves higher NMI, ARI, and Macro-F1 than all three Euclidean baslines. Results indicate the effectiveness of its integrated geometric design.

W2 & Q2: Comparisons with directly relevant baselines are insufficient, and the contributions of cross-client structural information and model design remain unclear.

We thank the reviewer for the insightful suggestions to include directly relevant baselines such as FedPAM [1] and FedNCN [2], and to evaluate our method with cross-client structural information held constant. These insights help us distinguish the contributions of additional structural information and model design. Following your suggestions, we have added comparisons with both methods and controlled experiments assessing the role of true cross-client connectivity and the gains from server learning given the same connectivity information.

Comparison with FedPAM/FedNCN. During the original experiments, we had not obtained public implementations of these methods or received replies to our inquiries to the authors. We have now reimplemented both methods based on their papers and evaluated them on five datasets with 10 clients and five training seeds, using the same partition files as FedSuCo.

The table reports ACC and NMI (%) as mean±sample standard deviation. FedPAM and FedNCN results are from our reimplementations; FedSuCo uses vMF clustering throughout.

DatasetFedNCN ACCFedNCN NMIFedPAM ACCFedPAM NMIFedSuCo-vMF ACCFedSuCo-vMF NMI
Cora24.74±2.178.35±3.5829.74±0.112.22±0.2374.13±1.8956.20±1.64
CiteSeer29.13±2.668.52±2.7221.18±0.170.70±0.2558.77±2.5932.45±1.64
PubMed41.96±3.533.70±2.2854.05±6.6816.85±4.9164.69±1.3423.68±1.27
Amazon Photo21.73±2.196.06±1.9125.36±0.020.16±0.0064.61±8.7058.71±6.80
Amazon Computers19.43±2.576.02±1.6237.49±0.010.12±0.0140.19±4.3241.32±2.57

Under these settings, FedSuCo-vMF achieves higher mean ACC and NMI than both reimplementations on all five datasets.

Comparison of structural information and server design. To distinguish the contributions of additional structural information and model design, we compare three variants:

  • Full: Uses true cross-client edges to construct a global weighted supernode graph and trains a weighted graph encoder through link reconstruction and directional constraints to learn supernode feedback directions.
  • No-cross: Retains Full’s server learning and feedback procedures but uses only intra-client edges to construct the coarse graph. True cross-client edges are never accessed, and unknown cross-client node pairs are masked out of the structural loss.
  • Simple-global: Uses the same true cross-client edge information as Full but generates feedback through a fixed rule: compute a weighted average of neighboring and self directions using the current superedge weights, add it to the original direction, and normalize. No graph encoder is trained to generate feedback directions.

All three variants start from the same local pretrained models and initial supernode groups and use the same rules for assignment updates, local training, client feedback, and final clustering. Simple-global’s graph and directions still evolve during training; only its server computation rule is fixed. Thus, Full versus No-cross evaluates the contribution of cross-client edge information, while Full versus Simple-global compares server feedback designs given the same cross-client edge information.

Experiments use 10 clients, five training seeds, and vMF clustering throughout, which is shown in the following table.

DatasetMetricFullNo-crossSimple-global
CoraACC72.35±5.4942.14±3.3647.06±4.18
CoraNMI55.94±1.9326.56±3.6328.07±2.51
CiteSeerACC58.77±2.5952.34±4.9957.05±2.99
CiteSeerNMI32.45±1.6427.07±3.1830.84±1.51
PubMedACC64.69±1.3456.51±2.88—
PubMedNMI23.68±1.2717.46±1.77—
Amazon PhotoACC64.61±8.7060.64±9.65—
Amazon PhotoNMI58.71±6.8053.91±6.23—
Amazon ComputersACC40.19±4.3236.37±5.92—
Amazon ComputersNMI41.32±2.5734.31±6.29—

Full achieves higher mean ACC and NMI than No-cross on all five datasets, supporting the contribution of true cross-client connectivity. Furthermore, with the same cross-client edge information, Full outperforms Simple-global on Cora and CiteSeer by 25.29 and 1.72 percentage points in ACC, and 27.87 and 1.61 percentage points in NMI, respectively. These results support the contribution of the trainable server feedback mechanism beyond access to additional structural information on these two datasets.

[1] Fu et al. Unsupervised Federated Graph Learning. NeurIPS 2025.

[2] Liu et al. Federated Node-Level Clustering Network with Cross-Subgraph Link Mending. ICML 2025.

Reviewer_ATb9OpenReview ↗

Thanks!

Thank you for your detailed responses and additional experiments. I appreciate the authors' efforts to address my concerns, particularly the inclusion of alternative supernode construction strategies, additional baselines, and controlled experiments on cross-client structural information.

These results help strengthen the empirical evaluation. However, I still have some concerns. First, the comparisons in W1 modify multiple components simultaneously, making it difficult to isolate the specific contribution of Riemannian geometry-guided supernode construction. Second, the relatively low performance of the reimplemented FedPAM and FedNCN baselines would benefit from further validation. Additionally, the comparison with Simple-global is limited to two datasets.

More importantly, my remaining questions regarding practical communication costs, privacy risks, and the assumptions underlying cross-client connectivity have not yet been addressed in the provided responses.

I appreciate the substantial additional effort and look forward to further clarification before reconsidering my rating.

Reviewer_m3xzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes FedSuCo, a geometry-guided framework for federated graph clustering. Clients learn node representations in a shared Riemannian representation space and organize nodes into supernodes according to geometric information. Node representations and node-to-supernode memberships remain local. Each client records cross-client edges incident to its nodes, and secure computation aggregates them into supernode-level connectivity statistics. The server learns structural information on a weighted quotient graph and sends feedback to node representations through the memberships retained locally by each client. Final clustering is performed by aggregating sufficient statistics, while node-level assignments are computed locally. Experiments cover Cora, CiteSeer, PubMed, Amazon Photo, and Amazon Computers, and compare FedSuCo with thirteen federated baselines. The paper also reports sensitivity to the number of clients, supernode capacity analyses, and secure-computation validation.

Critical strengths
  1. The paper uses supernodes as the basic units for cross-client structural collaboration. This allows the method to exploit coarse-grained graph structure without uploading node-level representations or membership assignments. The approach directly addresses the loss of cross-client structural information caused by graph partitioning.

  2. The paper clearly specifies a semi-honest threat model, the scope of authorized disclosures, and the secure aggregation procedure, and analyzes aggregation correctness and numerical precision. The experiments cover five datasets and include analyses of client-count and supernode-capacity sensitivity, as well as numerical consistency checks for secure computation.

Critical weaknesses
  1. FedSuCo-vMF encounters out-of-memory failures in several settings with few clients. Table 10 reports failures on PubMed with two and five clients and on Amazon Computers with two clients. The paper does not specify which stage causes the OOM failures or analyze the computational and memory complexity of the vMF variant. This limits the assessment of its scalability.

  2. Server feedback can cause substantial negative transfer. On CiteSeer, Table 3 reports an accuracy of 58.77 for the full method and 64.35 when server feedback is removed, an increase of 5.58 percentage points. This indicates that cross-client structural feedback is not beneficial on every dataset.

Questions for authors
  1. FedSuCo-vMF encounters OOM failures on PubMed with two and five clients and on Amazon Computers with two clients. Could the authors report its time and memory complexity and identify the stage at which the OOM failures occur?
2 comments

Dear Reviewer m3xz,

We sincerely thank you for your positive feedback and insightful suggestions.

W1 & Q1: Clarify the reported OOM failures and provide a stage-wise analysis of FedSuCo-vMF’s time and memory complexity.

We found that the “OOM” entries in Table 10 were incorrectly labeled and actually correspond to experiments that exceeded the preset runtime limit. We will provide these results in the updated PDF of our paper, which will be uploaded shortly.

To address the reviewer’s concerns about time and memory complexity, we provide a brief analysis of the two most time-consuming stages: geometric initialization and federated vMF clustering.

1. Geometric initialization.

For a connected component containing nn nodes, D-Mercator [1] is utilized to implement the geometric initialization. Specifically, it evaluates approximately Q(n)=100max⁡(1,ln⁡n)Q(n)=100\max(1,\ln n) candidate positions per node, computing the likelihoods of both connections and non-connections with other nodes. With TgT_g position-refinement sweeps and geometric coordinate dimension dgd_g, the time complexity of this refinement step is

=O\\!\left(T_gQ(n)n^2d_g\right) =O\\!\left(T_gd_gn^2\log n\right).$$ The **memory complexity** of the current geometric inference wrapper is $$M_{\mathrm{geo,wrapper}} =O\\!\left(n_c^2+n_cd_g+m_c\right),$$ where $n_c$ and $m_c$ denote the number of nodes and intra-client edges, respectively. Reference: [D-Mercator](https://arxiv.org/html/2304.06580v2). **2. Federated vMF clustering.** Let $N$ denote the total number of original nodes participating in clustering, $K$ the number of clusters, $d$ the representation dimension, $R$ the number of initializations, and $I$ the maximum number of EM iterations per initialization. The time complexity is $$T_{\mathrm{vMF}}=O(RINKd).$$ Each client primarily stores its node representations, responsibility matrix, and clustering parameters. The per-client memory complexity is $$M_{\mathrm{vMF},c} =O(n_cd+n_cK+Kd).$$ [1] Jankowski R, Allard A, Boguna M, et al. The D-Mercator method for the multidimensional hyperbolic embedding of real networks[J]. Nature Communications, 2023, 14(1): 7585. **W2: Server feedback can cause negative transfer, as removing it improves CiteSeer accuracy by 5.58 percentage points.** To address the concern that cross-client structural feedback is not beneficial on every dataset, we extended the comparison between the full model and its no-feedback variant to five datasets. Among the evaluated datasets, only CiteSeer performed better without feedback, as shown below. |Dataset|Setting|ACC|NMI|ARI|Macro-F1| |-|-|-|-|-|-| |Cora|Full|**72.35±5.49**|55.94±1.93|51.16±5.32|70.53±5.04| |Cora|No-feedback|39.44±4.12|21.92±6.94|13.84±4.25|32.77±7.53| |CiteSeer|Full|58.77±2.59|32.45±1.64|30.81±2.18|55.48±2.23| |CiteSeer|No-feedback|**64.35±2.34**|37.90±1.63|37.50±2.48|59.86±2.38| |PubMed|Full|**64.69±1.34**|23.68±1.27|23.40±1.90|64.84±1.24| |PubMed|No-feedback|57.22±3.84|16.93±3.36|14.84±4.05|57.43±4.07| |Amazon Photo|Full|**62.82±1.62**|50.21±1.96|41.16±2.96|57.50±1.70| |Amazon Photo|No-feedback|52.64±3.75|41.01±2.97|29.80±2.15|50.06±3.37| |Amazon Computers|Full|**36.58±1.26**|35.76±1.33|19.66±1.04|30.29±2.30| |Amazon Computers|No-feedback|31.73±3.83|29.63±4.77|13.98±2.95|25.55±4.48| Except for CiteSeer, incorporating global feedback improves ACC, NMI, ARI, and Macro-F1 on all four remaining datasets. Specifically, the relative ACC improvements over the no-feedback variant are **83.44%, 13.05%, 19.34%, and 15.29%** on Cora, PubMed, Amazon Photo, and Amazon Computers, respectively. We further analyzed a possible mechanism underlying the performance degradation on CiteSeer. We found pronounced topological differences between CiteSeer and the other datasets, which may help explain its different response to feedback. The structural statistics are reported below. |Dataset|Average degree|Largest connected component coverage|Cross-client edge fraction|Fraction of nodes incident to cross-client edges| |-|-|-|-|-| |Cora|4.08|100.00%|12.09%|28.33%| |CiteSeer|**2.74**|**63.72%**|**4.48%**|**8.12%**| |PubMed|4.50|100.00%|14.75%|26.56%| |Amazon Photo|31.13|97.87%|18.99%|57.27%| |Amazon Computers|35.76|97.30%|27.26%|67.18%| CiteSeer is sparser and more fragmented, with only **8.12%** of its nodes directly incident to cross-client edges. Given this limited coverage, direct structural interactions occur predominantly within local clients, while direct structural influence from other clients is relatively limited. This provides a possible explanation for why the current global feedback module does not improve performance on CiteSeer. These observations motivate exploring **adaptive feedback strength**, where topological statistics are mapped to feedback weights through a predefined function or a neural network to mitigate potential negative transfer. We leave this exploration to future work.

Thank you for your response. My concerns are mostly addressed, and I maintain my overall positive rating.

Reviewer_831oOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies unsupervised federated graph clustering, where an attributed graph is partitioned across clients and cross-client edges cannot be used directly in local message passing. The authors propose FedSuCo, which elevates supernodes, rather than individual nodes, to the unit of cross-client structural collaboration. Each client learns hyperbolic node embeddings, extracts unit spherical directions, and constructs supernodes through a geometric renormalization procedure driven by inferred spherical coordinates, hidden degrees, and a geometric parameter. Clients release only supernode-level statistics, namely directions, sizes, aggregated hidden degrees, and coarse edge counts, while cross-client superedge counts are computed with a two-server additive secret-sharing protocol. The server builds a weighted quotient supergraph, trains a weighted graph encoder with the binomial connectivity model of Eq. 4 that separates pair count, popularity, and directional similarity, and returns refined supernode directions that clients lift to nodes through their private membership matrices. Final clustering is a federated spherical k-means or von Mises-Fisher mixture that securely aggregates only cluster-level sufficient statistics. Experiments on five datasets against thirteen federated baselines report the best mean on all four metrics across all datasets, with scalability, capacity, ablation, communication, and small-scale MPC correctness analyses.

Critical strengths
  1. Motivation: The problem is well motivated and significant. Federated clustering over fragmented graphs is a real setting, as in interbank transfers and interhospital referrals, and the paper crisply articulates the core dilemma: prior structural collaboration remains tied to individual nodes or node-level proxies, conflicting with keeping node information local. The explicit system model, threat model, and disclosure boundary in Sections 3.2 and 3.3 and Table 1 are more rigorous than what most federated graph papers provide, and the question of whether global structure can be learned through higher-level units is clearly posed.

  2. Method design: The design is principled and coherent. Using geometric renormalization to form supernodes gives a defensible criterion for grouping beyond embedding similarity, and the separation of concerns is elegant: inferred topology determines membership, while learned representations determine the released directions. The binomial edge model on the supergraph in Eq. 4, which disentangles connection opportunity, hidden-degree popularity, and directional alignment, is a thoughtful probabilistic formulation. The coarse-to-fine feedback through the private membership matrix and the exact additive decomposition of spherical and vMF sufficient statistics for secure aggregation are technically sound.

  3. Experiments: The empirical coverage is broad: five datasets, thirteen baselines spanning four federated optimization frameworks crossed with three deep graph clustering backbones plus GCFGAE, four metrics, five seeds with standard deviations, client-count scaling, supernode capacity sensitivity with a structural explanation in Figure 4, ablations, analytical communication accounting, and MP-SPDZ numerical validation of the secure primitives. The main result margins on the citation networks are large, as shown in Table 2.

Critical weaknesses
  1. Core mechanism shows mixed evidence. The feedback ablation in Table 3 shows that removing server feedback drops Cora ACC by 28 to 35 points but improves CiteSeer ACC by 5.58 points, i.e., a central component of the method can cause negative transfer. The authors acknowledge this and suggest adaptive feedback strength as future work, but no mechanism or analysis is provided in this submission.

  2. Privacy claims are validated only indirectly. Main performance experiments use trusted simulation for cross-edge construction as described in Section 5.6; the actual MPC protocol is validated only at small scale and separately. Privacy is conditional on authorized releases, and the leakage risk of released supernode directions, sizes, and edge counts, especially at small capacity b, is acknowledged but never quantified with an attack or inference evaluation. No differential privacy guarantee is provided, which is fine, but the empirical leakage study would strengthen the paper.

Questions for authors

Please refer to the weaknesses.

2 comments

Dear Reviewer 831o,

We sincerely thank you for your detailed review and insightful suggestions. Below, we address your comments and concerns point by point.

Response for W1: Server feedback can cause negative transfer on CiteSeer, but the underlying mechanism remains insufficiently analyzed.

Among the evaluated datasets, CiteSeer is the only dataset on which performance improves when global feedback is removed. Table 3 presents performance comparisons on a subset of the datasets. We have extended the comparison between the full model and the variant without global structural feedback to all five datasets, with the results shown below.

DatasetSettingACCNMIARIMacro-F1
CoraFull72.35±5.4955.94±1.9351.16±5.3270.53±5.04
CoraNo-feedback39.44±4.1221.92±6.9413.84±4.2532.77±7.53
CiteSeerFull58.77±2.5932.45±1.6430.81±2.1855.48±2.23
CiteSeerNo-feedback64.35±2.3437.90±1.6337.50±2.4859.86±2.38
PubMedFull64.69±1.3423.68±1.2723.40±1.9064.84±1.24
PubMedNo-feedback57.22±3.8416.93±3.3614.84±4.0557.43±4.07
Amazon PhotoFull62.82±1.6250.21±1.9641.16±2.9657.50±1.70
Amazon PhotoNo-feedback52.64±3.7541.01±2.9729.80±2.1550.06±3.37
Amazon ComputersFull36.58±1.2635.76±1.3319.66±1.0430.29±2.30
Amazon ComputersNo-feedback31.73±3.8329.63±4.7713.98±2.9525.55±4.48

Except for CiteSeer, incorporating global feedback improves ACC, NMI, ARI, and Macro-F1 on all four remaining datasets. Specifically, the relative ACC improvements over the no-feedback variant are 83.44%, 13.05%, 19.34%, and 15.29% on Cora, PubMed, Amazon Photo, and Amazon Computers, respectively.

We further analyzed the possible mechanism underlying the performance degradation on CiteSeer, and found pronounced topological differences between CiteSeer and the other datasets, which may help explain its different response to feedback. The structural statistics are reported below.

DatasetAverage degreeLargest connected component coverageCross-client edge fractionFraction of nodes incident to cross-client edges
Cora4.08100.00%12.09%28.33%
CiteSeer2.7463.72%4.48%8.12%
PubMed4.50100.00%14.75%26.56%
Amazon Photo31.1397.87%18.99%57.27%
Amazon Computers35.7697.30%27.26%67.18%

CiteSeer is sparser and more fragmented, with only 8.12% of its nodes directly incident to cross-client edges. Given this limited coverage, direct structural interactions occur predominantly within local clients, while direct structural influence from other clients is relatively limited. This provides a possible explanation for why the current global feedback module does not improve performance on CiteSeer.

These observations motivate exploring adaptive feedback strength, where topological statistics are mapped to feedback weights through a predefined function or a neural network to mitigate potential negative transfer. We leave this exploration to future work.

W2: Privacy validation relies on trusted simulation and small-scale MPC tests, without empirical assessment of leakage from released supernode statistics.

Real MPC integration test. To address your concern about indirect validation, we have added a two-party MP-SPDZ test on Cora with 10 clients, using actual secure aggregation outputs directly in model updates. The test uses two independent processes on the same machine and covers one round of training collaboration followed by final vMF clustering with 10 initializations. Recorded communication totals approximately 3.65 GB, and execution takes 1,004.20 seconds (16.74 minutes), compared with 350.67 seconds for trusted simulation. Under the same single-round workflow, actual MPC and trusted simulation achieve ACC values of 51.23% and 51.19%, respectively, with differences below 0.08 percentage points across all four metrics.

Empirical leakage evaluation. We thank the reviewer for the constructive suggestion to add an empirical leakage study. Following this suggestion, we have added feature reconstruction experiments using a gradient inversion attack adapted from DLG [1] and Inverting Gradients [2]. We evaluate how public supernode statistics and aggregated gradients help an attacker recover original node features, and examine leakage across different supernode capacities bb, particularly small capacities.

Attack model. We consider a passive attacker. The attack takes as input the shared model parameters θ\theta, available node- or supernode-level information I\mathcal I, observed gradients gobsg_{\mathrm{obs}}, and known features, representations, and gradients from a subset of data partitions, denoted by Kaux\mathcal K_{\mathrm{aux}}. It outputs an anonymous set of reconstructed node features X^\widehat X for the target data:

X^=A(θ,Kaux,I,gobs).\widehat X=\mathcal A(\theta,\mathcal K_{\mathrm{aux}},\mathcal I,g_{\mathrm{obs}}).

Here, A\mathcal A denotes the gradient-matching reconstruction attack. Target features, adjacency matrices, and node-to-supernode assignments remain private.

Attack optimization. Both attacks reuse the same trained FedSuCo model. They keep model parameters fixed and optimize candidate node features and, where needed, latent representations by matching the shared feature decoder’s gradients. Observed gradients are recomputed from the actual local training objective. Auxiliary data and available representations support initialization and applicable constraints.

The node-level attack reconstructs each target partition separately. The supernode-level attack subtracts the auxiliary partitions’ known weighted gradient contributions from the aggregate and jointly reconstructs the remaining partitions. Each attack runs for 3,000 steps from each of three initial guesses, with the final result selected using the observable attack objective.

Experimental setup and results. We evaluate Cora, CiteSeer, and Amazon Photo with b∈3,8,20b\in\\{3,8,20\\} and five model training seeds per setting. Of the ten data partitions, seven are used for evaluation and two provide auxiliary knowledge; the remaining partition also participates in joint reconstruction in the supernode-level attack.

The two attack settings differ in the available information: for the original node-level attack, I\mathcal I contains full node representations and gobsg_{\mathrm{obs}} is the client-specific gradient; for the supernode-level attack, I\mathcal I contains supernode directions, feedback, sizes, and connectivity statistics, and gobsg_{\mathrm{obs}} is the aggregated gradient.

Both attacks are evaluated on the same target partitions using MSE after optimal one-to-one matching within each partition. Lower MSE indicates more accurate feature reconstruction. The table reports means and sample standard deviations across five model training seeds.

DatasetCapacity bbSupernode-level attackOriginal node-level attack
Cora30.010983±0.0000470.003535±0.000439
Cora80.011440±0.0006970.003619±0.000394
Cora200.011270±0.0000320.003833±0.000352
CiteSeer30.007601±0.0000630.004550±0.000532
CiteSeer80.007711±0.0000330.004468±0.000501
CiteSeer200.007806±0.0000170.004481±0.000466
Amazon Photo30.233483±0.0047680.203930±0.011497
Amazon Photo80.236677±0.0071170.203095±0.013182
Amazon Photo200.237991±0.0100930.212949±0.019237

Across all nine settings, the supernode-level attack yields higher MSE than the original node-level attack. Its MSE is 2.94–3.16, 1.67–1.74, and 1.12–1.17 times that of the node-level attack on Cora, CiteSeer, and Amazon Photo, respectively. These results support the role of FedSuCo’s information-sharing mechanism in limiting feature leakage under the evaluated attack settings.

[1] Zhu et al. Deep Leakage from Gradients. NeurIPS 2019.

[2] Geiping et al. Inverting Gradients—How easy is it to break privacy in federated learning? NeurIPS 2020.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.