- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper addresses hybrid federated learning (HBFL), where both sample and feature spaces are fragmented across clients. It proposes Fed-DREAM, a framework that integrates federated divisive clustering, multi-sample representation-level mixup, and an edge-conditioned gated graph neural network to enable cross-client representation fusion. Experiments on FashionMNIST, CIFAR-100, and Multi-News report improved predictive performance over existing methods, while membership inference attacks are conducted to evaluate its privacy-preserving capabilities.
Critical strengths
-
The paper addresses the dual heterogeneity of sample and feature spaces in hybrid federated learning, a problem with research value and practical relevance.
-
The proposed method combines clustering, representation-level mixup, and graph neural networks to facilitate cross-client collaborative learning without directly centralizing raw data.
-
The experiments cover various heterogeneous configurations and include ablation studies, privacy evaluations, and system cost analyses.
Critical weaknesses
-
Hybrid FedGraph (2024) has already proposed using a server-side GNN to aggregate client representations, modeling inter-client interactions based on feature-sharing relationships, and protecting representation privacy through clustering. It also introduces an encoder privacy score based on preimage diameter and bilevel optimization, which is highly similar to the approach adopted in this paper. However, this directly relevant work is neither cited nor discussed. The authors should clearly clarify the substantive novelty of Fed-DREAM over Hybrid FedGraph.
-
The proposed framework combines federated clustering, Dirichlet Mixup, edge-conditioned message passing, and gated attention into a three-stage pipeline. However, the paper does not sufficiently explain what fundamental challenges this combination addresses that existing methods cannot handle.
-
Each client independently samples Dirichlet weights within the same global cluster, and the resulting mixed representations with the same index are combined into a single graph sample. However, these representations do not necessarily correspond to the same original instances or mixing coefficients, so instance-level semantic consistency across client nodes is not guaranteed. This may also introduce a substantial distribution shift between training and inference.
-
Fed-DREAM additionally trains a server-side ECG-GNN on top of the local encoders, whereas some baselines primarily rely on local model optimization, parameter aggregation, or prediction fusion. Therefore, the reported improvements may partly result from the additional server-side model capacity and training procedure, raising concerns about the fairness of the comparisons.
-
The authors convert the original Multi-News summarization task into a multi-label classification task using labels automatically generated by gpt-4o-mini. This is not a standard Multi-News evaluation setting, and the paper lacks validation on established multi-label classification datasets with ground-truth annotations.
-
The 12 HBFL configurations evaluated in this paper are primarily constructed by the authors rather than directly following the original experimental protocols of HyFEM, FedHD, or FedLabSync. Although evaluating all methods under the same partitions is reasonable, the current results only demonstrate the effectiveness of Fed-DREAM under these selected synthetic settings and are insufficient to establish its general applicability to broader HBFL scenarios.
-
Privacy preservation is presented as a major contribution, but the privacy evaluation mainly relies on black-box, loss-based membership inference attacks against the deployed system. It does not directly assess the training-time privacy risks arising from the mixed representations, cluster memberships, and aggregated distances accessible to the server. In particular, when a client holds only one sample within a global cluster, Mixup degenerates into directly releasing the original embedding.
-
The experiments compare Fed-DREAM with only three methods published in or before 2024, without including more recent approaches. The lack of comparisons with newer methods raises concerns about whether the reported improvements adequately demonstrate the novelty and contribution of the proposed framework.
Questions for authors
-
The paper introduces three parameter-sharing configurations: no sharing, selective sharing, and global sharing. However, these strategies are applied to different datasets and experimental settings without a systematic comparison under the same data conditions. What are the respective applicability and limitations of these strategies?
-
In Tables 13–14, removing edge features results in only a minor performance degradation, whereas replacing ECG-GNN with a simple Mean-pool MLP leads to a much larger drop. Can the authors clarify whether the improvements primarily come from modeling feature relationships or from the additional server-side model capacity and nonlinear fusion capability?
-
When a client holds only one sample within a global cluster, Algorithm 5 directly releases the original embedding. Could the authors report how frequently this occurs across the experimental settings and explain whether the proposed privacy protection remains effective in such cases?
-
Table 10 reports three independent runs for each of the two Multi-News configurations, with the results pooled for a one-sided paired t-test. However, based on the six paired observations described in the paper, the reported means and standard deviations appear inconsistent with the p-values of 0.018, 0.016, and 0.006. Could the authors clarify how these p-values were computed?
9 comments
Response to Reviewer pC8o (2/5)
W2 — What fundamental challenge the combination addresses
The challenge is stated in the introduction but, we agree, not sharply: when a client's view covers only a small part of the feature space, static fusion of the clients' outputs fails, because neither averaged parameters (HyFEM, FedHD) nor averaged predictions (FedLabSync) can express which client is informative for which sample. Table 13 gives the clearest evidence: on FM1, where each client holds 2.3% of the features, fusing the clients' embeddings without relational structure (Mean-pool MLP, 0.6641) is worse than simply averaging their independent predictions (w/o node connection, 0.6940), because embeddings from clients that see almost disjoint features blur when averaged; only when the fusion is conditioned on which clients overlap with which (ECG-GNN, 0.7493) does early fusion become a gain. On the high-overlap settings the order is the expected one (late fusion < plain early fusion < relational early fusion, Table 14). The three stages thus serve one purpose: the clustering and mixup make a server-side relational fuser trainable without releasing individual embeddings, and the relational fuser is what the low-overlap regime needs.
W3 — Instance-level consistency across client nodes and train/inference shift
This is a correct description of the design, and it is deliberate. The mixing weights must be sampled independently on each client, because if the server generated or observed them it could solve the mixtures for the underlying embeddings exactly (Appendix A.4, "Server-Blind Weight Generation"); independence is the price of keeping the release one-way. Consistency is therefore approximate and enforced at the cluster level: all nodes of synthetic sample i are mixtures of samples from the same cluster .
How much this costs at inference, where the server sees real embeddings of one sample, is measured. The Raw-embedding reference of Sec. 4.5 trains the same ECG-GNN on the original embeddings and labels; Fed-DREAM is within 2.9 pp of it in every setting at P = 1.0 and above it in six of twelve, and FashionMNIST accuracy is flat up to C = 20 (Appendix F.3), so the shift is small where clusters are tight and the mixup regularization compensates for it, as in centralized mixup. Sec. 3.4 states both facts, and Sec. 4.5 measures the resulting shift with the Raw-embedding reference.
W4 / Q2 — Server-side capacity versus relational modeling
Collaboration in the benchmarks. None of the benchmarks is local-only: FedLabSync aligns the predictions of label-sharing clients over communication rounds, FedLabSync-WA also averages the shared parameters, HyFEM and FedHD couple the local models through a matched server model and global-gradient tracking, and Hybrid FedGraph trains a server-side graph model. Fed-DREAM uses the same local architectures, training budget and communication rounds as the benchmarks (Appendix C.5) and the same parameter sharing as FedLabSync-WA where available (removing that sharing on FM3–FM5 costs 0.5–1.5 pp, Appendix E.2); its collaborative stage is the ECG-GNN. The comparison is between collaborative procedures, not between a local model and one with an added server.
Capacity versus relational modeling. Three comparisons separate the two factors. (i) The Mean-pool MLP keeps the node-encoding module, width (256), dropout and schedule of ECG-GNN and removes only the single message-passing layer (edge-conditioned messages and gating); Fed-DREAM exceeds it in all ten settings, by 2.2 pp on average and by 8.5/7.3 pp on FM1/FM2 (Tables 13–14), where plain pooling even falls below averaging the clients' independent predictions (see W2). (ii) Hybrid FedGraph, a server-side graph model on the same encoders with the same width (see W1), trails Fed-DREAM in all ten settings, by 0.5–14.4 pp and 8.8–15.1 pp at P = 1.0. (iii) Fed-DREAM exceeds Oracle feature union, a centralized model with full feature access, in all five FashionMNIST settings. Server-side capacity alone does not produce the gain.
Q2, answered directly. The reviewer reads Tables 13–14 correctly. The large drop from replacing ECG-GNN by the Mean-pool MLP is the contribution of relational message passing and gating as a whole; the edge feature is one input to that mechanism, with the smallest marginal value where overlaps are large and uniform (CIFAR crops, FM3–FM5) and the largest on FM1–FM2. The smaller effects are consistent and significant: removing the edge feature lowers accuracy in nine of the ten settings (paired one-sided t-tests per dataset, p = 0.001 on FashionMNIST and 0.018 on CIFAR-100), and removing the gate lowers it in all ten (p = 0.013 and 0.002). Sec. 4.4 states this reading: relational fusion is the dominant factor, and edge features and gating add smaller but consistent and significant gains.
Response to Reviewer pC8o (3/5)
W5 — Multi-News labels generated by gpt-4o-mini
We agree that this is not the standard Multi-News evaluation, and we explain why we built the task this way. The hybrid setting requires a text dataset in which one sample is covered by several parties that hold different content and share one target. To our knowledge, no public text dataset with human annotations meets this condition. Multi-News supplies the structure naturally, several articles on the same event, one per client, but its original task, abstractive summarization, cannot serve as the shared target: the compared methods fuse the clients' outputs by averaging logits or by a graph-level prediction, which is defined for a label vector, not for generated text. We therefore derived a 54-label vocabulary from the reference summaries and assigned labels with gpt-4o-mini, using the prompts given in Fig. 1 (Appendix B.1). On the comparison itself: every method is trained and evaluated on exactly the same label set, so any noise in the generated labels affects all methods alike and cannot favor one of them; the prompts are provided so that the labeling can be reproduced and inspected.
W6 — Synthetic HBFL configurations
To our knowledge there is no public real-world HBFL dataset, i.e., one that records which parties hold which samples and which features. Every HBFL study we are aware of, including HyFEM, FedHD, Hybrid FedGraph and FedLabSync, therefore partitions standard datasets synthetically, and their protocols differ from each other (feature groups in HyFEM, random feature subsets in FedHD, a vertical FashionMNIST split in Hybrid FedGraph), so there is no single original protocol under which all methods could be compared. Evaluating all methods under one controlled family of partitions is the alternative, and the family is broad: sample ratios from 6% to 40%, feature ratios from 2.3% to full-size crops, 3 to 50 clients, three sharing regimes, and full and partial participation (Tables 5 and 9); FM3–FM5 follow the grouped-feature protocol of Zhang et al. (2020b) directly, as Appendix B.2 states.
Response to Reviewer pC8o (1/5)
We thank the reviewer for a thorough review. We take the first weakness especially seriously and answer it first, because it concerns what the paper contributes beyond prior work. For the remaining points we give the evidence already in the paper where it exists and report the additional experiments we ran: Hybrid FedGraph as a baseline, a reconstruction attack by the server, an analysis of single-member releases, and FM3–FM5 without parameter sharing.
W1 — Relation to Hybrid FedGraph (arXiv 2404.09443, 2024)
The reviewer is right that Hybrid FedGraph should have been cited and discussed; it is a directly related prior work, and omitting it was an error that we correct in the revision. We state what the two works share and what Fed-DREAM adds.
Both works fuse client representations at the server with a graph model whose edges encode feature sharing, and both cluster samples so that the server does not receive individual embeddings; the privacy score of Appendix A.2 (preimage diameter, bi-level estimation) is Hybrid FedGraph's construction, which we use unchanged and now credit. Fed-DREAM differs in the three stages that follow local training. Hybrid FedGraph clusters at the server, grouping samples at random within each class because following the data geometry there would require disclosing the embeddings; Fed-DREAM clusters in the shared distance geometry under threshold homomorphic encryption, so the server sees only encrypted pairwise distances, medoid indices and memberships (Sec. 3.3). Hybrid FedGraph releases one plain mean per cluster; Fed-DREAM releases Dirichlet mixtures with server-blind weights, which obfuscates and augments at once (Sec. 3.4). Hybrid FedGraph's GCN averages neighbor messages, with the edge vector entering only the first layer; ECG-GNN conditions message generation and gating on the edge attribute at every layer and is trained with client dropout for partial participation (Sec. 3.5). Hybrid FedGraph was evaluated in one vertical setting against clustering variants only.
Results. Hybrid FedGraph is now a baseline on all twelve settings, run under the same partitions, local encoders and server-side hyperparameters as Fed-DREAM (Tables 1, 2, 9 and 10). Fed-DREAM is higher in every FashionMNIST and CIFAR-100 setting at both participation rates: on average by 5.7/5.2 pp on FashionMNIST (P = 1.0/0.3) and by 10.7/9.3 pp on CIFAR-100. On Multi-News, Micro-F1 is higher by 3.5–7.1 pp and mAP by 4.2–6.0 pp; Hybrid FedGraph has the lower Hamming Loss in both settings, and Sec. 4.3 explains why we regard Micro-F1 and mAP as the representative metrics for this task.
Revision. Sec. 2; Appendix A.2; Tables 1, 2, 9 and 10; Sec. 4.3; Appendix D.