- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a Sheaf-based Federated Representation Learning framework (SFRL) to address representation alignment among heterogeneous clients in decentralized federated learning. The method uses orthogonal and Stiefel mappings to relate the latent spaces of neighboring clients and introduces a sheaf-Laplacian regularizer to encourage representation consistency. During optimization, the mapping matrices are updated using closed-form SVD solutions to Procrustes problems, while local models are updated through gradient descent. The paper also provides convergence analysis and evaluates private accuracy and communication accuracy on MNIST.
Critical strengths
-
The paper addresses representation compatibility among heterogeneous clients without requiring shared model architectures or a common latent space. This provides an interesting approach to decentralized federated representation learning.
-
The proposed method is clearly presented, and convergence analysis is provided for both deterministic and stochastic settings.
Critical weaknesses
-
The paper uses Communication Accuracy as a key evaluation metric, which measures whether a client's classifier can correctly classify representations received from neighboring clients. However, since each client already has a complete classification model, the practical motivation for transferring representations between clients is unclear. The authors should better explain the real-world applications of this setting and its advantages over local inference.
-
The sheaf consistency regularizer can be reduced to a pairwise representation alignment loss, while the mapping matrices are obtained using classical SVD-based solutions to Procrustes problems. Although the authors compare against several existing methods, more direct ablations are needed to justify the key design choices. For example, replacing Stiefel mappings with simple linear or MLP mappings, or aligning representations after projecting them into a common feature space, would help clarify the benefits of the proposed design.
-
The experiments are limited to MNIST, which is insufficient to demonstrate the effectiveness of the method in more challenging settings. Evaluations on standard benchmarks such as CIFAR-10, CIFAR-100, and Tiny ImageNet are missing. The experiments also lack more representative backbone architectures, such as ResNet and ViT.
-
Privacy is an important concern in federated learning. Although the paper discusses privacy risks and a stricter privacy protocol, transmitting latent representations may still expose information about the original inputs. The paper lacks empirical evaluations of privacy risks, such as feature inversion or reconstruction attacks, particularly for representations transmitted during inference.
-
The paper lacks discussion and experimental comparisons with representative model-heterogeneous federated learning methods, such as FedGH [1] and FedRE [2]. Additional comparisons under comparable settings would strengthen the evaluation.
References
[1] FedGH: Heterogeneous Federated Learning with Generalized Global Header. ACM MM 2023.
[2] FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learning. CVPR 2026.
Questions for authors
Could the authors address the concerns raised in the weaknesses above?
7 comments
Authors response (2/2)
[CW4] We thank the Reviewer and agree. Transmitting latent representations limits, but does not eliminate, the exposure of local data. This exposure is shared by semantic communication and split inference in general, rather than being specific to our framework, and our protocol is compatible with the standard mitigations, such as perturbing the transmitted representations or securing the channel. The strict-privacy variant of App. D reduces what is exposed during training, by exchanging only transported representations, but it does not by itself protect against reconstruction attacks. A systematic empirical evaluation of privacy risks, including feature inversion at inference time, requires a dedicated threat model and lies beyond the scope of this work; we now state this explicitly among the limitations (Sec. 6) and leave it to future work.
[CW5] In the revised version of the paper, we have included the most recent FedRE (which is shown to compare favorably with respect to FedGH in [2]) among the baselines. FedRE performs on par with the non-cooperative baseline in communication accuracy, well below both variants of Sheaf-FRL (Tab. 1). The empirical assessment is also substantially broadened: it now encompasses four different datasets as mentioned above with increased architectural heterogeneity.
References
- Gündüz, D. et al. “Beyond transmitting bits: Context, semantics, and task-oriented communications.” IEEE Journal on Selected Areas in Communications, 41(1), 5-41, 2022.
- Calvanese Strinati, E., et al. Goal-oriented and semantic communication in 6G AI-native networks: The 6G-GOALS approach. In 2024 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit) (pp. 1-6). IEEE.
Thanks for the detailed rebuttal. I appreciate your efforts, which address several of my concerns. However, two concerns remain:
-
Communication accuracy is evaluated only on samples whose ground-truth classes belong to the receiver's target class set. However, the framework lacks a mechanism for selecting an appropriate receiver at inference time when ground-truth labels are unavailable, limiting its applicability to practical collaborative inference.
-
While the authors emphasize limiting data exposure during training, the privacy risks associated with transmitting latent representations remain empirically unexamined. This leaves the privacy implications of the proposed framework insufficiently validated.
As a result, I maintain my original rating.
Authors response (1/2)
We thank the Reviewer for their effort and constructive comments, that we have addressed in the resubmitted version. Below is our response to the Reviewer’s concerns.
[CW1] Communication accuracy does not compare remote and local inference on the same input. It reflects the setting of semantic, or goal-oriented, communication, where the objective is no longer the faithful reconstruction of the transmitted data, but the delivery of the information the receiver needs to accomplish its task (Gündüz et al., 2022), a paradigm regarded as a key enabler of AI-native 6G networks (Calvanese Strinati et al., 2024). In this setting, a transmitter observes data that the receiver does not have, and conveys its meaning through a compact latent representation, rather than through the raw observation. The receiver then performs its task directly on what it receives. Transferring latent representations is thus what allows a receiver to act on observations it has never acquired, while reducing the transmitted payload to task-relevant features and keeping the raw data local. Since transmitter and receiver are trained locally, possibly with different architectures, their latent spaces are not compatible a priori, and a receiver cannot interpret a neighbor's representation unless the two spaces are aligned. Communication accuracy measures exactly this capability, that is, whether agent i can solve its task on data observed only by agent j, which local inference cannot do, since the input is not available at the receiver. Communication accuracy therefore reflects the quality of the alignment between latent spaces, which SFRL promotes during training, whereas the baselines either align independently trained latent spaces post hoc or map them onto a shared space. We have made this motivation explicit at the beginning of Sec. 5. Notably, the largest gains of Sheaf-FRL arise on the classes that the receiver knows but the sender has never been trained on (Tab. 1): the sender learns to encode classes it has no labels for in a form the receiver can decode.
[CW2] While each edge contributes a pairwise alignment term, the gluing penalty is the quadratic form of the sheaf Laplacian, which couples these terms across the network. We have added several ablation studies in App. L of the revised manuscript. Replacing the Stiefel maps with unconstrained linear maps, estimated by least squares on the pilots both during training and at test time (App. L.2), degrades communication accuracy, requires many more pilots, and shows a large spread across seeds. We did not consider MLP maps, since nonlinear maps are not compatible with a sheaf valued in vector spaces, on which the sheaf Laplacian, its global sections and the closed-form map updates rely. Aligning representations after projecting them onto a common feature space is the rationale of ComFed, which learns a projection of every agent onto a shared space jointly with the task, and of our extensions of FedProto and FedMuscle to heterogeneous latent dimensions. These baselines are compared in Sec. 5 under matched settings, with the same encoders, normalization layer and pilots. ComFed falls well below even the non-cooperative baseline on every benchmark, while FedProto and FedMuscle remain below Sheaf-FRL in communication accuracy (Tab. 1). Finally, the coupling across the network has effects that a pairwise alignment does not produce. In App. M, on topologies where global sections exist only if the restriction maps satisfy specific compatibility conditions, DiSheaf-FRL drives the smallest eigenvalues of the sheaf Laplacian two to three orders of magnitude below those of a sheaf fitted post hoc on independently trained agents, so that an approximate space of global agreement emerges from training.
[CW3] In the revised version of our manuscript we provide empirical evidence on MNIST, CIFAR10, FEMNIST, and FashionMNIST datasets. Additionally, we have included ResNets among the considered architectures (see App. I). We focused on ResNet and VGG-style encoders, which allow controlled variations of depth, width and latent dimension across agents; larger benchmarks and transformer encoders are left for future work.