acceptodds
Under review as a conference paper at ICLR 2027

Sheaf-Based Federated Representation Learning

Abstract

Heterogeneous federated systems require agents to learn and exchange informative representations despite differences in data distributions, model architectures, and latent dimensionalities. We propose Sheaf-based Federated Representation Learning (SFRL), a framework that jointly learns and aligns heterogeneous latent representations without sharing parameters, labels, or a single latent space. SFRL relates agent-specific latent spaces through learnable orthogonal transformations and isometric embeddings, promoting consistency via a sheaf-Laplacian gluing regularizer evaluated on a shared set of pilot samples. We develop Sheaf-FRL, a decentralized alternating algorithm combining local gradient updates with closed-form Procrustes updates of the alignment maps, and establish convergence to first-order stationary points in deterministic and stochastic settings. Applied to collaborative classification in semantic communication under model and data heterogeneity, Sheaf-FRL improves private and communication accuracy over federated baselines and is more robust to latent-space compression.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Cellular sheaves, federated learning, representation learning, sheaf Laplacian, Stiefel manifold

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_aWq3OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a Sheaf-based Federated Representation Learning framework (SFRL) to address representation alignment among heterogeneous clients in decentralized federated learning. The method uses orthogonal and Stiefel mappings to relate the latent spaces of neighboring clients and introduces a sheaf-Laplacian regularizer to encourage representation consistency. During optimization, the mapping matrices are updated using closed-form SVD solutions to Procrustes problems, while local models are updated through gradient descent. The paper also provides convergence analysis and evaluates private accuracy and communication accuracy on MNIST.

Critical strengths
  1. The paper addresses representation compatibility among heterogeneous clients without requiring shared model architectures or a common latent space. This provides an interesting approach to decentralized federated representation learning.

  2. The proposed method is clearly presented, and convergence analysis is provided for both deterministic and stochastic settings.

Critical weaknesses
  1. The paper uses Communication Accuracy as a key evaluation metric, which measures whether a client's classifier can correctly classify representations received from neighboring clients. However, since each client already has a complete classification model, the practical motivation for transferring representations between clients is unclear. The authors should better explain the real-world applications of this setting and its advantages over local inference.

  2. The sheaf consistency regularizer can be reduced to a pairwise representation alignment loss, while the mapping matrices are obtained using classical SVD-based solutions to Procrustes problems. Although the authors compare against several existing methods, more direct ablations are needed to justify the key design choices. For example, replacing Stiefel mappings with simple linear or MLP mappings, or aligning representations after projecting them into a common feature space, would help clarify the benefits of the proposed design.

  3. The experiments are limited to MNIST, which is insufficient to demonstrate the effectiveness of the method in more challenging settings. Evaluations on standard benchmarks such as CIFAR-10, CIFAR-100, and Tiny ImageNet are missing. The experiments also lack more representative backbone architectures, such as ResNet and ViT.

  4. Privacy is an important concern in federated learning. Although the paper discusses privacy risks and a stricter privacy protocol, transmitting latent representations may still expose information about the original inputs. The paper lacks empirical evaluations of privacy risks, such as feature inversion or reconstruction attacks, particularly for representations transmitted during inference.

  5. The paper lacks discussion and experimental comparisons with representative model-heterogeneous federated learning methods, such as FedGH [1] and FedRE [2]. Additional comparisons under comparable settings would strengthen the evaluation.

References

[1] FedGH: Heterogeneous Federated Learning with Generalized Global Header. ACM MM 2023.

[2] FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learning. CVPR 2026.

Questions for authors

Could the authors address the concerns raised in the weaknesses above?

7 comments
AuthorsOpenReview ↗

Authors response (1/2)

We thank the Reviewer for their effort and constructive comments, that we have addressed in the resubmitted version. Below is our response to the Reviewer’s concerns.

[CW1] Communication accuracy does not compare remote and local inference on the same input. It reflects the setting of semantic, or goal-oriented, communication, where the objective is no longer the faithful reconstruction of the transmitted data, but the delivery of the information the receiver needs to accomplish its task (Gündüz et al., 2022), a paradigm regarded as a key enabler of AI-native 6G networks (Calvanese Strinati et al., 2024). In this setting, a transmitter observes data that the receiver does not have, and conveys its meaning through a compact latent representation, rather than through the raw observation. The receiver then performs its task directly on what it receives. Transferring latent representations is thus what allows a receiver to act on observations it has never acquired, while reducing the transmitted payload to task-relevant features and keeping the raw data local. Since transmitter and receiver are trained locally, possibly with different architectures, their latent spaces are not compatible a priori, and a receiver cannot interpret a neighbor's representation unless the two spaces are aligned. Communication accuracy measures exactly this capability, that is, whether agent ii can solve its task on data observed only by agent jj, which local inference cannot do, since the input is not available at the receiver. Communication accuracy therefore reflects the quality of the alignment between latent spaces, which SFRL promotes during training, whereas the baselines either align independently trained latent spaces post hoc or map them onto a shared space. We have made this motivation explicit at the beginning of Sec. 5. Notably, the largest gains of Sheaf-FRL arise on the classes that the receiver knows but the sender has never been trained on (Tab. 1): the sender learns to encode classes it has no labels for in a form the receiver can decode.

[CW2] While each edge contributes a pairwise alignment term, the gluing penalty is the quadratic form of the sheaf Laplacian, which couples these terms across the network. We have added several ablation studies in App. L of the revised manuscript. Replacing the Stiefel maps with unconstrained linear maps, estimated by least squares on the pilots both during training and at test time (App. L.2), degrades communication accuracy, requires many more pilots, and shows a large spread across seeds. We did not consider MLP maps, since nonlinear maps are not compatible with a sheaf valued in vector spaces, on which the sheaf Laplacian, its global sections and the closed-form map updates rely. Aligning representations after projecting them onto a common feature space is the rationale of ComFed, which learns a projection of every agent onto a shared space jointly with the task, and of our extensions of FedProto and FedMuscle to heterogeneous latent dimensions. These baselines are compared in Sec. 5 under matched settings, with the same encoders, normalization layer and pilots. ComFed falls well below even the non-cooperative baseline on every benchmark, while FedProto and FedMuscle remain below Sheaf-FRL in communication accuracy (Tab. 1). Finally, the coupling across the network has effects that a pairwise alignment does not produce. In App. M, on topologies where global sections exist only if the restriction maps satisfy specific compatibility conditions, DiSheaf-FRL drives the smallest eigenvalues of the sheaf Laplacian two to three orders of magnitude below those of a sheaf fitted post hoc on independently trained agents, so that an approximate space of global agreement emerges from training.

[CW3] In the revised version of our manuscript we provide empirical evidence on MNIST, CIFAR10, FEMNIST, and FashionMNIST datasets. Additionally, we have included ResNets among the considered architectures (see App. I). We focused on ResNet and VGG-style encoders, which allow controlled variations of depth, width and latent dimension across agents; larger benchmarks and transformer encoders are left for future work.

AuthorsOpenReview ↗

Authors response (2/2)

[CW4] We thank the Reviewer and agree. Transmitting latent representations limits, but does not eliminate, the exposure of local data. This exposure is shared by semantic communication and split inference in general, rather than being specific to our framework, and our protocol is compatible with the standard mitigations, such as perturbing the transmitted representations or securing the channel. The strict-privacy variant of App. D reduces what is exposed during training, by exchanging only transported representations, but it does not by itself protect against reconstruction attacks. A systematic empirical evaluation of privacy risks, including feature inversion at inference time, requires a dedicated threat model and lies beyond the scope of this work; we now state this explicitly among the limitations (Sec. 6) and leave it to future work.

[CW5] In the revised version of the paper, we have included the most recent FedRE (which is shown to compare favorably with respect to FedGH in [2]) among the baselines. FedRE performs on par with the non-cooperative baseline in communication accuracy, well below both variants of Sheaf-FRL (Tab. 1). The empirical assessment is also substantially broadened: it now encompasses four different datasets as mentioned above with increased architectural heterogeneity.

References

  • Gündüz, D. et al. “Beyond transmitting bits: Context, semantics, and task-oriented communications.” IEEE Journal on Selected Areas in Communications, 41(1), 5-41, 2022.
  • Calvanese Strinati, E., et al. Goal-oriented and semantic communication in 6G AI-native networks: The 6G-GOALS approach. In 2024 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit) (pp. 1-6). IEEE.

Thanks for the detailed rebuttal. I appreciate your efforts, which address several of my concerns. However, two concerns remain:

  1. Communication accuracy is evaluated only on samples whose ground-truth classes belong to the receiver's target class set. However, the framework lacks a mechanism for selecting an appropriate receiver at inference time when ground-truth labels are unavailable, limiting its applicability to practical collaborative inference.

  2. While the authors emphasize limiting data exposure during training, the privacy risks associated with transmitting latent representations remain empirically unexamined. This leaves the privacy implications of the proposed framework insufficiently validated.

As a result, I maintain my original rating.

Reviewer_Cr9zOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Sheaf-based Federated Representation Learning (SFRL) for federated settings where agents differ in data, model architecture, and latent dimensionality. Rather than requiring a shared latent space, it treats each agent's latent space as a local coordinate chart on a graph and learns orthogonal/Stiefel restriction maps on the edges that isometrically transport neighboring representations into a common edge stalk. A sheaf-Laplacian gluing regularizer is defined on a shared set of pilot samples, and Sheaf-FRL alternates local gradient updates with closed-form Procrustes updates of the maps. The authors prove convergence to block-wise first-order stationary points (O(1/T) deterministic, O(1/√T) stochastic). Experiments use MNIST with 15 heterogeneous CNN agents plus a two-agent bottleneck sweep; Sheaf-FRL outperforms ComFed, Sheaf-FMTL, FedProto, FedMuscle, and a non-cooperative baseline on private and communication accuracy.

Critical strengths

• Clear motivation. Independently trained representations can differ by rotations, reflections, or isometric embeddings, so forcing a single shared latent space is unnecessarily restrictive.

• Elegant sheaf formulation. Node/edge stalks, orthogonal and Stiefel maps, the sheaf Laplacian, and the pilot-based gluing penalty fit together naturally and handle heterogeneous dimensions.

• Practical algorithm. The Procrustes updates are closed-form and simple; only pilot representations are exchanged, not parameters or labels. The homogeneous case reduces to orthogonal synchronization, which is a useful sanity check.

• Reasonable theory. The convergence results hold under standard assumptions, and the authors clearly state that the strict-privacy variant is not covered.

• Encouraging empirical signal. The advantage grows with distribution shift and with narrower bottlenecks.

Critical weaknesses

• The empirical scope is too narrow. All experiments are MNIST with synthetic label shift; there is no CIFAR, DomainNet, FEMNIST, modality heterogeneity, or real semantic-communication channel. No multi-seed runs or error bars are reported. The motivation mentions multimodal and robotics settings, but the experiments are single-modality and channel-free.

• The core mechanism is not ablated. There is no comparison of the Stiefel/isometry constraint against an unconstrained linear map, and no ablation of whitening/coloring, pilot count/selection, λ_ij, or graph density. It is therefore unclear whether the gains come from the sheaf geometry or merely from joint training, extra pilots, or the whitening layer.

• The theory is standard and has gaps. It only establishes block-wise first-order stationarity, with no bound on representation generalization or transport error; the strict-privacy algorithm is not analyzed; and the whitening matrix is updated during training but treated as fixed in the analysis. This should be clarified.

• Privacy claims are too strong. The main experiments exchange raw latent pilots; the strict-privacy variant only offers Prop. D.1 (identifiability up to an orthogonal transform), with no DP guarantee and no reconstruction/membership/attribute-inference experiments. The pilots are a shared 10% of MNIST, and the 15-agent graph is built from class overlap, i.e., label information.

• The assumptions are strong. The method requires known pilot correspondence across agents, same-modality pilots, a fixed graph and fixed λ_ij, and isometric maps. These are hard to satisfy in multimodal, cross-silo, and ad-hoc settings.

• The distinction from a shared latent space is not quantified. Shared pilots plus whitening effectively establish a common geometric reference. The paper does not measure how far the learned edge maps are from a single global orthogonal transform, nor whether a global section actually emerges.

Questions for authors

See the weaknesses

6 comments
AuthorsOpenReview ↗

Authors response (1/3)

We thank the Reviewer for their effort and constructive comment. As detailed below, we have addressed the raised concerns in the revised manuscript.

[CW1] We understand the Reviewer's criticism. In the revised manuscript we have substantially broadened the empirical assessment of the proposed method. First, we now provide experiments on four different datasets (MNIST, CIFAR10, FEMNIST, FashionMNIST). Second, we increased the heterogeneity among the considered architectures. Third, dispersion over 3 seeds is now provided. The results confirm that both variants of Sheaf-FRL outperform all baselines in communication accuracy on every dataset, with statistically significant gaps, while all methods are statistically tied in private accuracy (Tab. 1). As far as multi-modality and real communication channels are concerned, we agree with the Reviewer that these are important research directions. However, we believe that they deserve dedicated work and lie beyond the scope of this work.

[CW2] App. L in the revised manuscript provides the ablations on (i) number of available pilots, (ii) Stiefel geometry, (iii) the sensitivity to the choice of λij\lambda_{ij}, (iv) the comparison CenterNorm against analytical and learnable whitening, and (v) the communication topology, comparing star, ring and fully connected graphs (App. L.5). With a few hundred pilots, DiSheaf-FRL already communicates almost losslessly. On MNIST, it reaches 97-99% communication accuracy on the classes the sender has never been trained on, against 82-86% for the non-cooperative baseline, while on the classes known by both agents both methods are around 99%, as is its private accuracy on the target classes. Replacing the Stiefel maps with unconstrained ones, or CenterNorm with whitening, degrades communication accuracy with the same pilots and the same gluing penalty, and analytical whitening even falls below the non-cooperative baseline. The gains therefore stem from the isometric geometry and the geometric reference, rather than from joint training or the normalization layer alone. Moreover, DiSheaf-FRL is robust to λij\lambda_{ij} over more than one order of magnitude, and its gain persists on every tested topology, being slightly larger on sparser graphs. Pilot selection strategies are discussed in App. F . Finally, all the methods use the same pilots, so there are no “extra” pilots in our experiments.

[CW3] We answer the three concerns separately. Whitening matrices. In the revised version, the learnable whitening layer has been replaced by a center-and-normalize (CenterNorm) layer (Eq. (17), App. E), which performs better than whitening (App. L.4). CenterNorm has no trainable parameters, but its normalization statistics are updated during training, since it combines the minibatch mean with a running mean. The analysis in App. H now accounts for these updates explicitly: the running mean is treated as a state on which the objective depends, and the change of the objective induced by each update is bounded (Lemma H.2). Convergence to first-order stationary points is preserved in the deterministic setting, where the full-batch mean is used (Rem. 2), and holds up to a neighbourhood induced by the running statistics in the stochastic setting.

Strict-privacy. As stated at the beginning of App. H, the convergence results are not claimed for the strict-privacy variant (App. D), whose simultaneous updates of the coupled restriction maps do not constitute an exact minimization of the map block; we have made this explicit also in the main text (Sec. 4).

Scope of the guarantees. We agree that our results are optimization guarantees: block-wise first-order stationarity, at the rates standard for nonconvex alternating schemes (up to a neighbourhood in the stochastic setting, see above). Bounds on the generalization of the learned representations, or on the transport error of the maps outside the pilot set, address a different question. The identifiability conditions of App. G characterize when the maps are uniquely determined by the pilots, and the normalized total variation on the test set (App. M) provides an empirical counterpart. We regard these as natural extensions of the analysis rather than gaps affecting the validity of the stated results, and we have listed them among the directions for future work (Sec. 6).

AuthorsOpenReview ↗

Authors response (2/3)

[CW4] We agree that two formulations overstated our privacy claims. We have revised them: (i) Prob. (SFRL) now states that agents cooperate without exchanging raw data or model parameters, rather than “while preserving agents' privacy”, and (ii) the paragraph of App. D is now titled “Non-identifiability of the exchanged representations”, making clear that Prop. D.1 characterizes the non-identifiability of the latent coordinates only, and is not a formal privacy guarantee. We do not claim differential privacy or robustness to reconstruction or inference attacks; as discussed in our response on privacy to Reviewer aWq3, an empirical evaluation of these risks requires a dedicated threat model, and we now list it among the limitations (Sec. 6).

Regarding the sharing of pilots and the communication graph we believe there is a misunderstanding about what is shared. The pilots are common by construction, and should therefore be drawn from public data, calibration signals or standardized inputs (App. F, Ethics statement), as in other federated methods relying on a shared public dataset. In our experiments it is a held-out portion of the same dataset, disjoint from every local pool, so that no private training sample is exchanged. The communication graph topology is an input of the problem, which encodes structural relationships among agents, as is standard in graph-based federated multi-task learning. In the revised experiments the graph is fully connected, so that no information on the agents is used to build it, and no label is exchanged during training.

[CW5] Most of these assumptions are already stated among the limitations of the paper (Sec. 6), and we briefly clarify their scope. Pilots need not share the same modality: the pilot inputs xikx_i^k are agent-specific, and the formulation only requires that the pilots of different agents correspond to the same underlying sample (App. F). This includes paired observations across modalities (Sec. 3.1, footnote 2); our experiments are unimodal, as acknowledged in Sec. 6. A known correspondence between pilots is the assumption shared by alignment methods based on anchors (Moschella et al., 2023; Hüttebräucker et al., 2024) or on a public dataset (Li & Wang, 2019), and a fixed communication graph with fixed edge weights is standard in graph-based federated learning (Dinh et al., 2024). Inferring the correspondence and adapting the edge weights to the inconsistency measured on each edge are natural extensions that we list as future work. The isometric constraint is a design choice, motivated in Sec. 3.2 by the observation, which the Reviewer also highlights among the strengths, that independently trained representations can differ by rotations, reflections, or isometric embeddings. The same constraint yields the closed-form map update and the compact feasible set underlying the analysis. Its effect is quantified by the ablation with unconstrained linear maps in App. L.2. Finally, we note that cross-silo settings, with few participants, stable topologies and public or calibration data, are among those where these assumptions are most naturally met.

AuthorsOpenReview ↗

Authors response (3/3)

[CW6] We thank the Reviewer for this suggestion, which we now quantify in the new Apps. J and M. First, the geometric reference is common only up to isometries. The CenterNorm layer removes offsets and scales, but each agent remains free to rotate and anisotropically deform its latent space, and the restriction maps capture the isometric part of these differences. Second, we compare the sheaf Laplacian learned by DiSheaf-FRL with the one obtained by fitting Procrustes maps on independently trained agents, over four topologies with latent dimensions di∈{4,8,16,32}d_i \in \{4,8,16,32\}. With isometric maps, the space of global sections has dimension at most mini di\mathrm{min}_i\ d_i (D’Acunto et al., 2026), and in three of the four topologies global sections exist only if the restriction maps satisfy specific compatibility conditions, such as intersecting images. There, DiSheaf-FRL drives the smallest eigenvalues of the Laplacian two to three orders of magnitude below those of the post-hoc sheaf, i.e., an approximate space of global agreement emerges from training, which a post-hoc alignment does not produce. Third, the representations do not collapse onto this space. On the test set, the fraction of energy of the embeddings captured by the mini di\mathrm{min}_i\ d_i lowest-frequency eigenvectors saturates at about 0.50.5 (about 0.20.2 for the post-hoc sheaf), and the normalized sheaf total variation remains far from zero, although it decreases during training, whereas it increases for independently trained agents. Global agreement thus emerges on a low-dimensional subspace, 44 out of 6060 dimensions, while about half of the energy remains in agent-specific directions. The learned representations are therefore not glued into a single shared latent space, consistently with our claim that global consistency need not be imposed for effective communication. Finally, on the four benchmarks of Sec. 5, both variants of Sheaf-FRL attain the best alignment, as measured by (i) mean reciprocal rank, (ii) total variation, and (iii) the smallest eigenvalue of the sheaf Laplacian (App. J).

References

  • Moschella, Luca, et al. "Relative representations enable zero-shot latent space communication." ICLR 2023.
  • Hüttebräucker, Tomás, et al. “Relative representations of latent spaces enable efficient semantic channel equalization.” 2024 IEEE Globecom Workshops (GC Wkshps). IEEE, 2024.
  • Li, D., & Wang, J. FedMD: Heterogenous federated learning via model distillation. NeurIPS 2019.
  • Dinh, C. T., et al. “A new look and convergence rate of federated multitask learning with Laplacian regularization”. IEEE Transactions on Neural Networks and Learning Systems, 35(6), 8075-8085, 2024.
  • D'Acunto, G. et al. (2026) “Networks of causal abstractions: A sheaf-theoretic framework” arXiv preprint arXiv:2509.25236.
Reviewer_mwtKOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Sheaf-based Federated Representation Learning (SFRL), a framework for aligning heterogeneous latent representations across decentralized agents. It combines orthogonal and Stiefel mappings with sheaf-based regularization and alternating optimization. The authors analyze its convergence and evaluate it on MNIST using collaborative classification tasks. The results show improved private and communication accuracy over several baselines, particularly when latent dimensions are reduced.

Critical strengths
  1. The paper addresses representation alignment across heterogeneous federated models without requiring a shared latent space. The use of sheaf-based orthogonal and Stiefel mappings provides a useful formulation for handling different architectures and latent dimensions.
  2. The proposed decentralized algorithm uses closed-form Procrustes updates for alignment, avoiding iterative optimization of the transport maps. The convergence analysis also provides theoretical support for the proposed optimization procedure.
Critical weaknesses
  1. The gradient update appears inconsistent with the stated optimization objective. In Eq. (9), each edge contributes to the regularization terms of both endpoints, giving a combined edge penalty of λ_ij/K times the squared alignment error. Differentiating this term with respect to the encoder parameters introduces a factor of 2, which appears to be missing in Eq. (11) and the derivation in Appendix H. Consequently, Eq. (12) does not seem to perform gradient descent on the objective defined in Problem (SFRL).
  2. All experiments are conducted on MNIST, while the compression study in Fig. 2(c)–(d) uses equal-dimensional latent spaces despite heterogeneous dimensions being a central motivation. Moreover, in the fifteen-agent experiment, FedProto and FedMuscle are adapted using a shared 64-dimensional projection, while Sheaf-FRL retains the original latent dimensions. Different whitening procedures are also used for some baselines. Without a controlled ablation of the Stiefel constraints and these design choices, it is difficult to determine how much of the improvement comes from the proposed formulation itself.
Questions for authors
  1. Could the authors clarify the gradient derivation in Eq. (11)? It appears that a factor of two may be missing when differentiating the regularization objective in Eq. (9). Which formulation is used in the implementation?
  2. Are the whitening matrices treated as fixed during the convergence analysis? If they are updated during training, how are these updates accounted for in the theoretical guarantees?
  3. How sensitive are the reported improvements to the different projection dimensions and whitening procedures used by the baselines? Would the performance gaps remain under matched settings?
4 comments
AuthorsOpenReview ↗

Authors response

We thank the Reviewer for their effort and constructive comments. We have addressed the raised concerns as specified below.

[CW1]+[Q1] The Reviewer is right, the derivative of the combined edge penalty carries a factor 22. We have corrected the typo. Now, immediately before Eq.(9), we define RA=12∑iRA∣i\mathcal{R}_{\mathcal A} = \frac{1}{2} \sum_i \mathcal{R}_{\mathcal A}\mid_i, and update correspondingly Prob. (SFRL). Since the correction is a constant factor on the regularization term, it affects neither the algorithm nor the reported results. Analogously, the convergence analysis is unchanged. In the new experiments, SFRL includes the 22 factor. In the previous experiments, SFRL ran with λij′=λij/2\lambda^\prime_{ij}= \lambda_{ij}/2, with λij\lambda_{ij} within the range reported in App. I.

[Q2] We thank the Reviewer for raising this point, which led us to a more tailored convergence analysis in the revised manuscript. As specified in the general comment, in the new experiments the whitening layer is replaced by a center-and-normalize (CenterNorm) layer appended to the encoder (Eq. (17), App. E), which performs better than whitening (App. L.4). CenterNorm has no trainable parameters, but its normalization statistics are updated during training, since it combines the minibatch mean with a running mean, as in batch normalization. The convergence analysis in App. H accounts for these updates explicitly. The running mean is treated as a state on which the objective depends, and the change of the objective induced by each update is bounded (Lemma H.2). In the deterministic setting, where the full-batch mean is used, the objective is fixed and convergence to first-order stationary points is preserved (Rem. 2). In the stochastic setting, convergence is established up to a neighbourhood induced by the running statistics, which vanishes for pure minibatch statistics or with a summable momentum schedule, in which case almost-sure convergence also holds. We summarize these results and their scope in the paragraph “Convergence analysis” of Sec. 4.

[CW2]+[Q3] We have addressed the Reviewer’s concerns in the new experiments.

On the single dataset and homogeneous experiment. We have added experiments on CIFAR10, FEMNIST, Fashion-MNIST datasets. The bottleneck experiment was originally performed in the homogeneous dimensions also to provide a comparison against FedProto and FedMuscle in their native setting. However, we agree with the Reviewer suggestion and replaced that experiment with a new one in which we sweep over the bottleneck dd while the latent dimensions are dj=d/2d_j=d/2 and di=dd_i=d, so that heterogeneity is still under study (App. K). The results show that both variants of Sheaf-FRL are essentially insensitive to compression on the classes unseen by the sender at every bottleneck, whereas the baselines degrade as the bottleneck shrinks from 6464 to 88.

On the projection dimension. As both FedProto and FedMuscle assume a common latent dimension, the rr-dimensional projection is needed to extend these baselines to heterogeneous latent dimensions (see also footnote 1 in Sec. 3 of the FedMuscle paper (Setayesh et al., 2026), where the Authors comment on how to extend their method to handle model heterogeneity). We now report the upper-bound of the admissible range over our network of agents, that is, r=dmaxr=d_\mathrm{max} (no agent is projected into a space smaller than its own, requiring higher communication cost). The results show that, even with this choice, FedProto and FedMuscle remain below Sheaf-FRL in communication accuracy on every benchmark (Tab. 1).

On matched whitening. All the methods that require a geometric reference for alignment now use the same CenterNorm transformation (Eq. (17) in App.E). The post-hoc alignment of the baselines is also matched: it uses the same closed-form Procrustes map on the CenterNorm outputs, without any whitening.

On the Stiefel constraint. We now ablate the Stiefel geometry directly, and report a comparison against an unconstrained least-squares map. Unconstrained maps require far more pilots, show a large spread across seeds, and remain below Stiefel maps at every budget. On the classes known by both agents they even fall below the non-cooperative baseline (App. L.2). We did not consider nonlinear maps as they are not compatible with a sheaf-theoretic model valued in the category of vector spaces.

References Setayesh, M. et al. “Toward enhancing representation learning in federated multi-task settings”. ICLR 2026.

Follow-up

We would like to follow up to ask if our response effectively addressed the Reviewer’s concerns, highlighted weaknesses, and questions. We take this opportunity to remark that the revised manuscript has been substantially strengthened from several perspectives, as summarized in the general comment above.

Our goal is to encourage discussion to support a fair and thorough evaluation of our work. Thus, we would greatly appreciate prompt feedback, as it would allow us to clarify any remaining issues before the end of the discussion period, and further improve the quality of our manuscript.

In case the Reviewer is satisfied with our response and the clarifications, we would kindly ask them to reconsider their evaluation of our submission.

Thank you for your detailed response. The clarifications provided have been helpful, and I have sufficient information to finalise my assessment.

AuthorsOpenReview ↗

Final thanks

Thank you for the feedback. We are glad that our clarifications have been helpful, and we are grateful for the comments on the gradient derivation and the convergence analysis, which have substantially improved the manuscript.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General comment on the revised version and addressed concerns [1/2]

We thank the Reviewers for their careful reading and constructive feedback. The Reviewers recognize the relevance of aligning heterogeneous agents without requiring shared architectures or a common latent space (Reviewers mwtK, Cr9z, aWq3), and find our sheaf formulation with orthogonal and Stiefel maps elegant and well suited to heterogeneous latent dimensions (Reviewers mwtK, Cr9z). They appreciate the practicality of the closed-form Procrustes updates (Reviewers mwtK, Cr9z), the theoretical support provided by the convergence analysis in both the deterministic and stochastic settings (Reviewers mwtK, Cr9z, aWq3), and the clarity of the motivation and presentation (Reviewers mwtK, Cr9z, aWq3).

We have uploaded a revised manuscript, with all changes highlighted in blue. Besides addressing the Reviewers' concerns, it introduces DiSheaf-FRL, a confidence-gated variant of Sheaf-FRL (Sec. 4.2), in which, on each pilot, the endpoint of an edge whose head is more confident acts as a fixed target for the other. The gate leaves the gluing penalty, and hence the global sections and the map update, unchanged, and only redistributes the gradient between the endpoints. On CIFAR-10 and Fashion-MNIST, it further improves the communication accuracy of Sheaf-FRL. Below, we summarize how the reviewers' concerns have been addressed.

1. Empirical scope (Reviewers mwtK, Cr9z, aWq3). We have substantially broadened the evaluation (Sec. 5, App. I). The experiments now cover MNIST, CIFAR-10, FEMNIST and Fashion-MNIST, with increased architectural heterogeneity, including ResNet encoders, and results are reported as mean and standard deviation over 3 seeds. FedRE (CVPR 2026) has been added to the baselines. On every benchmark, both variants of Sheaf-FRL outperform all baselines in communication accuracy, while all methods are statistically tied in private accuracy (Tab. 1). The compression study (App. K) now sweeps the bottleneck dd with heterogeneous latent dimensions, dj=d/2d_j=d/2 and di=dd_i=d. As the bottleneck shrinks from 6464 to 88, the gap of Sheaf-FRL variants on the classes unseen by the sender grows.

2. Ablations and matched settings (Reviewers mwtK, Cr9z, aWq3). App. L now isolates the contribution of each design choice: the number of pilots (App. L.1), the Stiefel geometry against an unconstrained least-squares map (App. L.2), the regularization weights λij\lambda_{ij} (App. L.3), the geometric reference, CenterNorm against analytical and learnable whitening (App. L.4), and the communication topology, comparing star, ring and fully connected graphs (App. L.5). Notably, with a few hundred pilots, DiSheaf-FRL already communicates almost losslessly, including on the classes the sender has never been trained on, whereas unconstrained maps and whitening degrade communication accuracy. All methods that require a geometric reference use the same CenterNorm layer, all methods use the same pilots, and the projection used to extend FedProto and FedMuscle to heterogeneous dimensions is set to r=dmaxr=d_{\mathrm{max}}, so that no agent is compressed.

3. Geometric reference and convergence analysis (Reviewers mwtK, Cr9z). The learnable whitening layer has been replaced by a center-and-normalize (CenterNorm) layer appended to the encoder (Sec. 3.2, App. E), which has no trainable parameters and performs better than whitening (App. L.4). Since its statistics are updated during training through a running mean, the convergence analysis (App. H) now accounts for these updates explicitly. The objective is treated as depending on the running mean, and its drift is bounded (Lemma H.2). Convergence to first-order stationary points is preserved in the deterministic setting (Rem. 2), and holds up to a neighbourhood induced by the running statistics in the stochastic setting, which vanishes for pure minibatch statistics or with a summable momentum schedule. The scope of the guarantees is summarized in Sec. 4.2.

4. Gradient derivation (Reviewer mwtK). We have corrected the missing factor 22 by defining RA=12∑iRA∣i\mathcal{R}_{\mathcal{A}} = \frac{1}{2} \sum_i \mathcal{R}_{\mathcal{A}}|_i (Sec. 3.3), so that Eq. (11) is exactly the gradient of the objective of Prob. (SFRL). The correction amounts to a constant rescaling of λij\lambda_{ij} and affects neither the algorithm nor the conclusions.

AuthorsOpenReview ↗

General comment on the revised version and addressed concerns [2/2]

5. Shared latent space versus sheaf alignment (Reviewer Cr9z). A new App. M compares the sheaf Laplacian learned by DiSheaf-FRL with the one obtained by fitting Procrustes maps on independently trained agents, over four topologies. Where global sections exist only if the restriction maps satisfy specific compatibility conditions, DiSheaf-FRL drives the smallest eigenvalues two to three orders of magnitude lower, so that an approximate space of global agreement emerges from training. This space, however, has dimension at most mini di\mathrm{min}_i\ d_i and captures only about half of the energy of the embeddings: the learned representations are not glued into a single shared latent space, consistently with our claim that global consistency need not be imposed for effective communication. On the four benchmarks of Sec. 5, App. J further shows that both variants of Sheaf-FRL attain the best alignment.

6. Motivation of communication accuracy (Reviewer aWq3). Communication accuracy measures whether an agent can solve its task on data observed only by a neighbor, which local inference cannot do. This is the setting of semantic and goal-oriented communication (Gündüz et al., 2022; Calvanese Strinati et al., 2024), and we have made this motivation explicit in Sec. 5.

7. Privacy (Reviewers Cr9z, aWq3). We have toned down the privacy claims (Prob. (SFRL), App. D) and clarified that Prop. D.1 is a non-identifiability result rather than a formal privacy guarantee. Sheaf-FRL exchanges only pilot representations, never parameters, labels or private samples, and its strict-privacy variant (App. D) further limits the exposure during training to transported representations. Robustness to inversion and inference attacks, which concerns semantic communication and split inference in general, requires a dedicated threat model, and we discuss it as a direction for future work (Sec. 6).

We hope that these revisions address the Reviewers' concerns; we are happy to provide any further clarification.

AuthorsOpenReview ↗

Summary of the discussion

We thank all the Reviewers, who have helped us substantially improve the manuscript. We briefly summarize the discussion.

  • Reviewer mwtK. We addressed all the points raised in the revised manuscript (the gradient derivation, the convergence analysis, the compression study under heterogeneous latent dimensions, the matched experimental settings, and the ablation of the Stiefel geometry). The Reviewer found our clarifications helpful and stated that they have sufficient information to finalize their assessment, without raising further concerns.
  • Reviewer aWq3 acknowledged that our clarifications addressed several of their concerns. After the discussion, only 1 of the 5 weaknesses initially raised remains open, namely the lack of an empirical evaluation of privacy risks at inference time, which the Reviewer indicated as the sole reason for maintaining their rating. This limitation is explicitly stated in the manuscript (Sec. 6), concerns the exchange of latent representations in general rather than our framework specifically, and will be further discussed in the Ethics statement of the final version.
  • Reviewer Cr9z, who supports acceptance, raised concerns on the empirical scope, the ablations, and the distinction from a shared latent space. We have addressed them with new experiments on 4 datasets over 3 seeds, an extensive ablation study (App. L), and new analyses of the alignment and of the emergence of global sections (Apps. J and M), which support our claim that a single shared latent space need not be imposed a priori among agents. After reading our responses and the other reviews, the Reviewer confirmed their positive assessment.

We thank the Reviewers and the Area Chair once more for their time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.