- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes FedCF, a model-heterogeneous federated learning framework based on low-rank continuous feature fields. It represents class-conditional spatial features through compact continuous functions and aligns heterogeneous representations in a shared canonical space. Feature and logit distillation further facilitate knowledge transfer across clients. Experiments on three image classification datasets demonstrate improvements in accuracy and efficiency.
Critical strengths
-
The paper addresses an important challenge in heterogeneous federated learning by exploring a more expressive alternative to discrete prototypes without relying on auxiliary models.
-
The low-rank continuous feature representation offers an interesting approach to cross-architecture knowledge sharing. The proposed framework also considers both representation alignment and classifier calibration.
Critical weaknesses
-
The central claim of preserving rich intra-class structure is insufficiently justified, as the continuous fields primarily capture class-conditional mean features and may still lose sample-specific variations.
-
The experiments are limited to 30 fully participating clients, while communication costs remain substantially higher than prototype-based baselines. This weakens the claimed efficiency advantage and practical applicability.
Questions for authors
-
How do continuous feature fields preserve intra-class variations beyond spatially varying class means?
-
How does Eq. (5) prevent semantic misalignment across heterogeneous architectures?
-
Does Theorem 2.8 provide convergence guarantees for the complete federated training process?
-
How would FedCF perform against prototype-based methods under matched communication budgets?
8 comments
Q2: Cross-Architecture Semantic Alignment
Q2. Cross-Architecture Semantic Alignment in Eq. (5)
We thank the reviewer for the question. FedCF establishes semantic correspondence across heterogeneous architectures in a shared canonical space through semantically meaningful local features, class-conditioned global fields, and feature and coordinate Translators.
1. Mechanism of Cross-Architecture Semantic Alignment
First, alignment in FedCF is grounded in class semantics. Clients learn discriminative backbone features under ground-truth label supervision, and Eq. (3) separately fits a local continuous field for each class, providing class-specific feature references for subsequent alignment. Second, architectures differ in feature dimensions, spatial resolutions, and feature distributions. FedCF uses the feature mapping and coordinate mapping to transform their features and spatial locations, respectively, into a shared canonical space. Eq. (5) then requires same-class features from different architectures to jointly fit one shared global class field, while different classes have separate global fields. Thus, alignment is not merely about making feature values from different architectures numerically similar; it encourages consistent representations for the same class at corresponding spatial locations.
Specifically, Eq. (5) uses alternating optimization: with the Translators fixed, the global fields aggregate same-class features from different architectures; with the global fields fixed, the Translators adjust their mappings toward the shared target for each class. When the fixed class targets are distinguishable, incorrect class mappings increase alignment error, providing class-dependent optimization gradients. Furthermore, the learned canonical-space knowledge supervises features extracted from real client images through Eq. (7), together with the label-based classification supervision in Eq. (9), promoting continued semantic alignment across architectures.
2. Direct Verification of Cross-Architecture Semantic Consistency
We add cross-architecture semantic diagnostics and classification tests on real CIFAR-100 test images, with reference values:
Table R10: Cross-architecture semantic consistency and class separability
| Diagnostic | Reference | FedCF |
|---|---|---|
| Cross-architecture semantic gap ↑ | ≈0 (random class correspondence) | 0.25 |
| Inter-class cosine similarity of global fields ↓ | 1 (complete class collapse) | 0.62 |
| Classification accuracy on real test features ↑ | 1% (random guessing) | 42.0% |
The cross-architecture semantic gap is the difference between the mean same-class and different-class cosine similarities of mapped real test features from different architectures. FedCF achieves 0.25, showing that same-class features from different architectures are more similar than different-class features, indicating meaningful semantic correspondence. The inter-class cosine similarity of the global fields is 0.62, further indicating that different classes remain distinguishable in the canonical space.
We also pass independent test images through the client backbones and feature Translators, then feed the mapped features directly into the frozen global classifier. This inference procedure uses neither ground-truth class labels nor class-conditioned queries to the global fields. FedCF achieves 42.0% accuracy, far above the 1% random-guessing baseline. This demonstrates that class semantics are retained not only in the global fields but also in features extracted from real images and mapped from different architectures.
In summary, FedCF promotes cross-architecture semantic alignment through shared class-conditioned targets and alternating optimization. We have clarified the class-conditional alignment mechanism in Sec. 2.2.2 and added canonical-space diagnostics in Tables 2 and 12 of the revised manuscript.
Q3: Scope of Theorem 2.8
Q3. Scope of the Convergence Guarantee in Theorem 2.8
We thank the reviewer for the careful examination. Theorem 2.8 provides a conditional nonconvex stationarity guarantee for FedCF's client-side optimization under dynamically updated global-teacher supervision. It also explains how multi-point spatial supervision and global-teacher quality affect client optimization.
1. Theoretical Guarantee of Theorem 2.8
During FedCF training, clients optimize against a fixed global teacher within each communication round, while the teacher is updated across rounds. Thus, the analysis covers client optimization over multiple rounds rather than a single round. Specifically, Eq. (10) bounds the average gradient norm of the ideal client objective and reveals three key properties:
(a) The initial optimization-error term decreases with training. As the number of client iterations increases, the contribution of the initial optimization gap to the bound diminishes.
(b) Multi-point queries help stabilize training. When spatial query points are independently sampled, using more points reduces gradient-estimation variance, providing theoretical support for multi-point spatial supervision.
(c) Global-teacher quality affects optimization accuracy. The gradient discrepancy between the actual and ideal teachers contributes to the residual error in the stationarity bound, showing why more accurate global knowledge can improve client optimization.
Under the stated conditions, Theorem 2.8 therefore provides an explicit upper bound on the average stationarity measure of the ideal local objective.
2. Full Teacher Error and Scope of the Analysis
FedCF's actual teacher consists of the global continuous field, feature and coordinate Translators, and global classifier. Appendix E.4 already defines the full gradient discrepancy between the actual and ideal teachers in Eq. (34), and derives the nonconvex stationarity bound incorporating this discrepancy in Eqs. (36)–(47). To align the theorem statement precisely with the actual training process, we have revised the teacher-error term in Eq. (10), following Eq. (47), to be the average squared norm of the full teacher-gradient discrepancy over all local training steps. This accounts for the combined effects of all teacher components without introducing new assumptions or changing the algorithm or the main proof.
In summary, the value of Theorem 2.8 lies in its conditional nonconvex stationarity guarantee for client optimization across communication rounds and its explicit characterization of how multi-point queries reduce gradient variance and how teacher quality affects optimization accuracy. It provides direct theoretical support for guiding heterogeneous clients with continually updated global knowledge in FedCF.
Q1: Intra-Class Structure
Q1. Preservation of Intra-Class Structure and Sample-Specific Variations
We thank the reviewer for the insightful comments. FedCF uses continuous feature fields to preserve feature differences across spatial locations within the same class, while allowing clients to independently learn features from real images. We clarify the mechanism and provide direct experimental evidence below.
1. Mechanism for Preserving Intra-Class Structure
Under Assumption 2.2, Eq. (3) learns the expected class features at different spatial locations. Conventional class prototypes compress an entire class into a single vector, ignoring spatial feature differences. In contrast, the shared learnable coordinate functions and low-rank cores in Eq. (2) model both location-dependent features and their relationships, preserving richer class-level spatial structure and supporting continuous queries at different resolutions. Corollary 2.6 establishes that a class prototype is a spatially constant special case of a continuous field. Lemma 2.7 further proves that the continuous field has no greater optimal approximation error than a single class prototype.
Regarding sample-specific variations, the continuous field provides shared class-level spatial knowledge rather than replacing features extracted from actual images. Even when images of the same class share the same field target at a given location, each client still extracts features from its own input. Eq. (7) provides feature-alignment supervision, while Eq. (9) retains the ground-truth classification loss and prediction distillation; these objectives do not require features from different samples to be identical. Thus, FedCF facilitates collaboration through shared spatial regularities while preserving sample-specific feature learning from real inputs.
2. Verification on a Held-Out Test Set
Fig. 3 in the original paper illustrates differences in spatial responses across samples of the same class, while Fig. 4 demonstrates the reconstruction advantage of continuous fields. We further analyze spatial structure, sample residuals, and reconstruction errors on the held-out CIFAR-100 test set. Spatial mean variance measures variation in class-mean features across locations; sample residual variance measures feature differences between same-class samples at the same location relative to the class mean; and real test-feature MSE evaluates how accurately prototypes and continuous fields reconstruct actual features of unseen samples.
Table R9: Held-out reconstruction and variance analysis
The results show that: a. Spatial structure is effectively modeled. Nonzero spatial mean variance in both spaces confirms location-dependent intra-class structure that a single prototype cannot capture. FedCF also achieves lower reconstruction error for unseen test features in both spaces, demonstrating a more accurate representation of spatial features. b. Sample-specific feature differences remain. Sample residual variances of 0.090 and 0.010 in the local and global canonical spaces, respectively, show that features extracted from different images of the same class have not all collapsed to an identical class mean.
These findings are consistent with the classification gains in Table 1, on CIFAR-100, FedCF achieves 59.30% accuracy, outperforming the strongest prototype baseline (55.63%). In summary, continuous low-rank functions enable FedCF to more accurately represent and share spatial feature structure that a single class prototype cannot retain, while clients continue learning features from individual real images. This improves the expressiveness of shared knowledge without forcing all same-class samples' actual features into a single class vector.