- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces FedCensus, a federated learning framework that replaces gradient-based model aggregation with additive statistics and closed-form computations. The method learns fine and coarse codebooks using federated batch k-means, constructs representations from activation sums, pairwise activation products, and spatial moments, and trains ridge-regression classifiers at multiple deployment levels. The paper establishes partition invariance under full participation and exact arithmetic. Experiments on four image classification datasets demonstrate competitive accuracy under non-IID data partitions.
Critical strengths
- Strong empirical results and extensive analysis. The method achieves competitive accuracy, outperforming several computation-heterogeneous baselines under the reported configurations. The evaluation covers multiple datasets, partition settings, deployment levels, and feature ablations, including empirical validation of the theoretical partition-invariance property.
- Approach to partition-invariant federated representation learning. The paper proposes a framework in which both the encoder and classifier are constructed from distributed statistics rather than gradient aggregation. The theoretical analysis provides a clear explanation of partition invariance, representing a meaningful conceptual contribution.
Critical weaknesses
-
Insufficiently matched comparison with computation-heterogeneous federated learning methods. (a) Unmatched computational budgets: HeteroFL and FedRolex use heterogeneous client budgets of 25/98/390 MFLOPs, whereas the smallest FedCensus model requires 223 MFLOPs. Moreover, FedCensus-S/L require 385/790 MFLOPs, compared with the average 160 MFLOPs of HeteroFL. Thus, the reported improvements do not establish superiority under identical per-client computational constraints. (b) Incomplete efficiency comparisons: The reported 60% communication reduction compares FedCensus against FedAvg after 1,000 rounds rather than at matched accuracy. The reduction in communication rounds is also not accompanied by comparable measurements of total client computation, wall-clock time, or memory requirements.
-
Quadratic communication complexity and an unresolved communication–privacy trade-off. FedCensus requires uploading a Gram matrix with O(D^2) entries (approximately 3.8 GB per client for D = 43,520), potentially limiting scalability and deployment on resource-constrained devices. The installment scheme in Appendix A.13 does not clearly explain how sample-wise partitioning reduces the size of each transmitted Gram matrix. The proposed algorithm seems still limited when dimension goes 10x bigger, and authors need to clarify the effective regime.
Questions for authors
Q1. Fair comparison with heterogeneous FL methods.
Could the authors compare FedCensus against HeteroFL, FedRolex, and DepthFL under identical per-client inference budgets? In particular, how would FedCensus accommodate clients with budgets of 25 or 98 MFLOPs? Would the reported accuracy advantage remain under the same client budget distribution?
Q2. Communication complexity and practical scalability.
Given the O(D^2) communication complexity of the Gram matrix, could the authors clarify how the installment scheme reduces per-upload communication without increasing total communication? Could the authors report actual transmitted bytes, peak memory, and client-side computation costs, and discuss the practical limitations as the feature dimension D increases? An accuracy-matched communication comparison with FedAvg and heterogeneous FL baselines would also be helpful.
Q3. Privacy guarantees and attack evaluation.
Appendix A.14 demonstrates information leakage from individual class statistics, while the factor-based communication scheme is incompatible with secure aggregation. Could the authors clarify the intended privacy threat model? In particular, have reconstruction or inference attacks been evaluated against the per-image seeding centroids and secure-aggregated statistics when only a few clients contribute to a class?
4 comments
I will raise my score accordingly to 2.5, while the communication/memory cost and privacy issues above remain my main reservations.
Thank you for the detailed rebuttal and the additional experiments. Below is my response to each point.
Q1. Comparison at matched inference budgets. I appreciate the new smaller encoder (whitening rank 27, 128 coarse codewords) and the level-by-level comparison against FIARSE, FedRolex, and DepthFL on CIFAR-100 at comparable FLOPs (22.1 / 99.2 / 181.5 MFLOPs). This directly answers my question about clients with 25 or 98 MFLOPs budgets, and I consider this concern largely resolved. However, as Reviewer yvBB also pointed out, part of the large margin may come from the baselines being weak under your evaluation protocol (e.g., FedProto reaches 0.0217 on your split versus 0.2704 on the native HtFLlib split). I suggest stating this protocol difference clearly in the main text next to Table 4, so readers can interpret the size of the gains correctly.
Q2. Communication and practical scalability. Thank you for acknowledging that installments do not reduce the per-upload size (each upload is still 3805.8 MB and total communication grows by a factor of R), and for correcting Section 3.2 and Appendix A.13. I also appreciate that the 60% communication claim was replaced with an accuracy-matched comparison. Two concerns remain:
Peak memory. The rebuttal describes peak memory only qualitatively (arrays, encoder batches, solver workspace). Could you report measured peak client-side memory for both the direct ridge solver and the PCG variant? This number is central to judging deployability on constrained devices. Consistency with the target setting. In your response to Reviewer hYCJ, the target scenario is described as coordinated IoT systems with limited computation budgets. However, according to Table 29, each client uploads about 4 GB (3.99 GB) and uses 1.42× the client computation of FedAvg (21.39 vs. 15.06 TFLOPs). Moreover, at the 0.600 accuracy target, FedAvg needs only 1.90 GB. These numbers are difficult to reconcile with resource-constrained IoT devices. Could you clarify the realistic deployment regime (device memory, bandwidth, and feature dimension D) in which FedCensus is preferable, and state this limitation explicitly in the paper? Q3. Privacy. Thank you for specifying the honest-but-curious threat model and adding the attack experiments. However, the seeding-centroid leakage remains substantial: source-image identification reaches 0.158 against 0.0016 chance. Subsampling one fifth of the centroids reduces the amount of leaked information but does not remove it, and there is no formal guarantee such as differential privacy. In addition, the aggregated statistics are evaluated against only one attack (closed-form reconstruction), so the conclusion that aggregates are safe seems optimistic. I encourage the authors to (i) discuss DP-noised centroids or seeding schemes that avoid uploading per-image centroids, and (ii) evaluate stronger attacks such as optimization-based reconstruction or membership inference, or at least state these as limitations.
I will raise my score accordingly to 2.5-3, while the communication/memory cost and privacy issues above remain my main reservations.
Thank you very much for the detailed response and for increasing our score! I agree with your concerns about privacy and communication. I will add more experiments to better quantify privacy and communication costs and show how to address these issues.
We have conducted further analysis of the memory overhead.
Measured peak client memory is 10.31 GB for direct ridge regression and 2.92 GB for PCG in the default configuration. PCG uses 1024 preconditioner columns and computes local matrix products in batches without storing the Gram matrix. We profiled the largest CIFAR-100 client, with 3,507 images, three views per image, and feature dimension D = 43,520. Peak memory includes Python/PyTorch, local images, cached features, encoding and statistics workspaces, and communication buffers.
We hope this information addresses your concerns about the memory usage.
We thank the reviewer for recognizing partition invariance and for the detailed suggestions on fair comparisons, communication, and privacy. The references and main manuscript are revised for the rebuttal revision.
Q1. Comparison at the same inference budgets
We train a smaller encoder with whitening rank 27 and 128 coarse codewords, then derive its deployment levels from the same aggregated statistics. Table 4 in the revised main text compares it with the strongest baseline at each nearby budget on CIFAR-100.
FedCensus reaches 0.5326 macro accuracy at 22.1 MFLOPs, against Federated Importance-Aware Submodel Extraction (FIARSE) at 0.2336 and 25.0 MFLOPs. At 99.2 MFLOPs it reaches 0.5689, against FedRolex at 0.3313 and 98.3 MFLOPs. At 181.5 MFLOPs it reaches 0.5786, against DepthFL at 0.4295 and 172.8 MFLOPs. The first comparison uses fewer FLOPs, and the other two use 1% and 5% more. Table 27 reports all six levels.
Q2. Communication and practical scalability
You are correct about installments. Each partial Gram matrix has the full matrix dimensions. Splitting samples into R installments reduces computation per slot by a factor of R, but each statistics upload remains 3805.8 MB and total statistics communication grows by R. We have corrected Section 3.2 and Appendix A.13.
At approximately the same macro accuracy, FedCensus reaches 0.6246 in 18 rounds with 3.99 GB per client and 21.39 TFLOPs of client computation. FedAvg reaches 0.6250 in 535 rounds with 5.35 GB and 15.06 TFLOPs. We have replaced the 60% communication claim with the resulting 25% reduction. FedCensus uses 1.42 times the client computation. At the lower target of 0.600 accuracy, FedAvg needs 190 rounds and 1.90 GB. Table 29 gives the full comparison. The FedCensus run took 1 h 32 m on 24 CPU threads, while 1000-round FedAvg took 3 h 48 m on a GPU.
We also evaluate preconditioned conjugate gradient (PCG) with low-rank preconditioning (Frangella et al., 2023). With 1024 sampled columns, the largest array falls from 3788.1 to 178.3 MB. The solver takes 98 iterations and reaches 0.6234 macro accuracy, against 0.6237 for direct ridge regression. Total classifier communication decreases from 3805.5 to 3607.6 MB per client, and the full pipeline takes 116 rounds. With 256 columns, the largest array is 44.6 MB, but 168 iterations raise communication to 5911.1 MB (Table 28).
We solve the aggregated ridge system numerically and calculate communication from the exchanged arrays. Clients can compute the matrix products from their features without storing a Gram matrix, and the uploads support secure aggregation. Runtime peak memory includes these arrays, encoder batches, and solver workspace.
At feature dimension D = 435,200, the upper Gram triangle would occupy 378.8 GB. With 100 classes and 1024 sampled columns, PCG requires a 1.78 GB preconditioner upload and 174.1 MB in each direction per iteration. Each array grows linearly with D at the cost of repeated communication. Appendix B.4 gives more details and analysis.
Q3. Privacy and attack experiments
We consider an honest-but-curious server receiving secure aggregates and individual seeding centroids. On 500 CIFAR-100 targets, the centroid attack reaches a PSNR of 15.72 and a SSIM index of 0.556 against the closest true patch, compared with 11.71 and 0.390 for a random other image. Source-image identification within the client's images reaches 0.158 against 0.0016 chance (Table 30). The centroids therefore reveal source information. Sampling one fifth before upload reduces their count from 22,361 to 4,472 while changing micro accuracy by at most 0.0009 across deployment levels (Table 31).
We apply the closed-form reconstruction attack to aggregates of 1 to 50 images of the same class, keeping the encoder, recognizability classifier, and 200 target images fixed. With one image, top-1 recognizability is 0.025 against 0.400 for the originals. At 50 images, PSNR falls from 8.79 to 7.48 and recognizability remains 0.025. Table 32 reports all results, and Appendix B.5 describes the attacks.