- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes a new federated learning algorithm by only sharing learned thresholds instead of parameters/gradients across clients, which is supposed to reduce communication overhead.
Critical strengths
The idea of using learnable threshold is an interesting alternative to typical FL algorithms.
Critical weaknesses
-
The paper mentions better privacy protection via threshold sharing but the paper does not have formal analysis of privacy guarantee rather than heuristics.
-
The threat model of the claim is very unclear.
-
The experiments are very limited to small set of image datasets
Questions for authors
Does the attacker in your theory have access to model weights? It appears not but the theory assumes so?
2 comments
W3: Dataset coverage
Thank you for your suggestion to broaden the experimental evaluation.
We believe that the three datasets are well established and appropriate benchmarks for evaluating GatedAFL. Notably, TCT (Yu et al., 2022), SpaFL (Kim et al., 2024), and SkyMask (Yan et al., 2024) all use the same three datasets: FMNIST, CIFAR-10, and CIFAR-100. These studies address federated optimization, model sparsification, and security, which are closely related to the objectives of our work.
Furthermore, our experiments cover different classification complexities, ranging from 10 to 100 classes, and diverse model architectures, including LeNet, ViT, and ResNet-18. All experiments involve 100 clients with heterogeneous data distributions, allowing us to evaluate accuracy, model sparsity, communication efficiency, and resistance to reconstruction attacks across different datasets and model architectures.
Therefore, our benchmark selection aligns with established research practices and provides a comprehensive evaluation of GatedAFL across multiple experimental settings.
References for this response
Y. Yu, A. Wei, S. P. Karimireddy, Y. Ma, and M. I. Jordan. TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernels. NeurIPS, 35, 2022.
M. Kim, W. Saad, M. Debbah, and C. S. Hong. SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low Computational Overhead. NeurIPS, 37:86500--86527, 2024.
P. Yan, H. Wang, T. Song, Y. Hua, R. Ma, N. Hu, M. R. Haghighat, and H. Guan. SkyMask: Attack-Agnostic Robust Federated Learning with Fine-Grained Learnable Masks. ECCV, pp. 291--308, 2024.
W1: Formal privacy analysis
We would like to argue that our privacy claims are supported by a formal information-theoretic analysis, rather than by heuristics alone. Specifically, Theorem 1 establishes that, for the same model parameters, a single-step threshold gradient reveals no more information about the private data than the corresponding parameter gradient, as measured by mutual information. Theorem 2 and Appendix A.2 further quantify the information-leakage difference under their stated assumptions.
Mutual information is a widely used privacy measure. For example, Calmon and Fawaz (2012) use it to characterize privacy against statistical inference, and Elkordy et al. (2023) use it to analyze privacy leakage in federated learning with secure aggregation. Our results therefore provide a theoretical comparison between the two sharing mechanisms, in addition to the empirical reconstruction evaluation.
Manuscript revision. Section 4 now clarifies the information-theoretic basis and scope of our privacy analysis, including its relationship to optimal reconstruction risk and relevant prior work.
W2/Question: Attacker information in the theory and protocol
Thank you for the question.
Yes. In our theoretical comparison, the attacker is assumed to know the model parameters w. We provide the same w as side information when evaluating the information revealed by the threshold gradient and the parameter gradient. In the reconstruction experiments, the attacker is also given the initial local weights wk(t), the initial thresholds τ(t), and the true labels, and replays the local training steps to match the observed update. We consider known w for two main reasons.
First, our gradient-matching baselines, IG (Geiping et al., 2020) and GradInversion (Yin et al., 2021), assume that w is available as side information. Reconstructing images using only G without w is generally much more difficult, and we are not aware of a general method that achieves this without further assumptions. We therefore also provide w to align our analysis with these baselines. Methods that do not require the same weight information either exploit additional structure or recover more limited information. For example, IG's Appendix D gives a single-sample analytical recovery result using the specific structure of a biased fully connected layer, while BlindSage (Arazzi et al., 2024) recovers only data labels without knowing the server model in vertical federated GNNs. Therefore, to guarantee a relatively general and strong attack, we assume that the attacker has access to the model weights.
Second, we use the same W to generate the parameter gradient G and threshold gradient Z to ensure a fair comparison. If the weights differ, the model state itself may affect the privacy gap. In particular, different weights change the activations and derivatives through which private data affect the gradients. Comparing gradients generated at different weights could therefore confound the effect of the sharing mechanism with the effect of the model state.
Manuscript revision. Section 4 now states explicitly that w is theoretical side information, not a transmitted message. It explains the alignment with IG/GradInversion and defines both gradients at the same model state for a fair comparison.
References for this response
F. du P. Calmon and N. Fawaz. Privacy Against Statistical Inference. Allerton, 2012.
A. R. Elkordy et al. How Much Privacy Does Federated Learning with Secure Aggregation Guarantee? Proceedings on Privacy Enhancing Technologies, 2023(1):510-526, 2023.
J. Geiping, H. Bauermeister, H. Dr"{o}ge, and M. Moeller. Inverting Gradients: How Easy Is It to Break Privacy in Federated Learning? NeurIPS, 2020.
H. Yin et al. See through Gradients: Image Batch Recovery via GradInversion. CVPR, 2021.