- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes FedHPP, a hybrid privacy-preserving framework for federated learning that combines selective Gaussian noise injection with homomorphically encrypted binary masks. After clipping each client update, FedHPP partitions it into blocks and selects the top-(K) blocks according to an importance score. Gaussian noise is added to the selected blocks, while a binary mask is encrypted using CKKS and aggregated in the ciphertext domain. Clients subsequently decrypt the aggregated mask and use it to gate noise removal. The paper provides theoretical analyses of differential privacy, mask confidentiality, and nonconvex convergence, and evaluates the framework on seven datasets, three model architectures, and ten baselines. The reported results suggest that FedHPP achieves accuracy close to FedAvg while substantially reducing homomorphic-encryption overhead and ciphertext communication compared with selected HE-based baselines. The paper also evaluates its resistance to gradient inversion attacks.
The main question is whether the theoretical guarantees and privacy claims accurately characterize the mechanism implemented by FedHPP. In particular, the relationship between selective noise injection and the claimed differential privacy guarantee, as well as the extent to which encrypted masks conceal the locations of perturbed blocks from an honest-but-curious server, requires further clarification.
Critical strengths
-
An important problem and a potentially useful systems objective. The paper addresses the practical overhead of combining differential privacy with homomorphic encryption in federated learning. Selectively applying noise and reducing encrypted communication are reasonable directions for improving the efficiency of privacy-preserving training.
-
Broad empirical evaluation. The evaluation covers multiple datasets, model architectures, and baseline methods, and considers utility, communication overhead, encryption-related computation, and gradient inversion attacks. This breadth provides a useful starting point for evaluating the practical trade-offs of the proposed framework, assuming the privacy guarantees and implementation details can be clarified.
Critical weaknesses
-
The differential privacy analysis does not appear to establish the claimed guarantee for the implemented selective-noise mechanism. Theorem 1 assumes positive noise variance for every block, whereas the algorithm adds Gaussian noise only to selected top-(K) blocks and releases the remaining coordinates without noise. As written, the theorem therefore appears to analyze a different mechanism from the one deployed. Under standard differential privacy definitions, releasing data-dependent coordinates without noise generally cannot provide the claimed guarantee for the complete update unless additional conditions or protections are established. Moreover, the top-(K) selection itself depends on the client update, and the paper does not clearly establish how the privacy leakage from this selection is accounted for. The noise calibration also requires clarification: whether the stated sensitivity bound is valid depends on the neighboring-dataset definition, the clipping operation, and whether the analysis concerns add/remove or replace-one adjacency. These issues affect the validity of the paper's central privacy claim, rather than merely the tightness of its bound.
-
The confidentiality of the perturbed-block locations is not established against the stated server adversary. The paper argues that homomorphic encryption protects the binary masks, but the server is also assumed to have access to the clients' uploaded updates. If selected coordinates receive Gaussian noise with a standard deviation of approximately 2.1 under the default parameters, while individual clipped update coordinates are typically much smaller, the resulting magnitude distributions may be readily distinguishable. A simple thresholding attack on the visible updates could therefore reveal which coordinates were perturbed, even without decrypting the masks. The paper does not appear to evaluate this inference channel or explain how the encrypted-mask construction prevents it. Consequently, the cryptographic confidentiality of the mask ciphertext alone is insufficient to establish the broader claim that the locations of perturbed blocks remain hidden.
Questions for authors
-
Differential privacy of the implemented mechanism. Can the authors provide a formal differential privacy proof for the actual algorithm, in which only the selected top-(K) blocks receive Gaussian noise and the remaining coordinates are released without noise? How is the data-dependent top-(K) selection accounted for in the privacy analysis? Please also specify the neighboring-dataset definition and derive the corresponding sensitivity bound used to calibrate the noise. If the current proof applies only to a full-support Gaussian mechanism, would the authors revise the privacy claim for the implemented selective-noise mechanism?
-
Inference of perturbed-block locations. Given the stated noise scale and the server's access to uploaded client updates, have the authors evaluated a simple magnitude-thresholding or distribution-based attack to infer which blocks were perturbed? What prevents an honest-but-curious server from distinguishing noisy coordinates from unperturbed coordinates without decrypting the masks? Please report the attack's inference accuracy under the default parameters and discuss its implications for the stated threat model.
-
Mask-guided denoising and convergence. What is the exact role of the aggregated binary mask in the denoising procedure if each client can reconstruct its own noise from its local random seed? Please clarify the mask aggregation semantics, including whether the mask indicates selection by at least one client or some other condition. How does the method handle the possibility that different clients remove different noise components, and does Theorem 3 account for the resulting update differences and the full denoising procedure?
-
Importance scoring and comparisons. Why is the signed block sum used as the importance score instead of a magnitude-based statistic that avoids cancellation between positive and negative entries? How sensitive are the results to this design choice? In addition, can the authors clarify the precise technical differences between FedHPP and CLOVER, FedPHE, and FedSel with respect to privacy guarantees, mask confidentiality, and communication and computation overhead?
3 comments
Q3. Mask-Based Denoising and Convergence Analysis
We thank the reviewer for the constructive suggestions regarding aggregated mask semantics, client-side noise reconstruction, and post-denoising convergence analysis.
-
Aggregated Mask Semantics: FedHPP aggregates binary selection masks via block-wise summation: . This variable represents the number of clients adding noise to block in round . When , exactly one client adds noise, enabling client-side local denoising (). When , multiple clients add noise simultaneously, and FedHPP bypasses this correction to preserve aggregated noise statistics.
-
Ablation Study and Convergence: To evaluate the practical gains of the denoising mechanism, we conducted ablation experiments by disabling the denoising operation:
Variant Partition CIFAR-10(%) CIFAR-100(%) Tiny-ImageNet(%) FedHPP IID 74.93 36.57 34.13 FedHPP w/o denoising IID 72.13 35.22 32.48 FedHPP Non-IID 70.92 31.04 29.87 FedHPP w/o denoising Non-IID 68.43 29.93 28.02 The ablation results fully demonstrate that the mask-guided denoising mechanism consistently enhances model performance across diverse data distributions and secures convergence efficiency under high privacy constraints.
-
Theorem 3 Proof Remarks: In the proof provided in Appendix C.3, we explicitly model the residual aggregated noise term and its variance bound . The proof thoroughly accounts for stochastic gradient variance, client heterogeneity, and residual noise effects, rigorously deriving the convergence rate bound of FedHPP.
Q4. Block-Importance Scoring and Comparison with Prior WorkBlock-Importance Scoring:
We adopt the "mean absolute magnitude" () to prioritize blocks with significant net updates while preventing positive and negative gradient updates from canceling each other out, thereby precisely capturing core optimization directions. We have uniformly revised the corresponding description in the manuscript to "mean absolute magnitude".
Comparison with FedSel, FedPHE, and CLOVER:
we compared FedHPP with methods from other existing works. FedSel achieves local differential privacy through private dimension selection and numerical perturbation, with a privacy analysis that accounts for both the dimension selection process and the protection of the released numerical values. FedPHE employs CKKS-based packed homomorphic encryption (HE) and utilizes secure aggregation and client selection mechanisms to reduce the computational and communication overhead of encryption in heterogeneous federated learning, while also mitigating the impact of straggler clients. CLOVER uses a three-server protocol and secret-sharing techniques to perform secure aggregation while protecting both the values and indices of sparse updates; it further achieves differential privacy goals through distributed noise generation. In contrast, FedHPP does not apply HE to the model parameter values themselves. Instead, it uploads complete, selectively perturbed updates and uses encrypted masks during aggregation to provide selection-count information, which enables local denoising at the client side. The specific differences are as follows:
| Method | Privacy Scope & Positional Info | Primary Computation & Communication Overhead | Complexity (DP+HE) | HE Communication Overhead |
|---|---|---|---|---|
| FedSel | Assigns local DP budgets for dimension selection and selected values separately. | Involves private selection and value perturbation; communication load depends on sparse encoding. | ||
| FedPHE | Protects selected packets using CKKS-based packed HE; the server obtains selection masks. | Involves data packing, blinding, encrypted aggregation, and client selection traffic. | ||
| CLOVER | Utilizes three-server secret sharing to protect sparse indices and values; applies DP to aggregated results under specific assumptions. | Client traffic scales with sparse workload; secure aggregation and noise generation incur server-side inter-communication overhead. | ||
| FedHPP | Adopts dense selective perturbation updates and encrypted block masks; updates may risk positional information leakage. | Involves truncation, scoring, noise sampling, and HE-based mask handling. Uploads values plus ciphertexts (with packing capacity ). |
The authors have addressed some of my concerns. However, it remains unclear to me whether DP can be guaranteed when only top-K coordinates are selected,while the remaining coordinates are released without noise, given that a local model updates typically changes across all coordinates.
Q1. Privacy of the Actual Release, Adjacency, and Calibration
We thank the reviewer for the insightful and constructive guidance. We clarify and tighten the scope, adjacency definition, and privacy proof framework of Theorem 1 as follows:
[1] B. Borja, et al. Improving the gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. ICML, 2018.
[2] B. Mark, et al. Concentrated differential privacy: Simplifications, extensions, and lower bounds. TCC, 2016.
Q2. Inference of Perturbed Data Block Positions
We thank the reviewer for proposing the attack strategy of inferring perturbation positions via magnitude thresholds. We further clarify the noise calibration method, server-observable information, and the evaluation scope of position inference attacks:
The experimental results demonstrate that the server's inference accuracy remains at an extremely low level (around 3% to 8%), thereby strongly validating the confidentiality of the selection patterns.
[3] J. Du, et al. SoK: On Gradient Leakage in Federated Learning. USENIX Security, 2025.