acceptodds
Desk rejected at ICLR 2027

Allocate Before You Perturb: Adaptive Shared Release Spaces for Client-Level Differentially Private Federated Learning

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Federated learning enables collaboration without centralizing raw data. Client-level differential privacy (DP) protects each client's entire dataset. However, at a fixed per-coordinate noise scale, expected isotropic Gaussian noise energy grows linearly with perturbation dimension, degrading utility in high dimensions. Independent client supports can expand during aggregation, weakening control over this dimension. We propose FedEASE, an entropy adaptive framework for shared sparse release that allocates before perturbation. From a fixed public partition and historical DP statistics, FedEASE samples a fixed number of blocks shared across clients before current updates form. This controls aggregation dimension. Entropy regularization balances historical scores and continued block selection. Inverse marginal correction adjusts sparse updates before clipping and compensates for unequal observation frequencies in the history. Noise aware feedback updates history using only released DP aggregates, without querying unprivatized client statistics. We establish client-level privacy, support coverage, and nonconvex convergence guarantees. Extensive experiments on four image classification datasets and IMDB demonstrate FedEASE's accuracy advantages over the compared DP baselines. These advantages are more pronounced under stronger noise.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
federated learning, client-level differential privacy, privacy-preserving machine learning, private optimization, . sparse private release, convergence analysis.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_rWLDOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes FedEASE, an adaptive block-level sparse algorithm for client-level differentially private federated learning, aiming to reduce the impact of dimensionality on DP noise. Specifically, the model parameter vector is flattened and partitioned into a number of public contiguous blocks. The server computes a score for each block based on historical information and obtains the marginal inclusion probability of each block by solving an entropy-regularized optimization problem that maximizes a weighted sum of the entropy term and the historical score term under a fixed block budget. Based on these marginal probabilities, FedEASE uses dependent rounding to select exactly L shared blocks. Each client then performs full-dimensional local training, projects its update onto the shared support and retains only the selected blocks, followed by inverse marginal correction and clipping. On the server side, noise-aware feedback updates block scores to guide subsequent allocation. The paper further establishes privacy guarantees and a non-convex convergence upper bound for FedEASE. Experimental results show that under the same nominal noise scale, FedEASE achieves the highest accuracy among DP baselines with competitive runtime, and the gains are more pronounced under strong noise and non-IID settings.

Critical strengths
  1. This paper designs a block history score h (Eq. 15) and solves an entropy-regularized optimization problem that maximizes a weighted sum of the entropy term and the historical score term under a fixed block budget, thereby ensuring that every block has a positive marginal inclusion probability. This exploits training history while maintaining exploration, and avoids bias caused by non-uniform selection.
  2. The marginally corrected sparse updates proposed in this paper achieve conditional unbiasedness of the sparse update before clipping, providing a basis for the convergence analysis. The proposed noise-aware feedback mitigates the bias of Gaussian noise in the historical block energy statistics, improving the accuracy of the history score h and thus providing a more reliable historical basis for subsequent block selection.
Critical weaknesses
  1. Several key designs in this paper are heuristic, such as the choice of the objective function in Eq. (6) and the setting of \beta_t. While these designs appear reasonable, they lack theoretical justification.
  2. The theoretical contribution is limited. The proof of Theorem 5.3 largely follows existing analyses, and FedEASE does not achieve a faster convergence rate or a smaller convergence error than existing algorithms.
Questions for authors
  1. Compared with existing algorithms, does FedEASE achieve a faster convergence rate or a smaller convergence error?
  2. The proposed FedEASE flattens all parameters and partitions them into approximately equal-sized contiguous blocks, which may cross tensor boundaries. Why not partition blocks according to the model structure (e.g., by layer or tensor)? Have the authors evaluated the performance difference between these two partitioning schemes?
  3. The performance of differentially private LLMs has long been plagued by high-dimensional noise. Can FedEASE be extended to LLMs?
  4. As shown in Figure 2, FedEASE exhibits lower early-stage accuracy than several baselines. Could the authors explain this phenomenon?
Reviewer_H39gOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to choose sparse release spaces for client-level differentially private federated learning. FedEASE uses historical DP aggregates to assign inclusion probabilities to fixed public parameter blocks through entropy regularization. It samples exactly LL shared blocks, corrects selected updates by their inclusion probabilities, and clips them before adding Gaussian noise. Its main addition to shared sparse perturbation is adaptive block allocation with noise-aware historical feedback. The authors provide privacy, coverage, and finite-horizon nonconvex optimization bounds. Experiments on four image datasets and IMDB report higher accuracy than the tested DP baselines in most settings, including gains over uniform shared selection at the default participation level.

Critical strengths
  1. The paper uses previously released differentially private aggregates to choose which parts of the update to release next. This makes shared sparse perturbation adaptive without additional private queries or a representative public training dataset.
  2. The theory explains how the reduction in Gaussian noise energy trades off against errors from random block selection and clipping.
  3. The authors report accuracy gains while keeping uploads sparse and round times close to those of the baselines. They also report gains with deeper models and on text classification.
Critical weaknesses
  1. The main comparison does not establish a privacy–utility advantage at equal privacy budgets. Table 1 matches nominal σ\sigma, but the baseline noise calibration and resulting privacy guarantees are not reported, so the accuracy margins cannot be interpreted as gains under equivalent protection.
  2. The theoretical justification for adaptive allocation stops at coverage and a general convergence bound in Section 5. Neither result explains when the historical scores reduce support variance or clipping distortion relative to uniform sharing.
  3. The baseline optimization settings are insufficiently justified. Fixing C=1C=1 and task-level learning rates across methods with different update scaling and local objectives can affect their relative performance, yet no comparable tuning protocol is reported.
Questions for authors
  1. How do the methods compare in accuracy at the same ε\varepsilon and δ\delta, using a common definition of neighboring client datasets and privacy accounting based on each method’s actual releases?
  2. Under what concrete condition on the historical scores does adaptive allocation reduce support variance or clipping distortion relative to uniform sharing?
  3. Do the reported gains persist after each method’s clipping threshold and local learning rate are tuned with the same validation procedure and search budget?
Reviewer_vybcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Client-level differentially private federated learning suffers from Gaussian noise accumulation in high-dimensional updates, while independent client sparsification may expand the aggregate support. To address these issues, the authors propose FedEASE, an entropy-adaptive shared sparse release framework. FedEASE selects shared parameter blocks using historical DP information before current updates are formed, controlling the perturbation dimension. Entropy regularization balances historical importance and parameter coverage, inverse marginal correction mitigates selection bias, and noise-aware feedback updates block selection probabilities. The authors establish client-level privacy, support coverage, and nonconvex convergence guarantees. Experiments on four image classification datasets and IMDB demonstrate accuracy improvements over the compared DPFL baselines across different noise levels and data distributions.

Critical strengths
  1. FedEASE allocates a common release space before client updates are formed, preventing support expansion during aggregation. Unlike methods that only constrain individual update sparsity, this design directly reduces aggregate Gaussian noise energy and mitigates utility degradation in high-dimensional DPFL.
  2. FedEASE prioritizes potentially important parameter blocks using historical scores while preventing excessive selection concentration through entropy regularization. This enables adaptation to evolving optimization requirements while maintaining opportunities for underselected blocks.
  3. Inverse marginal correction mitigates selection-induced bias, while noise-aware feedback adjusts historical statistics using expected Gaussian noise energy. By relying exclusively on previously released DP aggregates, FedEASE supports adaptive allocation without additional access to unprotected client statistics.
  4. The authors establish client-level privacy, support coverage, and nonconvex convergence guarantees. Experiments across multiple datasets, data distributions, and noise levels provide evidence of improved model utility.
Critical weaknesses
  1. Existing methods such as FedSMP already employ shared sparse supports to reduce perturbation dimensions. Although FedEASE introduces history-driven adaptive allocation and feedback correction, the main comparisons do not sufficiently distinguish its contributions from the benefits of shared sparsification itself.
  2. Noise-corrected aggregate update energy does not necessarily reflect actual optimization contributions. Under Non-IID settings, opposing client updates may cancel out, resulting in low scores for potentially useful blocks. The experiments lack direct evidence linking historical scores to actual parameter contributions.
  3. Section 4.3 establishes conditional unbiasedness before clipping. However, inverse probability scaling may increase update norms and clipping frequency, particularly for blocks with low inclusion probabilities, potentially reintroducing bias and affecting variance. The actual bias reduction after clipping remains insufficiently evaluated.
Questions for authors
  1. Could the authors clarify the independent contribution of history-driven adaptive allocation?
  2. Do historical feedback scores reliably reflect actual optimization contributions?
  3. How does nonlinear clipping affect inverse marginal correction?
  4. Could the authors further evaluate model utility under matched privacy budgets?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.