- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes FedEASE, an adaptive block-level sparse algorithm for client-level differentially private federated learning, aiming to reduce the impact of dimensionality on DP noise. Specifically, the model parameter vector is flattened and partitioned into a number of public contiguous blocks. The server computes a score for each block based on historical information and obtains the marginal inclusion probability of each block by solving an entropy-regularized optimization problem that maximizes a weighted sum of the entropy term and the historical score term under a fixed block budget. Based on these marginal probabilities, FedEASE uses dependent rounding to select exactly L shared blocks. Each client then performs full-dimensional local training, projects its update onto the shared support and retains only the selected blocks, followed by inverse marginal correction and clipping. On the server side, noise-aware feedback updates block scores to guide subsequent allocation. The paper further establishes privacy guarantees and a non-convex convergence upper bound for FedEASE. Experimental results show that under the same nominal noise scale, FedEASE achieves the highest accuracy among DP baselines with competitive runtime, and the gains are more pronounced under strong noise and non-IID settings.
Critical strengths
- This paper designs a block history score h (Eq. 15) and solves an entropy-regularized optimization problem that maximizes a weighted sum of the entropy term and the historical score term under a fixed block budget, thereby ensuring that every block has a positive marginal inclusion probability. This exploits training history while maintaining exploration, and avoids bias caused by non-uniform selection.
- The marginally corrected sparse updates proposed in this paper achieve conditional unbiasedness of the sparse update before clipping, providing a basis for the convergence analysis. The proposed noise-aware feedback mitigates the bias of Gaussian noise in the historical block energy statistics, improving the accuracy of the history score h and thus providing a more reliable historical basis for subsequent block selection.
Critical weaknesses
- Several key designs in this paper are heuristic, such as the choice of the objective function in Eq. (6) and the setting of \beta_t. While these designs appear reasonable, they lack theoretical justification.
- The theoretical contribution is limited. The proof of Theorem 5.3 largely follows existing analyses, and FedEASE does not achieve a faster convergence rate or a smaller convergence error than existing algorithms.
Questions for authors
- Compared with existing algorithms, does FedEASE achieve a faster convergence rate or a smaller convergence error?
- The proposed FedEASE flattens all parameters and partitions them into approximately equal-sized contiguous blocks, which may cross tensor boundaries. Why not partition blocks according to the model structure (e.g., by layer or tensor)? Have the authors evaluated the performance difference between these two partitioning schemes?
- The performance of differentially private LLMs has long been plagued by high-dimensional noise. Can FedEASE be extended to LLMs?
- As shown in Figure 2, FedEASE exhibits lower early-stage accuracy than several baselines. Could the authors explain this phenomenon?