- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper identifies three major issues when applying traditional error feedback to differentially private federated learning and proposes Clean Error Residual Feedback which stores the coordinates dropped by sparsification and overwrites and feeds them back only at the coordinates the current mask keeps. Random allocation ensures that each client participates once per cycle. The paper presents privacy and convergence analyses and experiments on Fashion-MNIST and FEMNIST.
Critical strengths
- The empirical study is extensive, which includes both IID Fashion-MNIST and heterogeneous FEMNIST and evaluate several compression rates and participation schedules. The without-feedback comparison is an informative ablation study that shows CERF contributes beyond sparsification alone.
Critical weaknesses
-
The central theoretical result has unvalidated steps. Theorem 3 is used as a narrative hand-wave rather than a mathematical step. The authors state that Random Allocation is bounded by Poisson subsampling via Theorem 3, but then they proceed to prove Theorem 1 using standard Poisson subsampling with , completely ignoring the rate inflation and additive parameters () required by Theorem 3.
-
Lemma 4 holds with strict constraints on the Rényi order . However, in Appendix C.2 line 718, the authors set without proving that this specific falls within the allowed validity domain of Lemma 4 for all parameter choices.
-
In Remark 2, The Jensen step bounding the expected masked residual norm is valid, but its substitution into Eq. (11) is not justified. The function is increasing and strictly convex, so an upper bound on does not provide the claimed upper bound on .
Questions for authors
Please refer to weakness.