- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes FedExSplit (FES), a Douglas–Rachford splitting (DRS)-based framework for federated learning that explicitly models persistent client states, partial participation, and inexact local updates. Its key idea is to control the implicit error of the full update, providing a unified basis for convergence analysis. The paper establishes convergence guarantees for convex and nonconvex objectives, including stationarity and full-sequence convergence under additional assumptions, and instantiates the framework through two variants.
Critical strengths
One strength of this paper is its unified convergence analysis for inexact DRS updates. By considering the error in the full update rather than only the local solver error, the framework provides a useful way to analyze federated algorithms with partial participation and persistent client states. The paper also goes beyond the general theory by deriving an error bound for FES-MA and examining how client disagreement affects its updates. While the theoretical advantages are established under specific assumptions, the analysis offers some useful insights into the behavior of the proposed method.
Critical weaknesses
The main concern is that the theoretical contribution beyond existing DRS-based federated optimization methods, such as FedDR and iFedDR, is not entirely clear. While the full-update error formulation is useful, the nonconvex convergence rate is comparable to existing results, and the paper could better clarify what important cases can be handled by the proposed framework that were not covered before. In addition, some of the stronger convergence guarantees rely on restrictive assumptions. These issues do not invalidate the technical results, but they make it harder to assess the significance and practical scope of the theoretical advances.
Although the paper is primarily theoretical, the empirical evaluation could be a bit more comprehensive for an ICLR submission. I see this mainly as a suggestion to strengthen the paper rather than a major concern.
Questions for authors
Could the authors clarify more precisely how the proposed full-update error framework extends existing analyses such as FedDR and iFedDR? In particular, what important settings or convergence guarantees can be handled by this framework but are not directly covered by prior results?
We thank the reviewer for recognizing the complete-update analysis and for asking which concrete settings it covers. The framework retains the O(1/k) general stationarity-rate order and analyzes sampling and local-solve errors through a complete-update budget. The revision details these error estimates, separates the assumptions by conclusion, and expands empirical reporting. The italicized questions summarize the review.
Q1. Which concrete cases go beyond a shared reformulation of existing DRS analyses?
A1. The Introduction now motivates a common convergence argument by comparing the error and participation conditions in existing analyses. Appendix D verifies complete-update errors for FES-DR and FES-MA under partial participation and controlled local inaccuracies. For FES-MA, the shared gradient is evaluated at the aggregate mean. An unselected client's state can remain fixed while its gradient component changes with that mean. The error estimate tracks this dependence together with the local solver residuals.
Propositions D.5–D.6 establish expected descent and mean-square step control using the sampling structure. With exact correction and the local stopping rule (3.4), Corollary D.10 gives sufficient structured-descent regimes for every positive participation rate. Proposition C.4 supplies the common convergence argument. These implementation-specific estimates connect the two update designs to the same O(1/k) best expected squared-residual bound under the stated conditions.
Q2. What is the relation to FedDR, DANE, and iFedDR, and what remains to be verified?
A2. Section 3.1 states the correspondence conditions. MA recovers DANE's mean trajectory at γ=τ=1 with full participation, equal weights, exact correction and solves, gradient scale one, and penalty 1/β. DR at τ=1 recovers FedDR without server regularization through the stated transformed states, with exact correction and solves, consistent initialization, and identical selections.
The full-update framework separates an implementation-specific error calculation from the common convergence argument. For an iFedDR update, this calculation would track the complete error induced by its relative-error correction and adaptive steps.
Persistent states originate in existing methods such as FedDR. MA combines them with mean-anchored gradient-corrected local models. FedDANE-related sampled corrections can enter the local model subject to verification of their induced error. Algorithm 2 aggregates persistent-state increments, while FedDANE averages sampled responses.
Exact multicenter aggregation preserves the result when it computes the same projection and other required global quantities. Approximate gossip changes the update and needs a complete-error bound, including its effect on MA's gradient evaluation point.
Q3. How restrictive are the stronger convergence assumptions?
A3. Section 2.2 separates the conclusions. Basic nonconvex stationarity requires smoothness, a finite constrained lower bound, initial-energy integrability, a positive descent margin, and square-summable additive errors. Finite-length and whole-sequence convergence additionally require boundedness, local KL geometry, an energy-process condition, and stronger error-tail summability.
The PL condition discussed in Section 2.2 supplies one route to the required limiting-energy property. The weaker energy-process condition (2.10) permits random limiting energies and is assumed separately from local KL geometry. Appendix C extends the conditional-expectation argument to additive errors. These extra assumptions are used for finite-length and whole-sequence convergence. The basic O(1/k) expected squared-residual order uses the stationarity assumptions listed above.
Q4. What empirical evidence has been added, and what does it support?
A4. Section 4 includes a controlled MLP study over five new seeds and (β,γ) heatmaps across inner-solver step sizes, including γ=1. All four methods use five inner iterations. At LDA .075, the sample SD of A50 is 1.99 for MA and 7.70 for DR, showing greater consistency for MA across these five runs. In the MLP study, both FES variants perform better than Avg and Prox on all five seeds at the strongly heterogeneous LDA levels .15 and .2, and below these baselines at the more homogeneous LDA level 1.
The revised CIFAR-100 task trains a 746,852-parameter nonlinear head on frozen features, with participation .5 and ten paired seeds per LDA level .05, .03, and .01. MA's mean accuracy-AULC gains over DR are 1.900, 2.651, and 4.395 points; gains over Avg are .580, .705, and 1.083. The curves include pointwise 95% Student-t intervals. Appendix H reports the complete development search and selected settings.
DR trails Avg in CIFAR-100 accuracy-AULC. MA's per-iteration accuracy-AULC advantage does not persist at 100 matched communication round trips.