- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies asynchronous federated learning in the presence of both exogenous factors (stragglers, network delays, staleness) and endogenous factors (local distribution shifts). The authors argue that existing FL methods rely on a server-determined communication structure. They propose to “decouple” client and server updates: clients and the server run on independent clocks and interact only through two buffers, a client pool read by the server and a server pool read by the clients. Each side is formulated as a dynamic programming problem, and a discrete-time HJB equation is derived for each.
Critical strengths
-
The problem is relevant and under-explored. Linking client participation to the evolution of local utility, and treating exogenous and endogenous asynchrony within a single framework, is a meaningful perspective for practical FL deployments (e.g., the hospital and energy-grid examples in the introduction).
-
The algorithm is simple and easy to implement on top of existing buffered asynchronous FL, with moderate memory overhead.
-
Motivating the asynchronous update rules from a dynamic programming / HJB perspective is an interesting attempt at a unified view. The decomposition of the communication error into interpretable components (staleness, churn, magnitude variance, angular misalignment) is intuitive.
Critical weaknesses
-
The central claim of utility-driven communication is not reflected in the algorithm. The paper claims that clients can decide when to communicate based on how their local utility changes, but, as far as I can tell, Algorithms 1–2 contain no step that measures utility or uses the utility-deterioration condition of Section 3; the only client-side condition is the size of the server pool. In the experiments, participation follows an external schedule shared by all baselines (in the endogenous setting, clients with larger shifts even participate less often), so any utility-driven pattern is a property of the environment rather than a contribution of UAsyncFL.
-
The novelty of the proposed approach compared to existing asynchronous FL methods, as well as the source of performance improvement, remains unclear. Running clients and servers on independent clocks and introducing server-side buffering are already standard features in asynchronous approaches such as FedAsync and FedBuff, while a server pool of size one essentially reduces to the client pulling the latest global model. In UAsyncFL, decisions regarding when to aggregate (threshold C), how to weight (uniformly), and how far to go (α) are still controlled by the server, making it unclear in what sense control has been transferred to the client.
Questions for authors
See weakness.
Thank you for identifying the gap between the participation rule motivated in Section 3 and the procedure in Algorithms 1–2.
W1: Who decides when to communicate? Your observation that the participation pattern is a property of the environment is accurate and therefore, we do not want to measure the utility or assume any control of the communication pattern in our theory. The clients can communicate arbitrarily, the server only observes the client pool. Similarly, the client only observes the server pool and server can communicate arbitrarily. Depending on when client sends parameters, statistics calculated from the client pool change. In summary, server does not dictate when the client can communicate and the client do not dictate when server can communicate.
In the experiments, we prescribe arrival opportunities so that all methods face the same schedule for testing purposes. The rotation experiment makes clients with larger shifts participate less often, a difficult case for a server learning from their updates. We will clarify in the abstract and §3 that we evaluate learning under these arrivals. A client policy that measures utility and triggers reconnection remains untested.
W2: What is new relative to FedAsync and FedBuff? Note that independent clocks and buffering are established features of asynchronous FL. With a server pool of size one, a client pulls the latest global model. Eq. (2) identifies the difficulty, even with independent clocks, each side’s objective depends on the other side’s evolving parameters. This leads to a coupling that forces heterogeneity to be known beforehand such as in fedAsync to ensure that in the presence of staleness, FedAsync can compensate for it.
UAsyncFL claims something different, in our approach, all knowledge about heterogeneity (any type of heterogeneity that results in a bounded communication error) is obtained by looking at how local information (jvp at the client side and change in the server parameters at the server side) changes over communication rounds and how can we regularize for this change. The ability to mathematically perform such updates through the HJB-driven formalism is the core contributions of this work and UAsyncFL is effective in improving results over both FedBuff and FedAsync.
Table 4 compares the corrections under the same endogenous non-IID CIFAR-10 schedule. Notably, accuracy reaches 73.14% with both corrections, but removing the client JVP reduces accuracy to 72.01% (−1.13 percentage points), and removing server momentum reduces it to 69.60% (−3.54 points). With both removed, accuracy is 68.45%, compared with 68.70% for FedBuff.