- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes FedHV, a federated multi-objective optimization method that assigns task weights based on reference-relative objective slacks. To reduce communication and synchronization overhead, clients report task-loss estimates alongside model updates, and the server updates the weights for the next round. The paper provides a convergence analysis of this delayed-weight scheme under client heterogeneity, partial participation, local update drift, and finite-sample loss estimation. Under some assumptions, it establishes an convergence rate for the average squared gradient norm of the log-hypervolume objective, with an additional error term accounting for loss-estimation inaccuracies.
Critical strengths
-
FedHV has a simple communication protocol: clients send task-loss estimates along with their model updates, and the server computes weights for the next round. This avoids an extra synchronization stage and keeps the per-client communication at . This is a practical advantage, particularly when the model dimension is large relative to the number of tasks.
-
The paper does not simply apply the weighting rule in a federated setting and assume that it behaves as intended. It analyzes the effects of partial client participation, local update drift, and errors in the reported task losses. While the resulting guarantees rely on specific assumptions, the analysis addresses issues that arise directly from the proposed protocol.
Critical weaknesses
-
The paper's main contribution is to adapt inverse-slack weighting to federated multi-objective optimization through a delayed weight-update protocol. While this is a reasonable design, the core weighting rule is not new, and the paper does not make a particularly strong case that this adaptation offers a fundamentally better trade-off than existing approaches. The contribution therefore seems incremental.
-
The theoretical section is extensive, but the main convergence result follows a fairly standard non-convex local-SGD analysis, with additional terms accounting for delayed weights and loss-estimation errors. The resulting rate is plus a reporting-error term, while the Pareto-stationarity result follows relatively directly from the inverse-slack weights. I see value in formalizing the behavior of the proposed protocol, but the analysis does not appear to provide a particularly new or strong theoretical insight. The empirical gains are also mixed, so the practical results do not substantially strengthen the case for the contribution.
-
The paper is overly verbose and defensive. The authors seem to spend too much effort explaining and justifying every design choice and anticipating possible objections. This is particularly noticeable in the appendix, where many details are expanded into separate lemmas, corollaries, and discussions. The result feels bloated, and the amount of exposition is not matched by equally substantial insights. The paper would benefit from a more natural and focused presentation.
Questions for authors
What is the main technical novelty of the convergence analysis compared with existing non-convex federated optimization results? In particular, which part of the analysis requires a new argument beyond the standard descent analysis for local SGD with additional error terms?
2 comments
Thanks for the clarification. I agree that handling stale weights in the federated setting requires additional analysis, and I appreciate the authors' effort in addressing this issue. However, I am still not fully convinced that this makes the overall contribution sufficiently novel. I will therefore maintain my original score.
Response to Reviewer tBgh on the convergence proof
Thank you for the question. We agree that the inverse-slack rule, the basic local-SGD descent argument, and the resulting convergence order are established ingredients. The FedHV-specific difficulty is that the implemented update uses one-round-stale, estimated weights (wt=Wρclip(Ft−1;r)), whereas the convergence target is the gradient of the fixed negative log-hypervolume objective Φ, evaluated at the current model.
The analysis addresses this difficulty in two steps. First, the one-round ordering makes (wt) measurable before the current clients are sampled. Thus, Eq. (59) gives conditional unbiasedness for the frozen-weight population gradient before local drift. This conditional-unbiasedness identity would not follow automatically if the same sampled cohort determined both the weights and the gradients.
Second, Eq. (13) separates the discrepancy between stale and ideal weights into previous-round reporting error and inter-round objective change. Under the theorem’s positive true-slack assumption, let et denote the difference between the frozen population gradient and the normalized negative log-hypervolume gradient ct−1∇Φ(θt), where ct is the sum of the inverse true slacks (Eq. (78)). Combining the weight stability bound with the preceding model-displacement bound in Eq. (76) gives, for t≥1,
\leq C_{\mathrm{est}}\Xi_{\mathrm{loss},T} +C_{\mathrm{st}}\eta_{\mathrm{eff}}^2\mathcal V.$$ This bound is then incorporated into the descent inequality for the fixed objective $(\Phi)$ (Eqs. (78)–(86)), rather than establishing descent only for a changing scalarization. Under Corollary 4.2's schedule, the staleness contribution is $(O(T^{-1}))$, while partial participation can leave an explicit reporting-error term even with exact local reports. Thus, we do not claim a new local-SGD descent technique or a stronger asymptotic rate. The theoretical contribution is establishing convergence guarantees for the actual communication-saving delayed-weight protocol, including the effects of heterogeneous client sampling, local drift, and imperfect objective reports, without an additional synchronization stage.