acceptodds
Under review as a conference paper at ICLR 2027

FedHV: Low-Overhead Hypervolume Weighting for Federated Multi-Objective Optimization

Abstract

Task-wise federated multi-objective optimization (FedMOO) trains a shared model for competing prediction objectives under heterogeneous data, partial participation, and communication constraints. Existing methods commonly derive task weights from gradient or update geometry. This requires task-specific information or iterative server-side optimization. We introduce FedHV, which maps reference-relative objective slacks to closed-form inverse-slack weights. Each client optimizes one weighted loss and returns objective estimates with its model update. The protocol adds exactly auxiliary scalars per participating client, yielding total per-client communication, compared with the task-specific communication of FSMGDA, and requires no additional synchronization stage. We analyze the resulting one-round-delayed weights under client heterogeneity, multi-step local updates, partial participation, and finite-sample objective reports. Under a fixed-horizon positive-slack reference condition, with the prescribed horizon-dependent step size and vanishing report error, FedHV achieves an rate for the average squared log-hypervolume gradient norm; persistent report error determines the resulting stationarity neighborhood. The same bound controls the squared Pareto-stationarity residual. Across six Dirichlet-partitioned non-IID settings from four vision benchmark families and three training seeds, FedHV exceeds FSMGDA and FedCMOO in mean accuracy in five settings. Among these methods and uniform scalarization, it achieves the highest worst-task accuracy in four settings and improves the difficult CIFAR-10 objective in both CIFAR10–MNIST settings.

Primary area:
optimization
Keywords:
Federated Learning, Multi-Objective Optimization, Hypervolume Optimization, Adaptive Task Weighting, Communication-Efficient Learning, Client Heterogeneity, Pareto Stationarity

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tBghOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FedHV, a federated multi-objective optimization method that assigns task weights based on reference-relative objective slacks. To reduce communication and synchronization overhead, clients report task-loss estimates alongside model updates, and the server updates the weights for the next round. The paper provides a convergence analysis of this delayed-weight scheme under client heterogeneity, partial participation, local update drift, and finite-sample loss estimation. Under some assumptions, it establishes an O(T−1/2)O(T^{-1/2}) convergence rate for the average squared gradient norm of the log-hypervolume objective, with an additional error term accounting for loss-estimation inaccuracies.

Critical strengths
  1. FedHV has a simple communication protocol: clients send task-loss estimates along with their model updates, and the server computes weights for the next round. This avoids an extra synchronization stage and keeps the per-client communication at Θ(d+m)\Theta(d+m). This is a practical advantage, particularly when the model dimension is large relative to the number of tasks.

  2. The paper does not simply apply the weighting rule in a federated setting and assume that it behaves as intended. It analyzes the effects of partial client participation, local update drift, and errors in the reported task losses. While the resulting guarantees rely on specific assumptions, the analysis addresses issues that arise directly from the proposed protocol.

Critical weaknesses
  1. The paper's main contribution is to adapt inverse-slack weighting to federated multi-objective optimization through a delayed weight-update protocol. While this is a reasonable design, the core weighting rule is not new, and the paper does not make a particularly strong case that this adaptation offers a fundamentally better trade-off than existing approaches. The contribution therefore seems incremental.

  2. The theoretical section is extensive, but the main convergence result follows a fairly standard non-convex local-SGD analysis, with additional terms accounting for delayed weights and loss-estimation errors. The resulting rate is O(T−1/2)O(T^{-1/2}) plus a reporting-error term, while the Pareto-stationarity result follows relatively directly from the inverse-slack weights. I see value in formalizing the behavior of the proposed protocol, but the analysis does not appear to provide a particularly new or strong theoretical insight. The empirical gains are also mixed, so the practical results do not substantially strengthen the case for the contribution.

  3. The paper is overly verbose and defensive. The authors seem to spend too much effort explaining and justifying every design choice and anticipating possible objections. This is particularly noticeable in the appendix, where many details are expanded into separate lemmas, corollaries, and discussions. The result feels bloated, and the amount of exposition is not matched by equally substantial insights. The paper would benefit from a more natural and focused presentation.

Questions for authors

What is the main technical novelty of the convergence analysis compared with existing non-convex federated optimization results? In particular, which part of the analysis requires a new argument beyond the standard descent analysis for local SGD with additional error terms?

2 comments
AuthorsOpenReview ↗

Response to Reviewer tBgh on the convergence proof

Thank you for the question. We agree that the inverse-slack rule, the basic local-SGD descent argument, and the resulting convergence order are established ingredients. The FedHV-specific difficulty is that the implemented update uses one-round-stale, estimated weights (wt=Wρclip(F^t−1;r))(w^t=W_{\rho_{\mathrm{clip}}}(\widehat F^{t-1};r)), whereas the convergence target is the gradient of the fixed negative log-hypervolume objective Φ\Phi, evaluated at the current model.

The analysis addresses this difficulty in two steps. First, the one-round ordering makes (wt)(w^t) measurable before the current clients are sampled. Thus, Eq. (59) gives conditional unbiasedness for the frozen-weight population gradient before local drift. This conditional-unbiasedness identity would not follow automatically if the same sampled cohort determined both the weights and the gradients.

Second, Eq. (13) separates the discrepancy between stale and ideal weights into previous-round reporting error and inter-round objective change. Under the theorem’s positive true-slack assumption, let ete_t denote the difference between the frozen population gradient and the normalized negative log-hypervolume gradient ct−1∇Φ(θt)c_t^{-1}\nabla\Phi(\theta^t), where ctc_t is the sum of the inverse true slacks (Eq. (78)). Combining the weight stability bound with the preceding model-displacement bound in Eq. (76) gives, for t≥1t\ge 1,

\leq C_{\mathrm{est}}\Xi_{\mathrm{loss},T} +C_{\mathrm{st}}\eta_{\mathrm{eff}}^2\mathcal V.$$ This bound is then incorporated into the descent inequality for the fixed objective $(\Phi)$ (Eqs. (78)–(86)), rather than establishing descent only for a changing scalarization. Under Corollary 4.2's schedule, the staleness contribution is $(O(T^{-1}))$, while partial participation can leave an explicit reporting-error term even with exact local reports. Thus, we do not claim a new local-SGD descent technique or a stronger asymptotic rate. The theoretical contribution is establishing convergence guarantees for the actual communication-saving delayed-weight protocol, including the effects of heterogeneous client sampling, local drift, and imperfect objective reports, without an additional synchronization stage.

Thanks for the clarification. I agree that handling stale weights in the federated setting requires additional analysis, and I appreciate the authors' effort in addressing this issue. However, I am still not fully convinced that this makes the overall contribution sufficiently novel. I will therefore maintain my original score.

Reviewer_9DAfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FedHV, a federated multi-objective method that sets task weights using inverse slacks relative to a fixed reference. Clients optimize one weighted loss, while their loss reports determine the next round’s weights. The paper provides a convergence analysis for this delayed protocol and experiments showing competitive accuracy with lower overhead than gradient-based alternatives.

Critical strengths
  1. The protocol is simple. It needs only 2m additional scalars per participating client and avoids both task-specific model updates and an extra synchronization stage. The analysis also explicitly accounts for delayed weights, partial participation, and local updates.
  2. The gains on the harder CIFAR-10 task are fairly consistent. Worst-task accuracy improves by about 1.1–1.3 percentage points over the next-best method in the two CIFAR10–MNIST settings.
Critical weaknesses
  1. I’m less convinced about the benefit over simple scalarization. Uniform weighting is already competitive on MNIST–FMNIST and both CelebA settings, and it is missing from the runtime comparison. Since the inverse-slack rule comes from prior work, stronger evidence that adaptive weighting justifies its extra cost would help. Three training seeds on one fixed partition also leave some uncertainty about robustness.
  2. The reference choice is central to both the method and its theory, but it is not adequately tested. The convergence result assumes positive true slacks throughout training, whereas calibration and numerical clipping do not ensure this. Appendix B.8 acknowledges the gap, but there is no controlled reference-margin sensitivity study.
Questions for authors
  1. How sensitive are the results to the calibration margin and seed? What happens when the fixed reference is crossed during training?
  2. Can you compare against Uniform and tuned fixed task weights at the same total wall-clock budget, including calibration and loss reporting? This would help establish whether adapting the weights is worth the additional cost.
  3. Do the gains hold across different client-partition seeds? The current results use three training seeds on a single fixed partition, so it would help to see whether the improvements remain consistent when the client data split changes.
2 comments

Thank you for the questions. We ran the margin and calibration-seed sweeps and repeated the equal-time comparison on two fresh partitions.

Answer to Q1. On CIFAR10–MNIST-A, holding calibration reports fixed, worst-task accuracy averaged 53.37%, 52.87%, and 52.78% at margins 0.075, 0.15, and 0.30. Varying the calibration seed at margin 0.15 gave 52.63–52.99% with training seed and partition held fixed. In the margin sweep, one reported MNIST slack crossed the reference at round 6. Clipping assigned MNIST 99.9997% of the next round’s weight. The next report returned to positive slack. All 153 all-client checkpoints had positive slacks, with the first measured at round 30.

Answer to Q2. On CIFAR10–MNIST-B, we compared FedHV with Uniform and an adaptive loss-weighted baseline at 1,900 seconds. The budget includes FedHV’s calibration and each adaptive method’s loss reporting. The loss-weighted baseline sets wit+1=max⁡F^it,ϵ/∑jmax⁡F^jt,ϵw_i^{t+1}=\max{\widehat F_i^t,\epsilon}/\sum_j\max{\widehat F_j^t,\epsilon}, where F^it\widehat F_i^t is the round-tt reported loss for task ii and ϵ=10−12\epsilon=10^{-12}. This gives more weight to tasks with higher reported losses. Worst-task accuracy was 54.18% for FedHV, 53.03% for Uniform, and 54.14% for loss-weighted. Mean task accuracies were 73.60%, 73.73%, and 73.08%, respectively.

Answer to Q3. At the same 1,900-second budget, FedHV’s average worst-task accuracy relative to Uniform was +1.15 percentage points on the original split (seed 10), 0.82 points on split 11, and −0.08 points on split 12. The two fresh splits shared a frozen reference that differed from the original split’s reference, and round throughput varied across runs.

Thank the authors for the response. I remain concerned about the performance, as the improvements over baselines are very limited. I will maintain my original score.

Reviewer_NxEVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FedHV applies the established inverse-slack weighting rule from single-solution log-hypervolume optimization to task-wise federated multi-objective optimization. Clients report task objective values, use one scalarized local training trajectory, and transmit updates without a separate task-gradient exchange. The paper analyzes one-round-stale weights and reports accuracy, worst-task accuracy, communication, and runtime on several vision multi-task configurations.

Critical strengths
  1. The protocol is simple to implement.
  2. The analysis explicitly acknowledges the one-round staleness of objective reports, partial participation, and multi-step local drift rather than assuming instantaneous exact task weights.
  3. The evaluation reports both average and worst-task accuracy and also discusses cases in which alternative methods lead on the actual hypervolume/loss metrics.
Critical weaknesses
  1. Limited methodological novelty. The inverse-slack weighting rule (Eqs. 3–5) is adopted from Miranda and Von Zuben (2016). The proposed method mainly applies this existing weighting strategy to federated multi-objective optimization.

  2. Insufficient distinction in theoretical contributions. Theorem 4.1 establishes convergence for the proposed weighting mechanism, but it is unclear what theoretical advantages the closed-form weights provide over the iterative weight optimization in FedCMOO. The paper also lacks a clear discussion of the additional challenges introduced by the proposed weighting mechanism compared with existing FedMOO convergence analyses.

  3. Insufficient analysis of the claimed efficiency advantages. FedHV avoids task-specific gradient geometry and iterative server-side weight optimization. However, the reported efficiency gains do not clearly distinguish the benefits of closed-form weight computation from those of eliminating task-specific geometry and additional synchronization. A more detailed breakdown of these contributions would better justify the proposed design.

Questions for authors
  1. Does the proposed closed-form weight computation provide any theoretical advantage over the iterative weight optimization used in FedCMOO, beyond simplifying the implementation?

  2. What are the main technical challenges in proving the convergence of FedHV compared with FedCMOO? Which parts of the theoretical analysis are specific to the proposed weighting mechanism?

  3. Compared with FSMGDA and FedCMOO, FedHV does not require task-specific gradient geometry. How much of its computational and communication advantage comes from this design choice, rather than from replacing iterative server-side weight optimization with a closed-form solution?

2 comments
AuthorsOpenReview ↗

Response to Reviewer NxEV on FedHV’s convergence and efficiency

Thank you for the questions. The inverse-slack rule comes from Miranda and Von Zuben (2016). FedHV’s contribution is the delayed federated protocol that uses this rule and the analysis of the updates it produces.

Answer to Q1. The closed-form map gives an explicit link between loss reports and a fixed optimization target. Under positive true slacks, the ideal weights satisfy ∇Φ(θ)=c(θ)∑iwi⋆(θ)∇fi(θ)\nabla\Phi(\theta)=c(\theta)\sum_i w_i^\star(\theta)\nabla f_i(\theta), where Φ\Phi is the negative log-hypervolume objective. The stability of this map lets us turn loss-report error into a bound on the perturbation of the descent direction. Theorem 4.1 bounds log-hypervolume stationarity, and Proposition 4.3 relates it to Pareto stationarity. When the reporting term is O(T−1/2)O(T^{-1/2}), Corollary 4.2 gives the same order in rounds as FedCMOO’s Corollary 1.1, under different assumptions and for a different stationarity measure.

Answer to Q2. FedCMOO’s proof controls how a stochastic estimate of the task-gradient Gram matrix affects the projected weight update. Its theoretical variant uses unbiased compression, two sampled client sets to form the Gram estimate, and one projected weight step. The weight-update error depends on the projection step size and a second-moment bound on the Gram-based update direction.

FedHV’s proof must connect weights computed from noisy losses at the previous model to descent on Φ\Phi at the current model. The delay fixes wtw^t before the current clients are sampled, so Eq. (59) gives conditional unbiasedness for the frozen population gradient at the common model, before local drift. That gradient still has a direction error relative to ct−1∇Φ(θt)c_t^{-1}\nabla\Phi(\theta^t). Equation (13) bounds the weight mismatch using previous-round reporting error and inter-round changes in the true losses. The previous report and the current model can depend on the same previous cohort. The proof controls their effects through weight-map stability and model displacement, as in Eq. (77), and carries both terms into the descent inequality in Eq. (89). Under Corollary 4.2’s schedule, the staleness term is O(T−1)O(T^{-1}). Reporting error can persist even with exact local evaluations because a sampled cohort can differ from the population. The positive true-slack condition keeps the fixed objective well-defined at the points used in the proof.

Answer to Q3. The closed-form map reduces server computation by replacing projected weight updates with an O(m)O(m) calculation. In our FedCMOO experiments, the projected solver used 1,000 iterations per round. The communication savings come from replacing the model-order geometry message with mm loss estimates and carrying them alongside the model update. The one-round delay also removes the intermediate synchronization stage.

Compared with Uniform, the fixed-weight FedAvg baseline, FedHV retains one model exchange per round and adds mm scalars in each direction, local forward passes for the reports, and one-time calibration. Compared with FSMGDA, it replaces task-specific model updates with a single scalarized update. Compared with FedCMOO, it avoids explicitly constructing and compressing task-gradient Jacobians and forming the approximate Gram matrix. FedCMOO can reuse some of those gradients in its first local step.

The adaptive-round payloads in Table 13 imply about 49.4% lower uplink and 32.8% lower combined payload than FedCMOO under the same accounting. In the 300-round profile, FedHV’s wall-clock time was 24.7% lower than FedCMOO’s and approximately 32.8% lower than FSMGDA’s, including calibration. These are end-to-end measurements. Quantifying the runtime share of each component requires phase-level profiling.

The simpler protocol remained competitive in accuracy. At the matched round budgets in Table 1, FedHV exceeded both geometry-based baselines in mean accuracy in five of six settings. On CIFAR10–MNIST-B, worst-task accuracy was 57.55% for FedHV, 54.23% for FedCMOO, and 51.85% for FSMGDA.

Thank you for the detailed response. The clarification of the theoretical differences from FedCMOO and the additional efficiency results are helpful. However, my main concern regarding methodological novelty remains, as the core weighting mechanism is adopted from prior work, and the theoretical analysis does not clearly establish a substantial advantage over existing FedMOO methods. Therefore, I will maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.