- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose Fed-MOZOO, a federated method for multi-objective online convex optimization with intermittent communication and zeroth-order feedback. Clients scalarize loss vectors via preference vectors, compute two-point zeroth-order gradient estimates using shared random directions, and take projected local steps while the server aggregates models every steps. For convex Lipschitz losses under an oblivious adversary, they derive a global-model regret bound of , which recovers the standard rate under full communication. They further establish Pareto optimality and approximate non-domination for the scalarized comparator. Experiments on synthetic quadratic streams and convex classification tasks (Adult, MNIST) evaluate regret, trade-offs, and Pareto fronts.
Critical strengths
-
The analysis distinguishes local-play regret from global-model regret. It then bounds the gap between them with an explicit consensus-drift term that vanishes when clients communicate every round. The proofs, including the dimension-linear second-moment bound for the two-point estimator, are correct, and each error term is easy to trace to its source: estimation, smoothing, or drift.
-
The paper states plainly that its guarantee is preference-conditioned, assumes an oblivious adversary, and gives no -dependent speedup. It also explains why, and it does not overstate what the result implies about each objective individually. This candor makes the theoretical contribution easy to evaluate.
Critical weaknesses
The contribution seems somewhat limited. With the preference vector fixed, the problem appears to reduce to single-objective federated bandit optimization, and the method is close to FedAvg with a two-point estimator, which Patel et al. (2023) already study with gains in the number of clients. The bound here does not depend on , and a synchronized minibatch scheme may achieve a better bound. Some experimental results are also hard to reconcile with the algorithm. At , I would expect zero consensus error and identical results for Fed-MOZOO and Central-MOZO, but the reported results seem to show otherwise. The zeroth-order method also appears to beat the exact-gradient baselines. Since the experimental settings actually used are not clearly reported, I find the empirical claims difficult to assess.
Questions for authors
Clarification
-
How does the bound compare with that of Patel et al. (2023), who analyze FedAvg with two-point feedback in the same intermittent-communication setting? Could a synchronized minibatch scheme, in which all clients query the same model and their estimates are averaged at each communication, achieve a better bound with the same queries and communication? If so, what advantage do local updates offer here?
-
With , Algorithm 1 synchronizes after every step, so I would expect zero consensus error and identical results for Fed-MOZOO and Central-MOZO. Could you explain why the reported results seem to show nonzero consensus error and a gap between the two methods? Is Central-MOZO defined differently from Fed-MOZOO with ?
-
Fed-MOZOO appears to outperform Fed-MOGD, which uses exact gradients under the same federation. Were the step sizes tuned separately for each method? What explains this ordering, given Theorem 1?
More
- A guarantee that goes beyond preference-conditioned regret, such as Pareto regret or tracking of time-varying preferences.
- An analysis that shows explicit gains in the number of clients .
- Extensions to noisy function evaluations and nonconvex objectives.