- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The heterogeneous federated bilevel optimization with constrained LL problems and nonsmooth, nonconvex UL objectives is studied in this manuscript. There are two considered federated settings. One is GFBO where clients share a global LL solution. Another one is PFBO where each client has its own LL solution. The authors propose a unified zeroth-order framework, SCAFFOLD-RZO_bl to solve the problem. An adaptive penalty SCAFFOLD method is also developed for solving constrained LL problems. Theoretical results including convergence guarantees and communication complexity bounds are presented. Simulation results on CIFAR-10 hyperparameter optimization and MNIST personalized federated learning show improvements over selected baselines.
Critical strengths
The authors propose a unified zeroth-order framework for constrained federated bilevel optimization. Detailed theoretical results and simulation results are given. Overall, this paper is well written.
Critical weaknesses
- The comparisons with the existing federated bilevel optimization methods, e.g., some existing methods in Table 1, in terms of the complexity of computational, communication costs and convergence rate, are not given. Readers may care about these complexities and convergence performance. In simulation results, the computational costs, communication rounds, and running time compared with the existing methods are not provided either.
- Theorems 3.3 and 3.5 present convergence guarantees. However, the stationarity gap is not reported in the experiment section. Furthermore, in the experiments, do the parameters meet the requirements for convergence, and how should this be determined in practical applications? It is better to provide specific numerical calculations and discussions.
Questions for authors
- Please provide comparisons with existing federated bilevel optimization methods regarding computational complexity, communication costs, and convergence rate.
- Please provide simulation results on the computational costs, communication rounds, and running time compared with the existing methods.
- Are the assumptions too strict? Are these assumptions needed in this paper commonly used in federated bilevel optimization and zeroth-order optimization?
- Please provide results on the stationarity gap in the experiment section.
- Please provide specific numerical results on whether the parameters considered in the experiments meet the requirements for convergence.
- I am not sure the experiments of the proposed methods and the baseline are fair. Can the authors clarify this in detail?
- Can the proposed methods work in more complex problems? For example, deep neural networks. It would be better to add this experiment.