- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes SICO, a fully first-order penalty method for nonconvex-strongly-convex stochastic bilevel optimization. Its main contribution is to combine projected coupling of the two lower-level trackers with an exponential-moving-average upper-level estimator. The resulting method is single-loop and uses a constant batch size, while attaining under bounded variance and under lower-level mean-square smoothness.
Critical strengths
The result addresses a meaningful gap in first-order stochastic bilevel optimization: prior methods attaining these -rates require either nested loops or -dependent batches. The algorithmic idea is simple and appealing, and the analysis of the coupled tracking error appears to be the main technical contribution. The experiments also include relevant first-order baselines and an ablation indicating that the projection is useful.
Critical weaknesses
I do not find any apparent weaknesses for the paper.
Questions for authors
What is the complexity in the deterministic setting? I wonder whether the method in this paper can remove the additional log factor in the existing upper bound.