- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes GaCF, a single-loop framework for constrained bilevel optimization with potentially nonconvex lower-level objectives. The method constructs a continuation path from a strongly convex auxiliary potential to the original lower-level objective, and uses the same potential to define Bregman smoothing and mirror-descent updates. An auxiliary variable tracks the smoothed inner minimizer, while primal-dual updates handle the relaxed lower-level optimality constraint. The paper establishes a non-asymptotic stationarity guarantee for the resulting regularized envelope, with additional conditional results for the relaxed problem’s KKT residual. Experiments over synthetic problems, data hypercleaning, few-shot learning, and fine-tuning demonstrate the method's effectiveness compared with baselines.
Critical strengths
1.The methodological design is coherent and well motivated. Combining a continuation path, its smoothing geometry, and update geometry provides a useful extension of Euclidean envelope methods. The framework accommodates both simplex geometry and anisotropic scaling while avoiding second-order derivatives of the lower-level objective.
-
The convergence analysis is concrete. The appendix analyzes inner tracking, dual tracking, and changing continuation parameters within a common Lyapunov argument. Obtaining a stationarity guarantee without lower-level convexity or a PL condition is meaningful, subject to the scope qualifications below.
-
The experiments provide reasonable breadth and relevant comparisons. The evaluation includes several representative learning applications and compares against multiple baselines. Reporting wall-clock results alongside predictive performance helps assess practical efficiency. Ablation study in Appendix is also informative. They compare continuation only, geometry only, their combination, and a Euclidean baseline without continuation under matched initialization and iteration budgets. The combined method performs best across the reported metrics, supporting complementary benefits from the two components.
Critical weaknesses
Limited coverage of stochastic optimization. The convergence analysis is restricted to deterministic mirror descent using exact gradients of the upper- and lower-level objectives. The authors should clarify whether the learning experiments use stochastic gradients and discuss the main technical obstacles to extending the analysis to stochastic mirror descent (SMD). In particular, how would sampling noise interact with the inner-solution tracking, multiplier updates, and changing continuation parameters? Explaining which parts of the current proof would require modification, and whether additional assumptions or sampling strategies would be needed, would clarify the framework’s applicability to large-scale learning.
Applicability of the entropy-based convergence guarantee. Proposition 3 extends the analysis to negative-entropy kernels under the compact-interior condition in Eq. (9-35), which requires the iterates to remain uniformly separated from the simplex boundary. Although entropy mirror updates preserve strict positivity, this alone does not establish such a uniform bound . Could the authors explain under what conditions this assumption holds, or how it is ensured in the entropy-based implementations? Clarifying whether trajectories that approach the simplex boundary can be accommodated would help readers understand the scope of the guarantee.
Questions for authors
See weakness section.