From Easy to Hard: Geometry-Aware Continuation for Nonconvex Bilevel Optimization

Nonconvex bilevel optimization (BLO) is challenging due to its nested structure and potentially nonunique lower-level (LL) solutions. Value-function reformulations avoid explicit LL solution maps by recasting BLO as a single-level surrogate, but LL nonconvexity often makes the induced value function nonsmooth and difficult to optimize. Existing smoothed surrogates, such as Moreau-envelope-based reformulations, improve tractability by obtaining regularity through strong Euclidean regularization; however, when this regularization dominates the surrogate, the surrogate may become overly localized and mismatched with the anisotropic or non-Euclidean structure of constrained BLO. To overcome these issues, we propose GaCF, a single-loop geometry-aware continuation framework for nonconvex BLO. Instead of relying on a fixed Euclidean smoothing geometry, GaCF constructs a continuation path that starts from a well-conditioned auxiliary LL surrogate and progressively recovers the original nonconvex LL objective, thereby reducing early-stage dependence on rigid local surrogates. Along this path, GaCF uses the induced Bregman geometry to construct a Bregman-smoothed value-function surrogate and perform mirror-descent updates, so that the surrogate and update dynamics are aligned with a problem-adapted geometry. We establish a non-asymptotic convergence guarantee for GaCF under general nonconvex settings, and experiments on synthetic and real-world learning tasks demonstrate its convergence efficiency and predictive performance over existing BLO methods.