Sharpness-Aware Bilevel and Minimax Optimization

Bilevel and minimax optimization are foundational frameworks of nested optimization widely used in hyperparameter tuning, adversarial training, meta-learning, and reinforcement learning. Despite their versatility, these methods frequently suffer from poor generalization due to sharp minima and training instability, especially in overparameterized regimes. We introduce a unified first–order sharpness-aware optimization framework comprising Sharpness-Aware Bilevel Optimization (SABO) and Sharpness-Aware Minimax Optimization (SAMO). SABO improves generalization by jointly minimizing inner and outer objectives and their sharpness via a penalty-based reformulation that avoids costly implicit gradient computations. SAMO also yields a simplified algorithm tailored for sharpness-aware minimax settings. We establish theoretical convergence guarantees under the Polyak–ojasiewicz condition and derive generalization bounds. Experiments on synthetic and real tasks show that SABO and SAMO consistently enhance generalization over standard baselines.