BOLD: Learning Optimization Objectives via Bilevel Optimization and Symbolic Distillation

Many inverse problems are naturally formulated as optimization problems with stable solvers and interpretable objective functions. Recent deep learning approaches have mainly focused on learning optimizer parameters or replacing optimization pipelines with end-to-end black-box models, while the automatic discovery of optimization objectives remains comparatively underexplored. We propose BOLD, a framework for Bilevel Objective Learning and Symbolic Distillation. BOLD parameterizes optimization objectives as decomposable smooth components, trains them through finite-horizon bilevel optimization, and supports symbolic distillation of learned components into analytic expressions. We study three progressively broader instantiations: scalar-objective augmentation for finite-budget sparse recovery, learned local regularization for spring-system structure recovery, and full objective discovery for graph learning from smooth signals. Across these three settings, BOLD demonstrates complementary benefits of learning the objective itself: LP-ALISTA substantially improves finite-budget recovery over its ALISTA backbone; the learned spring-system regularizer can be distilled into an analytic penalty while largely preserving the neural teacher's support-recovery performance; and the learned graph objectives match or outperform classical Kalofolias-style baselines in most representative settings while revealing interpretable signal-distance and degree-shaping structure. Together, these results show that objective learning can improve downstream recovery while preserving an explicit optimization formulation and enabling symbolic interpretation.