Reshaping Guidance Objective with Large Language Models

Optimization problems typically have well-defined task objectives, yet the guidance objectives used to steer optimization algorithms toward high-quality solutions are often hand-designed or manually tuned and may be suboptimal. In this paper, we study *guidance objective reshaping* with large language models (LLMs), aiming to improve optimization outcomes while accounting for structural–numerical coupling and objective alignment, and reducing reliance on manual human effort. To this end, we formulate guidance objective reshaping as a structural-numerical optimization problem and propose *Joint Objective Reshaping (JOR)*. Specifically, JOR combines LLM-driven structural adaptation with zeroth-order numerical optimization over a reparameterized latent space, while aligning the reshaped objective by improving downstream performance and proximity to the high-quality solution region. This design enables structural–numerical modifications to translate more reliably into improvements under the task objective. Extensive experiments across combinatorial optimization, LLM inference acceleration, Bayesian optimization, and reinforcement learning reward shaping show that the guidance objectives discovered by JOR consistently steer algorithms or solvers toward better solutions across diverse settings. Further analysis demonstrates that the reshaped guidance objectives primarily improve the optimization landscape without substantially altering the high-quality solution region under the task objective. Our work establishes guidance objective reshaping as a promising paradigm for LLM-assisted search, learning, and optimization.