Generate and Select: A Framework for Bayesian Optimization with LLM Proposals

Optimizing structured objects, such as molecules and symbolic expressions, is difficult because the candidate domain is vast, often implicit, and contains many invalid or low-quality objects. Large language models (LLMs) can construct promising candidates, but generation alone does not allocate scarce black-box evaluations. We introduce generate-and-select (GaS), a modular framework that separates candidate generation from evaluation selection within an adaptive feedback loop. At each iteration, a black-box LLM generates $r$ candidates from previous observations, and Bayesian optimization (BO) with a Gaussian-process (GP) surrogate selects $q$ candidates from a cumulative pool. Assuming generation supplies improvement opportunities, we bound Bayesian simple regret relative to the best generator-reachable value in terms of generator improvement and cumulative missed improvements in selection. Expected regret vanishes exactly when average expected missed improvement vanishes, and a GP Thompson sampling (GP–TS) specialization depends on information gain over the generator-reachable domain. Across four structured and synthetic tasks, GaS matches or outperforms direct LLM optimizers and BO–LLM methods, with statistically significant gains over the strongest baseline on three. It achieves comparable or better performance than the closest BO–LLM baseline while using over two orders of magnitude fewer LLM tokens. Together, these results establish as a strong, modular, and efficient framework for BO with LLM-generated candidates.