ExLLM: Efficient Experience-Enhanced LLMs for Large-Scale Discrete Optimization

Large-scale discrete optimization involves vast, irregular search spaces where classical optimizers and learning-based methods struggle to incorporate expert knowledge and handle heterogeneous feedback under limited evaluation budgets. Recently, LLMs have been explored as optimizers, but existing approaches rely mainly on prompting or additional training, and lack mechanisms for scalable experience reuse. In particular, retrieval- or append-based memories accumulate redundancy, inflate prompts, and degrade exploration in long-horizon search. We introduce ExLLM (Experience-Enhanced LLM optimization), an LLM-as-optimizer framework designed for large discrete spaces. ExLLM integrates (i) a compact, evolving experience that distills non-redundant cues at low cost, (ii) a k-offspring sampling scheme that widens exploration per LLM call, and (iii) a lightweight feedback adapter that unifies objectives, constraints, and expert hints for iterative optimization. Starting from molecules, ExLLM achieves state-of-the-art performance on the PMO molecular optimization benchmark and generalizes strongly across domains, setting new records on circle packing and stellarator design, and delivering consistent gains on additional discrete and continuous optimization tasks. The framework transfers with only a task description and evaluation functions.