Meta-Delta: Evolving Knowledge through Global Review for LLM-Driven Optimization

LLM-driven evolutionary optimization accumulates experience about which changes improve a candidate solution. As search progresses, this experience must be interpreted in context: lessons may have limited scope, accumulated guidance may conflict, and repeated refinement may leave alternatives unexplored. We introduce **Meta-Delta**, a framework that jointly evolves candidate solutions and the knowledge guiding their generation. A **Delta Bank** stores structured **Delta Cards** with search lessons, applicability conditions, and supporting evidence. Routine local updates incorporate what each experiment teaches, while Global Review examines the Bank and broader search history to reconsider existing guidance and propose directions for subsequent testing. The updated Bank guides new experiments, whose outcomes further refine its knowledge. Empirically, Meta-Delta achieves the best or tied-best result on 10 of 11 tasks across mathematical, scientific, and systems optimization. On scientific law discovery, it achieves the highest mean numeric-fit score on SCILAWS-REAL, ties for the highest scientific-validity score at displayed precision, and achieves the highest mean structure score on SCILAWS-PARALLEL. Global Review ablations provide evidence that broader knowledge revision helps sustain progress, and case studies show how revised knowledge changes subsequent search and the structure of discovered scientific laws. These results support treating search knowledge as an evolving component of long-horizon optimization.