- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LEVI, a framework for making LLM-based evolutionary search more effective and less expensive. It combines a diverse solution archive based on CVT-MAP-Elites, a routing strategy that uses smaller models for routine edits and stronger models for occasional larger changes, and a proxy benchmark for reducing evaluation costs in prompt optimization. The authors evaluate LEVI on system optimization and prompt optimization tasks. The results are promising: LEVI performs well against several existing search frameworks under matched settings and achieves similar aggregate prompt-optimization performance to GEPA with fewer rollouts.
Critical strengths
The paper addresses an important problem in LLM-based evolutionary search: how to improve the quality of discovered solutions without relying on expensive models for every iteration. LEVI combines a diverse solution archive, different LLMs for local refinement and larger changes, and a proxy benchmark for reducing evaluation cost. The controlled experiments on system optimization show that these choices can improve search performance under a matched model and evaluation budget. The evaluation also covers both program optimization and prompt optimization, with ablations of the main components. The results are not uniformly better on every task, but they provide useful evidence that search architecture and model allocation can affect both solution quality and cost. This makes the work relevant beyond simply proposing another LLM-based search system.
Critical weaknesses
- The contribution of LEVI beyond existing methods is not fully established. LEVI combines a CVT-MAP-Elites archive, role-based LLM selection, and proxy-based evaluation. However, the underlying ideas have relevant precedents. MAP-Elites is an established approach for maintaining diverse, high-quality solutions, while ShinkaEvolve (ICLR 2026) explores novelty-aware search and bandit-based LLM selection, and AdaptEvolve (Findings of ACL 2026) studies adaptive model selection during evolutionary refinement. Although LEVI compares against several existing frameworks, it remains unclear what additional benefit its particular combination of components provides over the closest existing methods. The concern is not that LEVI builds on existing ideas, but that the paper does not sufficiently establish the contribution of combining them in this way.
- The evidence for LEVI's efficiency and robustness is not equally strong across all experiments. The matched-model and matched-budget experiments provide useful evidence for the effectiveness of the framework, but some broader comparisons rely on results reported in other papers under potentially different models and evaluation settings. These results provide useful context, but do not offer the same level of evidence as controlled comparisons. In addition, reducing LLM API costs does not necessarily reduce total computational cost, and the paper's own analysis shows that evaluator costs and wall-clock time can reduce the apparent savings. Finally, some results are based on a single run, making it difficult to assess their sensitivity to random initialization. These limitations leave uncertainty about how consistently LEVI delivers its claimed performance and efficiency gains in practice.
Questions for authors
Q1. What does LEVI contribute beyond combining existing ideas in quality-diversity search, LLM selection, and proxy-based evaluation? Since the paper already compares against several existing frameworks, could the authors clarify which specific design choices account for the gains over these baselines? Q2. How much does the proposed routing strategy contribute to the final results? In particular, how does it compare with using only the smaller model, only the larger model, or an adaptive routing method such as AdaptEvolve under comparable model and evaluation budgets? Q3. The paper reports substantial savings in LLM API costs, but evaluator computation and wall-clock time reduce some of these savings. Could the authors clarify how LEVI compares with existing methods when total computational cost is considered, and in which settings the overall cost advantage remains substantial? Q4. The proxy benchmark is constructed using an initial set of candidates and then used to guide subsequent search. How well does it preserve the ranking of candidates generated later in evolution? Could the authors provide further evidence on later-stage candidates and clarify how the validation and test splits are kept separate from proxy construction and final candidate selection?