acceptodds
Under review as a conference paper at ICLR 2027

LEVI: Stronger Search Architectures Can Substitute for Larger LLMs in Evolutionary Search

Abstract

LLM-guided evolutionary methods such as AlphaEvolve have proven effective in domains like math, systems research, and algorithmic discovery, but their reliance on frontier models makes each run expensive. We argue this is largely an artifact of how existing frameworks allocate search: archives that fail to preserve solution diversity force compensation through stronger mutation models; blind model use spends frontier dollars on local edits a smaller model could handle; and full-set evaluation wastes rollouts on redundant examples. We introduce LEVI, a harness-first evolutionary framework built on the bet that stronger search architectures can substitute for or even outperform larger LLMs in evolutionary search. LEVI improves on three core components of evolutionary search: a solution database that establishes diversity from the beginning, and then maintains it throughout the run; a smarter mutation router that plays into the strengths of large and small LLMs; and a rank-preserving proxy benchmark for rollout-heavy settings. Across six systems-research benchmarks LEVI attains the highest score on a budget 3.3–6.7× smaller than the published frontier-model runs of existing frameworks like ShinkaEvolve, GEPA, and AdaEvolve; on one problem, LEVI matches the existing best at 35× lower cost. On prompt optimization, LEVI matches or exceeds GEPA at less than half of its rollout budget on four different benchmarks. LEVI will be full open sourced upon acceptance.

Primary area:
optimization
Keywords:
AlphaEvolve, FunSearch, Evolutionary Search, LLMs

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UcQ1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LEVI, a framework for making LLM-based evolutionary search more effective and less expensive. It combines a diverse solution archive based on CVT-MAP-Elites, a routing strategy that uses smaller models for routine edits and stronger models for occasional larger changes, and a proxy benchmark for reducing evaluation costs in prompt optimization. The authors evaluate LEVI on system optimization and prompt optimization tasks. The results are promising: LEVI performs well against several existing search frameworks under matched settings and achieves similar aggregate prompt-optimization performance to GEPA with fewer rollouts.

Critical strengths

The paper addresses an important problem in LLM-based evolutionary search: how to improve the quality of discovered solutions without relying on expensive models for every iteration. LEVI combines a diverse solution archive, different LLMs for local refinement and larger changes, and a proxy benchmark for reducing evaluation cost. The controlled experiments on system optimization show that these choices can improve search performance under a matched model and evaluation budget. The evaluation also covers both program optimization and prompt optimization, with ablations of the main components. The results are not uniformly better on every task, but they provide useful evidence that search architecture and model allocation can affect both solution quality and cost. This makes the work relevant beyond simply proposing another LLM-based search system.

Critical weaknesses
  1. The contribution of LEVI beyond existing methods is not fully established. LEVI combines a CVT-MAP-Elites archive, role-based LLM selection, and proxy-based evaluation. However, the underlying ideas have relevant precedents. MAP-Elites is an established approach for maintaining diverse, high-quality solutions, while ShinkaEvolve (ICLR 2026) explores novelty-aware search and bandit-based LLM selection, and AdaptEvolve (Findings of ACL 2026) studies adaptive model selection during evolutionary refinement. Although LEVI compares against several existing frameworks, it remains unclear what additional benefit its particular combination of components provides over the closest existing methods. The concern is not that LEVI builds on existing ideas, but that the paper does not sufficiently establish the contribution of combining them in this way.
  2. The evidence for LEVI's efficiency and robustness is not equally strong across all experiments. The matched-model and matched-budget experiments provide useful evidence for the effectiveness of the framework, but some broader comparisons rely on results reported in other papers under potentially different models and evaluation settings. These results provide useful context, but do not offer the same level of evidence as controlled comparisons. In addition, reducing LLM API costs does not necessarily reduce total computational cost, and the paper's own analysis shows that evaluator costs and wall-clock time can reduce the apparent savings. Finally, some results are based on a single run, making it difficult to assess their sensitivity to random initialization. These limitations leave uncertainty about how consistently LEVI delivers its claimed performance and efficiency gains in practice.
Questions for authors

Q1. What does LEVI contribute beyond combining existing ideas in quality-diversity search, LLM selection, and proxy-based evaluation? Since the paper already compares against several existing frameworks, could the authors clarify which specific design choices account for the gains over these baselines? Q2. How much does the proposed routing strategy contribute to the final results? In particular, how does it compare with using only the smaller model, only the larger model, or an adaptive routing method such as AdaptEvolve under comparable model and evaluation budgets? Q3. The paper reports substantial savings in LLM API costs, but evaluator computation and wall-clock time reduce some of these savings. Could the authors clarify how LEVI compares with existing methods when total computational cost is considered, and in which settings the overall cost advantage remains substantial? Q4. The proxy benchmark is constructed using an initial set of candidates and then used to guide subsequent search. How well does it preserve the ranking of candidates generated later in evolution? Could the authors provide further evidence on later-stage candidates and clarify how the validation and test splits are kept separate from proxy construction and final candidate selection?

Reviewer_Sc6AOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces LEVI, a cost-efficient framework for LLM-guided evolutionary search. It combines three components: (1) diverse initialization and a CVT-MAP-Elites archive to preserve solution diversity, (2) role-aware routing that assigns local refinements to smaller models and paradigm shifts to larger models, and (3) rank-preserving proxy benchmarks to reduce evaluation costs. Experiments on systems-research and prompt-optimization tasks demonstrate competitive or improved performance with substantially lower LLM costs.

Critical strengths

1. Practical and relevant motivation.

The paper investigates an important question of whether better search architectures can reduce reliance on expensive frontier models. Its focus on improving cost efficiency through harness design is practically valuable.

2. Strong empirical performance.

The controlled comparisons using identical models and evaluation budgets show consistent improvements over OpenEvolve, ShinkaEvolve, and GEPA across four systems-research tasks. The prompt-optimization experiments also demonstrate substantial rollout savings.

3. Informative ablation analysis.

The results provide useful insights into the task-dependent effectiveness of diverse initialization, behavioral descriptors, and large-model paradigm shifts. In particular, the observation that larger models are not always beneficial is interesting.

Critical weaknesses

1. Limited methodological novelty relative to existing approaches

The proposed framework primarily integrates established techniques, including CVT-MAP-Elites, heterogeneous model allocation, and representative subset selection. Multi-model collaboration and adaptive model selection have already been explored in AlphaEvolve, ShinkaEvolve, and PACEvolve++, while broader cost-aware routing methods such as FrugalGPT and RouteLLM investigate model allocation based on expected quality and cost. In particular, ShinkaEvolve already incorporates bandit-based model selection. The paper does not sufficiently clarify the incremental contribution of its fixed role-aware routing strategy over these existing approaches.

2. Insufficient isolation of the proposed architectural components

The ablations do not fully disentangle the contributions of several key design choices. Removing bootstrapped initialization also changes archive calibration, making it difficult to isolate their individual effects. The paper also lacks comparisons between CVT-MAP-Elites and simpler archive alternatives under identical initialization conditions. Similarly, the no-role-aware routing variant disables the dedicated paradigm-shift operator, confounding model allocation with search strategy. Consequently, the individual contributions of these components remain unclear.

3. Insufficient evidence that stronger search architectures can substitute for larger models

The paper lacks a controlled large-model-only baseline under the same LEVI architecture. While the matched-model comparisons demonstrate the effectiveness of the overall search framework, comparisons against published frontier-model baselines involve different models, search configurations, and budgets. Therefore, the results do not establish whether replacing larger models with smaller ones preserves comparable solution quality. Moreover, the benefits of larger models may depend on task difficulty and search state rather than predefined mutation roles. Systematic comparisons across model capabilities and routing strategies are needed to substantiate the central claim.

4. Limited generality of the cost-efficiency claims

The framework primarily targets reducing LLM API costs, but these may not be the dominant computational bottleneck in many practical optimization tasks. For example, automated ML research, architecture search, and simulation-based optimization can involve expensive evaluations that dominate the overall search cost. Although the paper accounts for evaluator compute in its systems benchmarks, the reported cost advantages are smaller when these costs are included, and wall-clock efficiency does not improve. The benefits of trading expensive model calls for additional evaluations may therefore be substantially reduced in evaluation-intensive settings. Further experiments across different evaluation-cost regimes would strengthen the generality of the efficiency claims.

5. Limited validation of proxy benchmark generalization

The proxy benchmark is constructed using initial candidates, but its ranking fidelity may deteriorate as evolution discovers new solution families. Although the appendix provides some full-set verification of evolved candidates, the evidence remains limited, and most prompt-optimization results rely on single runs. Further evaluation of ranking consistency throughout evolution would strengthen the robustness of this component.

Questions for authors
  1. Can the authors separately ablate diverse initialization, archive calibration, and CVT-MAP-Elites, including comparisons with simpler archive structures under identical conditions?

  2. How does LEVI perform with large-only, small-only, and mixed-model configurations under the same search architecture? Can the authors demonstrate that smaller models preserve comparable solution quality when replacing larger models?

  3. What distinguishes LEVI's role-aware routing from existing heterogeneous or adaptive model selection approaches, such as ShinkaEvolve's bandit-based selection? How would it compare with difficulty-aware or cost-aware routing under matched budgets?

  4. How does LEVI's efficiency change when evaluation costs dominate LLM inference costs? Can the authors evaluate its performance under different ratios of mutation cost to evaluation cost?

  5. How reliably does the proxy benchmark preserve rankings for newly evolved candidates throughout the search process?

Reviewer_Le7gOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

According to the paper, LLM-guided evolutionary methods such as AlphaEvolve have proven effective in domains like math, systems research, and algorithmic discovery, but their reliance on frontier models makes each run expensive. We argue this is largely an artifact of how existing frameworks allocate search: archives that fail to preserve solution diversity force compensation through stronger mutation models; blind model use spends frontier dollars on local edits a smaller model could handle; and full-set evaluation wastes rollouts on redundant examples. The authors introduce LEVI, a harness-first evolutionary framework built on the bet that stronger search architectures can substitute for or even outperform larger LLMs in evolutionary search. LEVI improves on three core components of evolutionary search: a solution database that establishes diversity from the beginning, and then maintains it throughout the run; a smarter mutation router that plays into the strengths of large and small LLMs; and a rank-preserving proxy benchmark for rollout-heavy settings.

Across seven systems-research benchmarks LEVI attains the highest mean score on five on a budget 3.3–6.7×smaller than the published frontier-model runs of existing frameworks like ShinkaEvolve, GEPA, and AdaEvolve; on one problem, LEVI matches the existing best at a 35 × lower cost.

Critical strengths
  • LLM-guided evolution is expensive. LEVI matches or beats published frontier-model runs on most ADRS tasks while spending dollar 4.50 of LLM calls per problem instead of dollar 15–30.
  • The ablations explain where model scale is worth paying for. Bootstrapped diverse seeds are the largest single factor.
Critical weaknesses
  • Table 1 compares LEVI against numbers copied from other papers, and those runs differ in search effort as well as model.
  • Most margins are also within noise and some are only single runs
  • The gains may come mostly from seeding.
  • The paper should also discuss prior work on routing between cheap and strong models inside evolutionary or test-time-scaling loops (e.g., AlphaEvolve’s Flash/Pro ensemble, ShinkaEvolve’s adaptive selection, Squeeze Evolve).
Questions for authors
  1. How many evaluations did each LEVI run and each baseline run use in Table 1?
  2. Can you add AdaEvolve and EvoX to the controlled comparison?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.