- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes EVADE, an LLM-guided evolutionary framework that estimates whether a behavioral region can still generate improved or new offspring. The estimated evolvability controls search modes, parent selection, inspiration programs, and experience retention. Experiments on 11 tasks with two LLM backbones show strong performance.
Critical strengths
-
Evolvability shows a meaningful alternative to selecting programs only by fitness.
-
EVADE shows strong performance across 11 tasks.
Critical weaknesses
- The proposed signal is not convincingly validated as evolvability. Since the main experiments use scalar fitness as the behavior descriptor, it largely captures score transitions and recent mutation success.
- EVADE is considerably over-designed. Its performance depends on numerous manually specified modes, weights, clustering rules, lineage operations, complexity preferences, and memory parameters, while the limited ablations change multiple factors simultaneously.
- The experimental comparison is not fair or reproducible. LLM calls for experience distillation are excluded from the budget.
Questions for authors
-
Does estimated evolvability predict future offspring improvement after controlling for parent fitness, visit count, mutation mode, and prompt context?
-
Why is scalar fitness an adequate behavioral descriptor?
-
Can you provide an ablation that changes only the evolvability term while keeping cells, prompts, parent complexity, inspiration programs, and memory rules fixed?
-
What principles determined the search modes, weights, clustering threshold, smoothing parameters, and experience-retention rules, and were any tasks used to tune them?
-
How many LLM calls, tokens, wall-clock hours, and monetary cost does each method use?
8 comments
Response to Reviewer QHt1 [Part 2/4]
[W1] Since the main experiments use scalar fitness as the behavior descriptor, it largely captures score transitions and recent mutation success.
[Q2] Why is scalar fitness an adequate behavioral descriptor?
A behavior descriptor for estimating evolvability should (i) separate programs that behave differently, (ii) be aligned with the objective, so that reaching a new behavior is a change that matters for the task, and (iii) be cheap to compute and available for every task. We use the combined score because it satisfies (ii) and (iii) on all 11 tasks without task-specific engineering (§4); it satisfies (i) only coarsely, which we acknowledge. The combined score is one choice of descriptor: the framework does not depend on it, and the descriptor can be replaced by a richer one without modifying the method (§4).
To illustrate, we compare two richer descriptors built from the sub-scores that make up the combined score (Prism: 1/max_kvpr on each test instance; Signal Processing: slope changes, lag error, average error, false reversals, correlation, and noise reduction). The sub-score vector of Table 5 uses the sub-scores directly. The two-level descriptor first groups programs by combined score as in the default method, and then splits a group with at least two programs by clustering on the sub-scores, with the same 25th-percentile threshold.
| Descriptor (DeepSeek-V4-Flash, 3 runs) | Prism ↑ | Signal Processing ↑ |
|---|---|---|
| Combined score | 26.50 ± 0.16 | 0.713 ± 0.029 |
| Sub-score vector (Table 5) | 26.33 ± 0.10 | 0.707 ± 0.075 |
| Two-level | 26.57 ± 0.10 | 0.733 ± 0.018 |
All three descriptors give comparable results, and the two-level descriptor attains the highest mean fitness on both tasks.
[W2] EVADE is considerably over-designed. Its performance depends on numerous manually specified modes, weights, clustering rules, lineage operations, complexity preferences, and memory parameters,
EVADE centers on two main contributions: (1) behavioral-clustering-based evolvability, and (2) its integration into the general LLM-EC framework. Mode selection, parent and inspiration selection, and experience are standard components of existing LLM-EC frameworks such as AdaEvolve and ShinkaEvolve; EVADE does not add new components, but drives all of them with a single quantity, the evolvability estimate. In our initial ablation study, we replaced evolvability with alternative strategies in each of these components (parent selection, exploration–exploitation, and experience). Nevertheless, we agree that more ablation studies would better show the value of evolvability as the control signal; we present such new evidence below.
[W2] while the limited ablations change multiple factors simultaneously.
[Q3] Can you provide an ablation that changes only the evolvability term while keeping cells, prompts, parent complexity, inspiration programs, and memory rules fixed?
Thank you for this suggestion. In each variant, we replace or remove only the evolvability estimate, keeping cells, prompts, parent complexity, inspiration programs, and memory rules fixed (DeepSeek-V4-Flash, 3 runs):
- Cell selection. Evolvability is removed from cell selection (Eq. 7): cells are selected uniformly, or by fitness only ().
- Signal. The signal (Eq. 1) is replaced, with the cell estimator unchanged: global behavioral novelty, where a child is credited for a new behavior only if the behavior is new to the whole archive rather than to its parent's lineage; historical improvement, if the child surpasses the best fitness and (Eq. 2) otherwise; and mutation success rate, .
| Variant | Prism ↑ | Signal Processing ↑ |
|---|---|---|
| EVADE | 26.50 ± 0.16 | 0.713 ± 0.029 |
| Cell selection | ||
| Uniform | 26.26 ± 0.00 | 0.670 ± 0.039 |
| Fitness only | 26.26 ± 0.00 | 0.661 ± 0.035 |
| Signal | ||
| Global behavioral novelty | 26.33 ± 0.10 | 0.656 ± 0.023 |
| Historical improvement | 26.34 ± 0.07 | 0.698 ± 0.027 |
| Mutation success rate | 26.28 ± 0.03 | 0.657 ± 0.009 |
Every variant scores lower than EVADE on both tasks. If the gains came from the surrounding design, these variants, which keep all of it, would retain them; they do not.
Response to Reviewer QHt1 [Part 3/4]
[Q4] What principles determined the search modes, weights, clustering threshold, smoothing parameters, and experience-retention rules, and were any tasks used to tune them?
The mode anchors are fixed and evenly spaced, and the mode weights are fixed across all tasks and backbones (Appendix A.2). The clustering threshold is not a fixed constant: it is set adaptively to the 25th percentile of pairwise distances (§3.5). The smoothing parameters and and the weight were chosen from the sensitivity analysis in Table 5, run on Prism and Signal Processing, and the selected values were applied unchanged to all 11 tasks and both backbones.
We further report the sensitivity to the clustering percentile, with all other components fixed (DeepSeek-V4-Flash, 3 runs).
| Percentile | Prism ↑ | Signal Processing ↑ |
|---|---|---|
| 10 | 26.32 ± 0.11 | 0.682 ± 0.007 |
| 15 | 26.38 ± 0.14 | 0.673 ± 0.040 |
| 25 (default) | 26.50 ± 0.16 | 0.713 ± 0.029 |
| 40 | 26.27 ± 0.02 | 0.703 ± 0.008 |
| 50 | 26.27 ± 0.03 | 0.704 ± 0.034 |
The default percentile attains the highest mean on both tasks, and performance varies moderately across the other settings.
[W3] The experimental comparison is not fair or reproducible. LLM calls for experience distillation are excluded from the budget.
We follow the budget convention of prior work on this benchmark suite (Cemri et al., 2026; Liu et al., 2026a): all methods share a budget of iterations (§4), where each iteration consists of one generation call, which produces a candidate program, and one evaluation (§3.4). LLM calls made for other purposes, which we refer to as auxiliary calls, are not counted, for EVADE and the baselines alike. Such calls are common in existing frameworks: AdaEvolve uses them to generate meta-guidance when the search stagnates, EvoX to generate new search strategies when progress stalls, and ShinkaEvolve to query a novelty judge and generate meta-recommendations. GEPA, as implemented in SkyDiscover (Liu et al., 2026b), combines reflection and generation in a single prompt and makes no auxiliary calls.
Response to Reviewer QHt1 [Part 1/4]
We sincerely thank Reviewer QHt1 for the thoughtful review. We are glad that the reviewer found that evolvability "shows a meaningful alternative to selecting programs only by fitness" and that EVADE "shows strong performance across 11 tasks". Below we respond to the reviewer's comments.
In this paper, we define evolvability as a candidate's capacity to generate offspring that either improve upon the best fitness found so far or exhibit behaviors distinct from their ancestors, following prior work that operationalizes evolvability through offspring outcomes (Wagner & Altenberg, 1996; Mengistu et al., 2016; Gajewski et al., 2019).
While this behavioral criterion can be viewed as capturing lineage-relative novelty, we treat it as evidence of evolvability because it reflects a parent's capacity to unlock previously inaccessible regions of the behavioral space. The intuition aligns with biological evolution: flight evolved independently in both birds and mammals (bats). Although flight was not globally novel when bats emerged, acquiring it from a non-flying mammalian ancestor required a lineage capable of generating viable, functionally distinct phenotypes. We verify empirically below that this estimate predicts future improvement, and isolating the effect of evolvability is presented in our answer to Q1 below.
Thank you for this insightful question. To verify whether estimated evolvability predicts future improvement, we compare it with other selection signals on the same search states.
Mutation mode and prompt context are held fixed: all mutations use the same prompt, containing only the parent and its metrics, with no inspiration programs or experience. Parent fitness and visit count are not fixed, but selecting by fitness yields the lowest success rate, so the advantage of evolvability is not explained by choosing fitter parents.
Each entry is the number of successful parents, out of 3 per iteration and 9 in total.
Prism
Signal Processing
Evolvability attains the highest total in every case, tied only with fitness + diversity under Sequential on Prism. Pooled over tasks and settings, its selected parents succeed in 26 of 36 cases (72%), against 5/36 for fitness, 14/36 for fitness + diversity, and 18/36 for random selection. Its success rate does not decline at later iterations, when the global best is higher, so the estimate remains predictive as the search progresses.