acceptodds
Under review as a conference paper at ICLR 2027

Component-Aware Feedback for Self-Evolving Programs

Abstract

LLM-guided evolutionary search can discover complex programs, but existing methods mostly only save candidate programs and fitness scores while discarding which component edits produced which fitness metric changes. Existing methods force the mutator LLM to infer the effect of prior edits from cluttered histories, making program search slow and unstable. This is especially true for locally servable LLMs to evolve multi-component systems. We introduce component-aware feedback, which compares each evaluated program with its parent, identifies the components that changed, and logs them with the associated metric differences into an attribution memory that later mutations read. The memory keeps each change in two reference frames, local against the parent it came from and global against the seed program, which shows both the immediate effect of a change and the cumulative progress made since the seed. We study this on LLM reranking, a multi-objective optimization problem where a multi-stage pipeline must balance quality against serving cost. Across twelve Bright datasets, our method reaches the strongest baseline's final quality after a median of one third of the search budget and ends 7.2% higher in held-out nDCG@10, and under a cost-aware objective it finds pipelines that are on average more accurate while using 11% fewer tokens per query, showing component-aware feedback to be a promising direction for more efficient self-evolving systems.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM Reranking, Evolutionary Search

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_6EZrOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes component-aware feedback for LLM-guided program evolution. It records which functions and constants changed, along with the corresponding changes in evaluation metrics, and uses this history to guide future edits. Comparisons with the parent program guide individual mutations, while comparisons with the initial seed help suggest new directions when search stalls. The authors integrate the method into AdaEvolve and evaluate it mainly on retrieval and reranking across twelve BRIGHT datasets, reporting better held-out ranking quality and reaching comparable training scores in fewer iterations. Under a cost-aware objective, it also finds pipelines that use fewer tokens per query than AdaEvolve’s.

Critical strengths
  1. I think the idea is clear and well motivated. I believe the idea of showing the LLM which components changed and how the metrics responded is novel and helpful in the evolution.

  2. The paper is cleanly written, readable, and the results are encouraging, the authors show better average held-out ranking quality than across twelve BRIGHT datasets, and they use AdaEvolve as the backbone and the reference. I believe the cost-aware experiments are also valuable.

Critical weaknesses
  1. The results show an improvement over AdaEvolve, but the advantage over other feedback-based methods is a bit unclear. AdaEvolve is a meaningful baseline for testing the proposed addition. Still, a comparison with at least one related method, such as ReEvo, DeltaEvolve, or GEPA’s optimize_anything extension, under comparable conditions would help establish the benefit more clearly. I would also like to understand what the component-aware representation itself adds. Table 4 suggests that richer metrics account for about half the gain on the three evaluated datasets, with attribution providing the rest. This is useful evidence, but it does not distinguish the benefit of organizing changes by functions and constants from simply showing previous edits alongside their metric changes.

  2. This is less critical, but I think the authors need to tone down their claim about search-compute savings. Section 3.2 suggests that one-third of the iterations means roughly one-third of the compute, but evaluation costs can change as programs evolve. For example, the final quality-only pipelines use 71.8k tokens per query versus AdaEvolve’s 53.7k. These figures do not establish total search cost, but they show why fewer iterations need not mean proportional savings.Otherwise, the claim could be limited to fewer search iterations.

Questions for authors
  1. I noticed that in the tables, several per-dataset gains are small relative to the reported variation. Do you think using three-seeds is enough for these settings?

  2. In Prompt 7, the current best score is first listed as 0.607106, but later in the analysis framework part, it is given as 0.380053, which matches the seed score. Was this also present in the experimental prompts, or is it just an appendix error?

  3. Could you clarify what exactly changes in the “without attribution” ablation? Does the model still see the same recent attempts and their metric changes?

  4. Are the same query splits used across methods and runs, and do they stay fixed during each search? I also noticed that the seed program’s training score varies across runs, could you explain what causes this since you're also using greedy reranking?

  5. As an optional ablation, have you tried local and global feedback separately? I’m curious whether global feedback really helps the search recover when it stalls.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 6EZr

Thank you for taking the time to review our paper and for your thoughtful comments.

What the component-aware representation adds (Weakness 1)

The component-level representation itself drives the gain, as describing edits by their components outperforms both plain code diffs and LLM-written deltas under the same metrics. We extended the ablation to six BRIGHT datasets and added two other ways of describing past edits, plain code diffs and LLM-written deltas following DeltaEvolve (Table B, paper Table 4). Adding the metrics alone barely changes the score, so most of the gain comes from the attribution. It gives the best mean and reaches AdaEvolve’s best score fastest. Showing previous edits alongside their metric changes is therefore not enough on its own, and how each edit is described matters.

Search compute claim (Weakness 2)

We now report compute directly instead of inferring it from iterations. Our search reaches AdaEvolve’s best score with half the mutation and evaluation tokens AdaEvolve needs to get there (Table A), and 77% and 55% under the cost-aware objective (paper Table 2). We have revised Section 3.2 accordingly (paper Tables 1, 2 and 7). Every iteration pairs one mutation call with one evaluation, and for many tasks, such as training recipes or large scale distributed systems, the evaluation dominates. Reranking is one such case, where evaluation takes over 97% of all tokens the search spends (Table A). Each iteration saved is therefore one fewer expensive evaluation. Our mutation prompts are longer, but mutation is such a small share of all tokens that saved iterations outweigh them.

Are three runs enough? (Question 1)

Three runs per dataset is the typical configuration, as used in the original AdaEvolve paper. We evaluate our method by its mean across the 12 datasets rather than by any single dataset or run. Our method achieves a higher mean test score on 9 of 12 datasets, and in terms of raw best score, we find a better solution on 10 of 12 datasets (paper Table 1). The gains hold in the same direction across every setting we test, the main results, ablation, OpenEvolve, transfer, and a different mutator model (Tables B to D, paper Table 3).

Prompt 7 score (Question 2)

This is an error in the appendix only. We have corrected it in the revision.

What the "without attribution" ablation changes (Question 3)

The without attribution ablation adds the decomposed metrics as feedback and nothing else. It does see a basic version of recent attempts with just the score changes, which the default search controller already supports. Our method replaces this basic version of recent attempts with the score changes plus the component-aware feedback. We have clarified this in Section 3.2 (Ablations).

Query splits and seed score variation (Question 4)

Yes, every method and run uses the same fixed 60/40 train/test split per dataset, and it does not change during a search. The seed's training score varies because vLLM batching is nondeterministic, which changes outputs slightly even under greedy decoding (vLLM batch invariance docs). This affects every method equally.

Local versus global feedback (Question 5)

We have not tried local and global feedback separately, and leave this ablation to future work.

We hope the additional experiments address your concerns and would appreciate your reassessment in light of these results. Thank you again for your time.

Thank you for the additional experiments. The comparison with plain diffs and LLM-written summaries, and the token measurements address my points. I’ve increased my score from 3 to 4.

Thank you for raising your score. We are glad our additional experiments and revisions have addressed your concerns. Thank you again for your time and constructive feedback.

Reviewer_N7zAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes component-aware feedback for LLM-guided evolutionary program search. After each evaluation, an observe step decomposes the parent and child programs into named components, namely functions and module-level constants. It then records which components were added, removed, or modified, together with the resulting change in every metric the evaluator reports. These records are stored in an attribution memory and presented to the mutator in two reference frames: a local frame that compares each child with its parent, and a global frame that compares it with the seed program. The approach is added to AdaEvolve and evaluated mainly on evolving LLM reranking pipelines on the BRIGHT benchmark.

Critical strengths
  1. The problem formulation is reasonable and addresses an underexplored part of LLM-guided evolutionary search. Most prior work improves how the search is controlled, whereas this paper asks what information each evaluation should retain to guide the mutator. The observation that showing whole programs with a single fitness score leaves the mutator to infer the effect of earlier edits is well motivated.

  2. The proposed feedback is simple and practical. It is constructed entirely from information the search already produces, namely source differences between programs and the metrics returned by the evaluator. It therefore requires no additional LLM calls, extra evaluations, or learned models, and it can be added to an existing framework without changing its search controller.

Critical weaknesses
  1. The design choice of feeding component-level edit records to the mutator, and the evaluation used to support it, are not sufficiently justified. The paper motivates the method by noting that LLMs struggle to use long and cluttered contexts, yet the proposed feedback may add to this burden rather than reduce it. Although the memory is windowed, each prompt still contains the parent and context programs, now displayed component by component, and additionally includes recent edit records with their metric changes. The context the mutator must process is therefore likely to grow rather than shrink. However, prompt length is not reported, so it remains unclear whether the method alleviates the very problem it is designed to address. Because a component counts as modified whenever its source text differs, trivial edits such as comment or formatting changes are recorded in the same way as substantive ones, which can add further noise to this context. Metric changes are also attributed to the whole set of changed components rather than to individual ones. A higher-level description of what each edit did may therefore be a reasonable alternative. The paper argues that such descriptions are expensive and hard to trust, but this is not tested. On the main task, the method is compared only with AdaEvolve, and no method that summarizes past edits at a higher level, such as reflection-based or semantic-delta approaches, is evaluated under the same search controller. In addition, although the method is presented as a general approach for self-evolving programs, the evaluation focuses almost entirely on LLM reranking, apart from two small mathematical tasks. The choice of reranking as the primary testbed is motivated by its multi-component structure and quality–cost trade-off, but it is not well justified relative to the problems commonly used to evaluate LLM-guided evolutionary search. The benefit also appears to depend on an evaluator that exposes several task-specific metrics designed for reranking. The authors themselves note that attribution adds little on circle packing, which reports a single score. This makes it unclear how well the approach transfers to other settings.

  2. The experimental results do not clearly demonstrate the claimed advantages, and they raise questions about where the gains come from. The gains over AdaEvolve vary widely across datasets. They are large on a few datasets, modest on most, and negative on several. All main results come from a single benchmark, and the average improvement is driven largely by a small number of datasets. The efficiency claims are also difficult to assess. Based on the convergence curves, the paper argues that reaching the baseline’s final score in fewer iterations implies a proportionally smaller compute cost. However, the cost of the optimization itself, such as the time needed to complete the evolution and the total token usage of the search, is not reported. Each iteration of the proposed method may also be more expensive because of its longer prompts. Fewer iterations therefore do not necessarily mean a cheaper or faster search. More importantly, the method combines two parts: exposing additional evaluator metrics, and attributing metric changes to the edited components. The first essentially extends the set of metrics shown to the mutator. This is closer to an evaluator design choice than a methodological contribution, and it could be applied to any baseline. However, the main comparison is against AdaEvolve without these metrics. In the ablation that separates the two parts, the more consistent gains, including on the training score that the search optimizes, come from the additional metrics rather than from attribution. The paper interprets this as attribution narrowing the gap between training and test scores, but this is not the mechanism the method is motivated by, and the evidence comes from only three datasets. Since attribution is the paper’s main contribution, the fact that a large share of the improvement appears to come from the extended metrics raises questions about how much the proposed method itself contributes.

Questions for authors
  1. The introduction cites reflection-based search (ReEvo) and semantic-delta memories (DeltaEvolve) as prior approaches that represent earlier modifications, and argues that they are expensive and hard to trust because they rely on an LLM to describe each edit. Could the authors compare component-aware feedback with these methods? For example, ReEvo-style reflections and DeltaEvolve-style semantic deltas could be implemented on the same search controller with the same metrics. In addition, could the authors provide results on tasks beyond LLM reranking, such as problems commonly used to evaluate LLM-guided evolutionary search?

  2. What is the cost of a search run for each method, including the total token usage of mutation and evaluation calls and the time needed to complete the evolution? It would also help to show how performance at different stages of the search relates to the cost spent up to that point, for example by plotting progress against cumulative tokens or time rather than iterations.

3 comments
AuthorsOpenReview ↗

Response to Reviewer N7zA

Thank you for taking the time to review our paper and for your thoughtful comments.

Prompt length and context clutter (Weakness 1)

At every step our mutation prompt is longer, but a sliding window caps the edit history, so the prompt grows only modestly. Our search still reaches AdaEvolve’s best score with half the tokens (Table A). AdaEvolve's prompt already lists the score changes of the parent's earlier children (Appendix B, Prompt 4), but with no description of what each mutation did. We display the source code component by component and replace that list with component-level records. The edit history window holds only the W=10 most recent edits, which accounts for about 2.4k tokens. Over a 100 iteration search, the mutation prompt grows from 3.3k to 8.4k tokens versus 3.5k to 6.9k for AdaEvolve.

Comparison with higher-level edit descriptions (Weakness 1, Weakness 2, Question 1)

Under the same search controller and metrics, describing edits by their components outperforms both LLM-written deltas (DeltaEvolve) and plain code diffs. We extended the ablation to six BRIGHT datasets and implemented two other ways of describing past edits on the same search controller with the same metrics, plain code diffs and LLM-written deltas following DeltaEvolve (Table B, paper Table 4). Adding the metrics alone barely changes the score, so most of the gain comes from the attribution. It gives the best mean and reaches AdaEvolve’s best score fastest. Showing previous edits alongside their metric changes is therefore not enough on its own, and how each edit is described matters.

Tasks beyond reranking (Weakness 1, Question 1)

Our method remains competitive beyond reranking, and is best where objectives compete. We now compare against ShinkaEvolve, CORAL, AdaEvolve, plain diffs, and LLM deltas on four mathematical optimization tasks commonly used for LLM-guided evolutionary search (Table E, paper Figure 4). Our method is comparable to the strongest methods on the three single-score tasks, where a component-level record says little beyond the score change itself, and has the highest mean on signal processing, where objectives compete.

Cost of the search (Weakness 2, Question 2)

We now report the cost of every search run in paper Tables 1, 2, and 7, and our search reaches AdaEvolve’s best score with half of its mutation and evaluation tokens, and with 77% and 55% under the cost-aware objective (paper Table 2). We also plot the best score against cumulative evaluation tokens in Appendix A.2 (summary in Table A). Evaluation accounts for over 97% of all tokens, so when task evaluation is expensive, as in LLM reranking, training recipes, or large scale distributed systems, fewer search iterations greatly reduce cost.

We hope the additional experiments address your concerns and would appreciate your reassessment in light of these results. Thank you again for your time.

Dear Reviewer N7zA,

As the discussion period ends today, we wanted to check in and see whether our response has addressed your concerns. Please let us know if you have any remaining questions, and we would be happy to clarify.

Best,

The Authors

Thank you for the response and the additional experiments, which address some of my concerns. However, the framing remains broader than the evidence: the paper presents the method as making self-evolving systems more efficient, while the results show a benefit mainly where multiple objectives compete and comparable performance on single-score tasks. In addition, the conclusion of the ablation changes with the choice of datasets, and with three runs per dataset the gains are small relative to the variation. The concern about trivial edits being recorded as modifications was not addressed. I will maintain my original rating.

Reviewer_xkpUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This is an interesting paper that adds component-aware feedback to LLM-guided evolutionary search. Their proposed approach works as follows: it first evaluates a child program, then it records the functions or constants that changed alongside differences in the evaluator’s metrics; afterwards it stores changes relative to both the parent and the seed; finally it gives this attribution memory to later mutation and strategy prompts. The method is evaluated on a few tasks, including LLM reranking across Bright dataset (composed of 12 retrival datasets), in addition to circle packing and signal-processing tasks. Compared to plain AdaEvolve, the authors report higher average held-out score, faster convergence and better quality-cost trade-offs.

Critical strengths

I believe, one of the core benefits of the method is that it is simple to implement and does not require an additional evaluator call or a learned attribution model. Additionally, it seems reasonable to compare local and seed-relative views to show both recent changes and cumulative progress.

Furthermore, it is a plus that the experiments cover multiple datasets and include a cost-aware objective. I also believe that it is useful to report the gains on held-out reranking results, and the ablation separating metric decomposition from edit attribution.

I do appreciate that the paper provides substantial implementation detail, which includes prompts and example programs. This will make it easier to reproduce the paper in the future.

Critical weaknesses

There are a few weaknesses that would need to be addressed before acceptance. For example, the paper argues that reaching a score in fewer iterations means roughly a proportional reduction in compute. However, the paper does not report total mutation-model tokens, wall-clock time, or end-to-end search cost for either method, with the attribution records adding prompt context. The current evidence does not seem to directly establish a comparable reduction in total search compute, even though if fewer evaluations may still be valuable, especially when evaluation is expensive.

The baseline evidence also does not seem to isolate the broader contribution as well as it could? The method is implemented within AdaEvolve’s search controller while the central comparison is against AdaEvolve. Therefore, I believe the ablation partially helps isolate attribution from metric decomposition; however, only on three datasets. Comparisons with other search methods appear only on two additional tasks (as far as I could tell).

Finally, it would help to establish the importance of the method further, by adding a comparison against a compact diff summary or other non-LLM edit-history representations. Currently, it is a bit unclear to me how much of the gain comes from component-level records specifically, versus simply giving the mutator a more organized and informative prompt. In my opinion, this would be an important addition to the paper.

Maybe most imporatntly, the method identifies which components changed; however what happens when a mutation changes several components at once? In other words, it is not clear if the resulting metric reflects difference of individual edits, interactions among them, or both.

Minor comments:

  • I found two typos, in the component-aware feedback section: “programatically” → “programmatically.”
  • In the circle-packing results: “acorss runs” → “across runs.”
Questions for authors
  • How large do the attribution memories and mutation prompts become over a full run, and how do total search tokens and runtime compare with AdaEvolve?
  • Does the feedback provide similar gains when applied to another search controller or another open-weight model?
  • Could the authors report how many components typically change per mutation and test whether the feedback remains useful when multiple components change together?
3 comments
AuthorsOpenReview ↗

Response to Reviewer xkpU

Thank you for taking the time to review our paper and for your thoughtful comments.

Prompt size and total search cost (Question 1)

Our mutation prompts are longer, but our search reaches AdaEvolve’s best score with half of its mutation and evaluation tokens (Table A), and with 77% and 55% under the cost-aware objective (paper Table 2). Attribution memories use a sliding window, so only the W=10 most recent edits enter the mutation prompt at any time, which keeps the memory at about 2.4k tokens by the end of a run. Our mutation prompt goes from 3.3k tokens over the first 10 iterations to 8.4k over the last ten, compared to 3.5k to 6.9k for AdaEvolve. Across a full run, our search uses more mutation tokens per prompt. However, evaluation accounts for over 97% of all tokens the search spends (Table A, paper Table 7). Given how expensive evaluations are for tasks like LLM reranking and other tasks such as evolving training recipes or large scale distributed systems, any reduction in the number of search iterations can greatly reduce the runtime.

Edits that change several components (Question 3)

Every result we report is already in the multi-component setting, and the feedback remains useful there. On average, the base AdaEvolve search controller changes 5.3 components per mutation (Section 2.3), and our method runs on the same controller. Attribution is therefore set-level by design, and every metric change is attributed to the whole set of components. The mutator can still separate effects because the same component appears across edits with different companions and different outcomes within a window. We clarify this in Section 2.3.

Comparison with other edit descriptions and tasks beyond reranking (Question 2)

Describing edits by their components outperforms both plain code diffs and LLM-written deltas, and our method stays competitive on four tasks beyond reranking. We extended the ablation to six BRIGHT datasets and added two other ways of describing past edits, plain code diffs and LLM-written deltas following DeltaEvolve (Table B, paper Table 4). It gives the best mean and reaches AdaEvolve’s best score fastest, with the fewest tokens among comparable baselines. Showing previous edits alongside their metric changes is therefore not enough on its own, and how each edit is described matters. Beyond reranking, we now compare against ShinkaEvolve, CORAL, AdaEvolve, plain diffs, and LLM deltas on four mathematical optimization tasks (Table E, paper Figure 4). Our method is comparable to the strongest methods on the three single-score tasks, where a component-level record says little beyond the score change itself, and has the highest mean on signal processing, where multiple objectives compete.

Other search controllers and models (Question 2)

The gains carry over to another search controller and another mutator model. We conducted additional experiments on OpenEvolve (Sharma, 2025) with component-aware feedback added and no other changes. Our feedback raises the mean test nDCG@10 and reaches OpenEvolve's best score with fewer mutation and evaluation tokens (Table C, paper Table 5). With Gemma-4-26B-A4B as the mutator and Qwen still as the reranker, our method similarly raises the mean test nDCG@10 (Table D, paper Table 6).

Typos

Thank you, we have fixed both typos.

We hope the additional experiments address your concerns and would appreciate your reassessment in light of these results. Thank you again for your time.

Thank you for the additional experiments, which address most of my concerns and strengthen the paper. Separating individual component effects from interactions seems to remain untested, so I encourage tempering that claim. Overall, I will raise my score.

Thank you for raising your score. We are glad our additional experiments and revisions have addressed your concerns, and we will clarify this point in the revised paper. Thank you again for your time and constructive feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response (new experiments)

We thank all reviewers for their time and feedback.

Our paper introduces component-aware feedback, which records which components change and how each metric moves, so the mutator LLM can build on past edits instead of inferring their effect from cluttered histories. Because evaluating each candidate program is the expensive step of LLM-guided evolutionary search, reaching a good program in fewer iterations directly saves compute. The new experiments support three claims.

  • The savings are real compute, not just fewer iterations. Our search reaches AdaEvolve’s best score with half of its tokens (Table A).
  • The gain comes from describing edits by their components, not from the extra metrics or from simply showing past edits (Table B).
  • The feedback generalizes to another search controller and another mutator model, and remains competitive on four tasks commonly used for LLM-guided evolutionary search (Tables C to E).

Below we summarize the new experiments, which the replies refer to by letter. All changes in the revised paper are marked in blue.

Table A. Search cost (paper Tables 1 and 7)

Quality-only objective, mean over 12 BRIGHT datasets. Token ratios are relative to what AdaEvolve spends to reach its own best (×0.50 means 50%). Both methods run the same 100-iteration budget, so the full-run totals also include the iterations our search spends improving beyond AdaEvolve’s best.

AdaEvolveOurs
mean prompt (k tok.)5.87.4
mutation tokens (millions)1.602.19
evaluation tokens (millions)247.5348.3
share of search time, mutation / evaluation34% / 66%29% / 71%
reaches AdaEvolve's best at iterationn/a32
tokens to reach it, mutation / evaluationn/a×0.50 / ×0.50

Summary: Evaluation is over 97% of all tokens, so despite longer prompts, our search reaches AdaEvolve's best score with half of its mutation and evaluation tokens.

Table B. How past edits are described (paper Table 4)

All rows use AdaEvolve's controller, and the bottom three differ only in how each past edit is described. Means over 6 BRIGHT datasets. iter. is the first iteration at which a row's best train score, averaged over three runs, reaches AdaEvolve's, with the number of datasets that reach it in parentheses. mut. and eval are the tokens spent up to then, as in Table A.

AdaEvolve withtest nDCG@10prompt (k tok.)iter.mut.eval
nothing added0.4056.0n/an/an/a
metrics only0.4096.349 (5/6)×0.68×0.81
metrics + edits as code diffs0.39920.967 (4/6)×2.05×1.15
metrics + edits as LLM deltas (DeltaEvolve)0.4136.262 (6/6)×0.96×0.67
metrics + edits as components (Ours)0.4318.136 (6/6)×0.59×0.54

Summary: Adding the metrics alone barely changes the score, while describing edits by their components gives the highest mean test nDCG@10 and reaches AdaEvolve’s best score fastest with the fewest tokens.

Table C. Another search controller (paper Table 5)

Component-aware feedback added to OpenEvolve (Sharma, 2025) with no other changes. Means over 6 BRIGHT datasets, and other columns as in Table B.

test nDCG@10prompt (k tok.)iter.mut.eval
OpenEvolve0.3403.1n/an/an/a
OpenEvolve + ours0.3845.942 (4/6)×0.72×0.68

Summary: Our feedback raises the mean test nDCG@10 from 0.340 to 0.384 and reaches OpenEvolve’s best score with fewer mutation and evaluation tokens.

Table D. Another mutator model (paper Table 6)

Gemma-4-26B-A4B as the mutator for both methods, with Qwen3.6-35B-A3B still as the reranker. Means over 6 BRIGHT datasets, and other columns as in Table B.

test nDCG@10prompt (k tok.)iter.mut.eval
AdaEvolve0.4158.3n/an/an/a
Ours0.4269.240 (2/6)×0.40×0.38

Summary: Our method raises the mean test nDCG@10 from 0.415 to 0.426.

Table E. Mathematical optimization (paper Figure 4)

Final best score, mean over 3 runs, higher is better.

ShinkaEvolveCORALAdaEvolveLLM deltasplain diffsOurs
Circle packing2.3222.5222.6192.6142.5732.617
Signal processing0.5600.6730.6510.6870.6570.698
Heilbronn0.03030.01520.02970.02900.03090.0307
Min-max distance0.23920.22840.23440.23380.24000.2346

Summary: Our method is comparable to the strongest methods on the three single-score tasks, where a component-level record adds little beyond the score, and is best on signal processing, where multiple objectives compete.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.