acceptodds
Desk rejected at ICLR 2027

EvoTreeNAD: Genealogy-Guided Evolution for LLM-Driven Neural Architecture Discovery

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

AI-driven scientific discovery accelerates research by autonomously developing solutions and designs. Large language model (LLM) agents support this process through iterative generation and evaluation. Yet these iterations alone do not ensure cumulative progress or establish which directions to pursue next. Costly evaluation further constrains the scope of exploration. Neural architecture discovery brings these challenges together, coupling open-ended design with resource-intensive experimentation. We introduce EvoTreeNAD, a genealogy-guided evolutionary algorithm that constructs trainable architectures without a supplied seed or a hand-specified search space. Starting from an empty root, it grows a persistent genealogy in which each new node represents a complete architecture. Top-percentile values computed from each node and its descendants guide lineage selection. Using the selected design history, an Idea Agent proposes a variant and a Code Agent implements it. Each evaluated variant becomes a child node, expanding the genealogy while providing evidence for subsequent lineage selection. Our theoretical analysis establishes the existence of stationary variation regimes as the genealogy grows. Under specified variation assumptions, sustained top-percentile family values quantify the probability of generating high-reward architectures in these regimes. EvoTreeNAD discovers architectures that outperform the compared NAS and NAD baselines, achieving CIFAR-10/100 test errors of % and %. On all six MedMNIST-v2 tasks, the discovered architectures surpass the strongest listed baselines. A controlled CIFAR-10 study further shows that EvoTreeNAD outperforms direct generation, best-of-\(N\) greedy continuation, and full-family-mean routing.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
LLM agents, neural architecture discovery, evolutionary algorithms, neural architecture search, automated discovery, genealogy-guided evolution

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WGk8OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This article proposes EvoTreeNAD, a lineage guided LLM neural architecture discovery method. The architecture and its evolutionary descendants are constructed into a lineage tree, which relies on the Top percentile family value to guide the selection of evolutionary branches and explore potential branches. The paper conducted experiments on CIFAR-10/100 and MedMNIST-v2, including multiple rounds of independent search, multiple training seeds, failure rate and cost statistics; Under a fixed OSS20B and 100 search budgets, this strategy outperforms baselines such as greed and family mean. At the same time, two relevant theoretical theorems are presented in the article to characterize the statistical properties of the upper tail of the reward spectrum. There are three core questions raised in the review: the lack of comparison between Top percentile lineage routing and mature tree searches such as UCT/MCTS, which can easily degrade to maximum value selection in small family sizes; The main result (GPT-4.1, 300 search rounds) is inconsistent with the protocol of controlled validation in Table 2, making it difficult to transfer conclusions; The setting of modifiable rules and loss functions results in attribution ambiguity, and the theoretical asymptotic conclusion has limited explanatory power for limited search budget scenarios.

Critical strengths

This work proposes an evolutionary paradigm of lineage persistence storage and value routing based on family branches, which can trace back and continue to develop historical branches with clear search logic. The experimental system is relatively solid: covering multiple datasets for image classification, reporting multiple independent searches, training seeds, code generation failure rates, and computational costs; Under the control setting of unified OSS20B and 100 trials, repeated experiments consistently outperformed baselines such as greed and family mean, with reproducible empirical support. The analysis of matching theory is not simply formalized in words, but provides mathematical conclusions on the distribution of family rewards, enriching the analytical perspective of search mechanisms.

Critical weaknesses

The independent innovation boundary of the Top penalty family routing is unclear, and there is a lack of comparison with classic tree search strategies such as UCT/MCTS in the same framework; When the family size is small, this mechanism is prone to degenerate into max selection, and the unique gain under limited budget is not quantified.

Inconsistent experimental protocols: The controlled ablation in Table 2 is all based on OSS20B with 100 searches, while the best result in the paper uses GPT-4.1 with 300 searches and continues the existing search trajectory, which cannot directly prove that this lineage selection strategy has equal advantages under the main result setting.

There is ambiguity in performance attribution: the evaluation rule is to pre select multiple candidates and select the best report, which can easily lean towards optimism; At the same time, Idea Agent allows modification of the loss function, but cannot guarantee that the performance improvement comes entirely from the architecture itself; In addition, the theoretical conclusion is asymptotic, and its guiding and explanatory role for limited search budget scenarios such as 100-300 times has not been fully demonstrated.

Questions for authors

1、How does the proposed highest percentile family routing compare to UCT/MCTS under the same search protocol and budget? Will it bring sustained benefits when the family size is small and the mechanism is reduced to its maximum selection?

2、The controlled ablation in Table 2 was tested 100 times using OSS20B, while the best reported results were tested 300 times using GPT-4.1 with continuous trajectory. Can you verify if the advantage of your family tree selection still exists under the GPT-4.1 300 experimental setup?

4 comments
AuthorsOpenReview ↗

Response to Reviewer WGk8 (1/2)

We thank the reviewer for recognizing the genealogy-guided search design, the evidence across tasks and discovery runs, and the mathematical analysis. EvoTreeNAD outperforms UCT in the 100-iteration comparison. With GPT-4.1/OSS20B and 300 iterations, it also outperforms greedy continuation, full-family mean, and UCT. The routing traces include switches to branches with lower family maxima.

1. UCT comparison and small-family behavior

UCT selects children using empirical mean rewards augmented by a visit-dependent exploration bonus, whereas EvoTreeNAD uses top-percentile family values for lineage routing without an explicit exploration bonus.

We added three independent UCT runs (Kocsis and Szepesvari, 2006) with the same OSS20B agents, 100-iteration budget, discovery split, and evaluation protocol. Three candidates per run were preselected by discovery reward. The table reports the highest full-fidelity test accuracy among them.

Controlled CIFAR-10 comparison of EvoTreeNAD and UCT:

Routing ruleRun 1Run 2Run 3Mean accuracy (%)
EvoTreeNAD, τ=0.1\tau=0.197.3997.2697.1197.25
UCT (new)97.1796.9996.6596.94

Under the matched 100-iteration protocol, EvoTreeNAD achieves 97.25% mean accuracy versus 96.94% for the exploration-oriented UCT variant.

A family includes the node and all its descendants (Section 3.2, "Architecture Genealogy and Family Values"). With τ=0.1\tau=0.1, the top-percentile family value equals the maximum for families of at most ten members, then averages additional rewards as the family grows. Under maximum-family routing, the winning child remains selected at a saturated parent (Proposition 1). Under top-percentile routing, additional descendant rewards can change the child ranking.

The table shows two routing decisions from a 300-iteration EvoTreeNAD run using GPT-4.1/OSS20B and τ=0.2\tau=0.2. Before each decision, the previously followed family grew from five to six members, bringing a second reward into its top-percentile average. All values are measured at the corresponding decision.

Branch switches under top-percentile routing:

IterationBranchTop-percentile family valueFamily maximum
34Previously followed0.363830.71748
34Newly selected0.677860.67786
135Previously followed0.756550.76416
135Newly selected0.763290.76329

At both decisions, the newly selected branch has a higher top-percentile family value despite a lower family maximum. Top-percentile routing therefore redirects development where maximum-family routing would retain the previous branch. The first switch occurs at iteration 34, showing that this distinction can emerge early in discovery.

2. Routing under GPT-4.1/OSS20B and 300 iterations

We compare the existing EvoTreeNAD run using τ=0.1\tau=0.1 with four new 300-iteration discovery runs, one each for greedy continuation, full-family mean, UCT, and top-percentile routing with τ=0.2\tau=0.2. The new runs start from empty roots. All five use GPT-4.1 as the Idea Agent and OSS20B as the Code Agent, with the same discovery split and evaluator.

The Reward-Top1 comparison selects the architecture with the highest discovery reward from each run. The table reports its best test accuracy during full-fidelity training and its test accuracy at epoch 500.

CIFAR-10 results with Reward-Top1 selection (GPT-4.1/OSS20B, 300 iterations):

Routing ruleBest test (%)Final-epoch test (%)
EvoTreeNAD, τ=0.1\tau=0.197.7897.63
EvoTreeNAD, τ=0.2\tau=0.297.6897.50
UCT97.1097.00
Full-family mean96.9696.75
Greedy continuation96.7896.73

With architecture selection based on discovery reward and evaluation fixed at epoch 500, EvoTreeNAD outperforms greedy continuation, full-family mean, and UCT. The advantage persists under the agent configuration and budget used for the main results.

Under candidate-set reporting, we select multiple candidates from each discovery run by discovery reward and evaluate them under the full-fidelity protocol. We report the best test accuracy among these candidates.

CIFAR-10 results with candidate-set reporting (GPT-4.1/OSS20B, 300 iterations):

Routing ruleBest test (%)Final-epoch test (%)
EvoTreeNAD, τ=0.1\tau=0.198.0097.96
EvoTreeNAD, τ=0.2\tau=0.297.6897.50
UCT97.1097.00
Full-family mean96.9896.89
Greedy continuation97.0996.94

EvoTreeNAD maintains its advantage over the three routing controls under both Reward-Top1 selection and candidate-set reporting, including at epoch 500.

AuthorsOpenReview ↗

Response to Reviewer WGk8 (2/2)

3. Theoretical interpretation of routing behavior

For a fixed family size, averaging multiple top rewards reduces the worst-case influence of any single reward compared with using only the maximum (Appendix A.2, "Properties of Family-Value Rules", Proposition 2). Maintaining a top-percentile value a fixed amount above a reward threshold requires a nonvanishing fraction of high-reward family members as the family grows. Isolated early successes therefore cannot sustain that gap indefinitely, and weak descendants can change branch rankings even when the family maximum is unchanged. Under its variation and convergence assumptions, Theorem 2 gives an exact relation between limiting top-percentile family values and long-run high-reward production.

4. Architecture design and performance attribution

Section 3.1 ("Task Setting") defines an architecture as a complete trainable model specification, including its model-defined loss. Within each controlled comparison, all routing variants have the same freedom to design the network and its loss, and use the same LLM agents, task interface, evaluator, and external training protocols (Appendix D.3, "Full-Fidelity Evaluation Protocol", Table 8). The matched CIFAR-10 routing comparisons show that top-percentile routing discovers higher-performing trainable architectures than the alternative routing rules.

We will add the UCT variant to Table 2. The GPT-4.1/300-iteration routing comparisons and branch-switch analysis will be included in the appendix and referenced from the main text.

I think my concerns are resolved and I raise my rating.

Thank you for your thoughtful review and constructive feedback. We are glad that our clarifications and additional experiments have addressed your concerns, and we sincerely appreciate your time, engagement, and positive reassessment of our work! Thank you so much!

Reviewer_vxaAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EvoTreeNAD discovers complete neural architectures from an empty root. It preserves a genealogy and uses each family's top-percentile mean reward to guide further LLM-generated variations. Separate agents propose and implement models. The analysis relates family scores to the frequency of high-reward outcomes under explicit assumptions. Experiments cover CIFAR and six MedMNIST tasks, including controlled CIFAR-10 comparisons of routing strategies.

Critical strengths
  1. With the same 100-iteration budget and OSS20B agents, EvoTreeNAD averages 97.25% CIFAR-10 accuracy versus 96.77% for greedy continuation. Both report the highest test accuracy among three candidates fixed by discovery reward. This common protocol makes the 0.48-point mean gap useful evidence for the routing choice.
  2. The analysis gives a useful interpretation of the routing controls: family maxima lead to greedy continuation, while top-percentile scores can track high-reward frequency under fixed-component mixture and convergence assumptions. This connects the choice of score to whether weak variants obscure a productive lineage.
Critical weaknesses
  1. Selecting reported results by test accuracy may overstate the gains. The strategy comparison takes the best test accuracy among three candidates, and the main CIFAR results also include a best-seed row. The released CIFAR evaluator selects checkpoints by test accuracy; MedMNIST code reports the highest test accuracy across epochs. Their use in the reported results remains unclear. The best-of-three reporting rule is shared across strategies, but small cross-paper margins remain difficult to interpret.
  2. The cross-paper margins do not isolate gains from the search method. Comparisons with published baselines use different training recipes and resolutions, including EvoTreeNAD's disclosed 64×64/64³ MedMNIST inputs. The controlled CIFAR-10 study provides matched comparisons, but equivalent controls are not reported for the medical tasks.
Questions for authors
  1. Does the routing advantage persist when selection never uses test results? From each existing strategy run's three candidates, report final-epoch test accuracy for the one with the highest discovery reward. On OCTMNIST, also compare final-epoch accuracy with accuracy at the epoch selected on the official validation split.
  2. How much of the medical-task gain comes from the discovered architecture? On OCTMNIST, train ResNet-18 and the candidate chosen by discovery reward using the same 64×64 inputs and full 60-epoch training recipe. Select checkpoints for both on the official validation split; their test-accuracy gap would help isolate the architecture's contribution.
2 comments
AuthorsOpenReview ↗

Response to Reviewer vxaA (1/2)

We thank the reviewer for recognizing the controlled evidence for our routing choice and its theoretical interpretation. In the Reward-Top1 comparison, EvoTreeNAD retains the highest mean final-epoch accuracy. Both released OCTMNIST architectures also outperform ResNet-18 with fewer parameters under the same training and checkpoint-selection protocol.

1. Reward-Top1 and final-epoch CIFAR results

Each run contributes one architecture selected by discovery reward. Best is its highest test accuracy during full-fidelity training, and Final-epoch is its accuracy at epoch 500. Each Run entry gives Best / Final-epoch for the same architecture, in percent. The means are taken across the three discovery runs.

CIFAR-10 results with Reward-Top1 selection (100 iterations):

StrategyRun 1Run 2Run 3Mean Best (%)Mean Final-epoch (%)
Direct generation96.94 / 96.8596.59 / 96.5596.60 / 96.5096.7196.63
Greedy continuation95.21 / 95.1796.92 / 96.8497.06 / 97.0296.4096.34
Full-family mean96.75 / 96.7595.66 / 95.6596.49 / 96.4696.3096.29
UCT (new)96.85 / 96.7896.99 / 96.8696.45 / 96.2896.7696.64
EvoTreeNAD, τ=0.1\tau=0.197.17 / 97.1296.78 / 96.7896.97 / 96.8596.9796.92

EvoTreeNAD's mean Best and Final-epoch accuracies are 96.97% and 96.92%, compared with 96.76% and 96.64% for UCT. It ranks first among the five strategies under both reporting rules. The routing advantage remains when each run contributes a single architecture selected by discovery reward and accuracy is reported at epoch 500.

Discovery and candidate preselection use only training and validation data. The three candidates are fixed before full-fidelity training and testing (Section 3.4, "Final Architecture Selection", and Section 5.1, "Protocol"). Short-budget scores correlate with full-fidelity performance but remain imperfect predictors (Pearson r=0.7381r=0.7381, Appendix E.2, "Architecture Performance Along Principal Lineages", Figure 3). We therefore preselect three candidates rather than relying on a single score.

2. Table 1 reporting

For EvoTreeNAD, Table 1 reports the mean and standard deviation across four training seeds, using the best test performance observed during full-fidelity training for each seed. The best-training-seed result is a separate row.

For the same architectures and four training seeds, we report both best and final-epoch test errors. Values are mean ±\pm std, and Final-epoch refers to epoch 500.

Best and final-epoch CIFAR test errors:

DatasetArchitecture in Table 1Best test error (%)Final-epoch test error (%)
CIFAR-10Best discovered architecture2.05±0.062.05 \pm 0.062.11±0.092.11\pm0.09
CIFAR-10Smaller discovered architecture2.38±0.062.38\pm0.062.49±0.072.49\pm0.07
CIFAR-100Best discovered architecture15.09±0.2215.09\pm0.2215.38±0.4115.38\pm0.41
CIFAR-100Smaller discovered architecture15.18±0.1015.18\pm0.1015.50±0.1515.50\pm0.15

Across these four architectures, using epoch 500 instead of the best test checkpoint increases mean test error by 0.06-0.32 percentage points.

The baseline results in Table 1 come from the original papers, which do not always state how the reported checkpoint was selected.

3. Matched OCTMNIST comparison

The anonymous GitHub repository includes two discovered architectures per task, obtained from independent discovery runs. For OCTMNIST, evotreenad_run1_best.py is the architecture reported in Table 3, and evotreenad_run2_best.py is the other released architecture. We evaluate both alongside the official ResNet-18 under the same protocol.

All three use native 64×6464\times64 inputs, the official data splits (Appendix D.2, "MedMNIST-v2 Tasks and Data", Table 7), and the same 60-epoch training recipe.

For validation-selected test accuracy, we use the checkpoint with the highest accuracy on the official validation split. The table reports mean test accuracy across three training seeds per architecture. Best test uses the highest test accuracy across epochs for each seed.

Matched OCTMNIST comparison across three training seeds:

ModelParameters (M)Best test (%)Final-epoch test (%)Validation-selected test (%)
Official ResNet-1811.17191.6788.8789.30
evotreenad_run1_best.py (the arch reported in Table 3)1.62692.3789.7389.70
evotreenad_run2_best.py0.97892.8090.5090.43

Both discovered architectures outperform ResNet-18 under all three reporting rules. Validation-selected accuracy improves by 0.40 and 1.13 percentage points, with 1.626M and 0.978M parameters versus 11.171M. The advantage remains after controlling for resolution, training recipe, and checkpoint selection, supporting a contribution from the discovered architectures.

AuthorsOpenReview ↗

Response to Reviewer vxaA (2/2)

4. MedMNIST reporting across six tasks

For the accuracies in Tables 3 and 4, we record the highest test accuracy during full-fidelity training for each seed and report the mean and standard deviation across four training seeds.

The original four-seed evaluations did not record validation metrics. For each of the six MedMNIST tasks, we therefore retrain and evaluate the two discovered architectures released in our anonymous GitHub repository, using three additional training seeds per architecture. We record both validation and test metrics and report best-test, final-epoch, and validation-selected test accuracies as mean and standard deviation across these seeds. The Original table column identifies the architecture reported in Table 3 or Table 4.

MedMNIST results under three reporting rules (accuracy, three training seeds per architecture):

TaskReleased architectureOriginal tableParameters (M)Best test (%)Final-epoch test (%)Validation-selected test (%)
PathMNISTevotreenad_run1_best.pyTable 30.94595.60±0.3195.60 \pm 0.3194.35±0.1794.35\pm0.1794.50±0.5094.50\pm0.50
PathMNISTevotreenad_run2_best.pyNot shown3.92596.01±0.3696.01\pm0.3695.29±0.1895.29\pm0.1895.33±0.2295.33\pm0.22
OCTMNISTevotreenad_run1_best.pyTable 31.62692.37±0.7492.37\pm0.7489.73±1.0189.73\pm1.0189.70±1.0189.70\pm1.01
OCTMNISTevotreenad_run2_best.pyNot shown0.97892.80±0.2692.80\pm0.2690.50±0.5690.50\pm0.5690.43±0.6890.43\pm0.68
TissueMNISTevotreenad_run1_best.pyNot shown1.12876.63±0.0976.63\pm0.0976.23±0.2276.23\pm0.2276.58±0.0776.58\pm0.07
TissueMNISTevotreenad_run2_best.pyTable 36.12876.73±0.0376.73\pm0.0376.41±0.1876.41\pm0.1876.62±0.0576.62\pm0.05
Vessel3Devotreenad_run1_best.pyTable 40.36497.29±0.3097.29\pm0.3096.86±0.2696.86\pm0.2696.86±0.4596.86\pm0.45
Vessel3Devotreenad_run2_best.pyNot shown0.22196.07±0.6996.07\pm0.6995.38±0.8495.38\pm0.8495.38±0.6095.38\pm0.60
Synapse3Devotreenad_run1_best.pyTable 41.47396.12±0.1696.12\pm0.1695.27±0.5995.27\pm0.5995.74±0.4995.74\pm0.49
Synapse3Devotreenad_run2_best.pyNot shown1.47394.79±0.7194.79\pm0.7194.13±0.1694.13\pm0.1694.03±1.3094.03\pm1.30
Organ3Devotreenad_run1_best.pyTable 41.28995.52±0.3895.52\pm0.3895.14±0.5095.14\pm0.5094.64±0.9094.64\pm0.90
Organ3Devotreenad_run2_best.pyNot shown0.31296.01±0.5096.01\pm0.5095.79±0.5095.79\pm0.5095.79±0.5095.79\pm0.50

The discovered architectures retain strong performance under final-epoch and validation-selected evaluation, supporting the conclusions of Section 5.5, "Architecture Discovery across MedMNIST-v2 Tasks," without relying on test-based checkpoint selection.

We will specify the best-test reporting rule in the captions of Tables 3 and 4 and include the complete MedMNIST results in the revised appendix.

Reviewer_9WugOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This submission intend to use LLM agent for neural architecture discovery (NAD). Current NAD suffers dual challenge of open-ended design and resource-intensive evaluation: 1) simple and naive iteration cannot guarantee sustained, cumulative progress and 2) high evaluation costs limit the breadth and depth of exploration. To this end, this submission proposes EvoTreeNAD which employs an "Idea Agent" and a "Code Agent" to automatically generate neural architectures. And use the tree search method to find the most possible architectures. Experimental results demonstrate that this method performs well on classification tasks. Furthermore, comparative studies on the CIFAR-10 dataset confirm that the "path selection rules" employed by EvoTreeNAD outperform alternative strategies, such as direct generation, greedy continuation, etc.

Critical strengths
  • Interesting method design: the introducing of the tree-based evolution strategy is meaningful.
  • Useful theoretical insight about greedy search: Proposition 1 explains why merely storing alternative options is insufficient to guarantee their subsequent reconsideration. To be specific, proposition 1 demonstrates that maximum-family routing is equivalent to a greedy continuation strategy based on selecting the best among NN options. In other words, extending the winning subtree does not lower its maximum value, meaning that the unselected sibling branches cannot surpass it.
  • Sound performance: The proposed method achieves superior performance at the tested benchmarks.
Critical weaknesses
  • The performance comparisons do not support the improvement of search quality results in the imporvement of the performance.
  • The scale of the evaluated benchmarks is limited. CIFAR-10/100 and MedMNIST is too small and the claim may not generalize to larger benchmarks. Furthermore, sensitivity to the percentile fraction, branching capacity, and history length is not systematically evaluated.
  • The compared baselines is limited. The evolutionary architecture generation, persistent solution-space trees, and historical feedback mechanisms are already proposed, e.g., EvoPrompting and AIDE. However, the proposed method is evaluated only against methods such as direct generation, greedy continuation, and population-mean-based strategies, ignoring comparisons with more robust, exploration-oriented tree-search strategies.
Questions for authors

What is the total cost including local LLM inference, unsuccessful attempts, all discovery runs, and full-fidelity evaluation? Others pleasse refer to weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 9Wug (1/2)

We thank the reviewer for recognizing the genealogy-guided design, theoretical analysis, and benchmark performance. To address the concerns about search quality and baseline coverage, we conducted a controlled routing comparison with UCT under a matched discovery protocol and a system-level comparison using the official AIDE implementation under a comparable discovery budget. The new results show that EvoTreeNAD discovers higher-performing architectures in both comparisons, supporting the effectiveness of its routing mechanism and complete discovery pipeline.

1. Search quality under a matched discovery protocol

We evaluated an EvoTreeNAD variant with UCT routing (Kocsis and Szepesvari, 2006) in three independent runs for the Table 2 comparison, using the same OSS20B agents, 100-iteration budget, discovery split, and evaluation protocol. Three candidates per run were preselected by discovery reward and evaluated under the full-fidelity protocol. Under our standard candidate-set reporting, each Run entry gives the best-performing candidate's highest test accuracy during training (Best) and its accuracy at epoch 500 (Final-epoch).

Controlled CIFAR-10 comparison of discovery strategies (100 iterations):

StrategyRun 1Run 2Run 3Mean Best (%)Mean Final-epoch (%)
Direct generation96.94 / 96.8596.59 / 96.5596.60 / 96.5096.7196.63
Greedy continuation96.34 / 96.3496.92 / 96.8497.06 / 97.0296.7796.73
Full-family mean96.81 / 96.6995.66 / 95.6596.49 / 96.4696.3296.27
UCT (new)97.17 / 97.0596.99 / 96.8696.65 / 96.6196.9496.84
EvoTreeNAD, τ=0.1\tau=0.197.39 / 97.3797.26 / 97.1997.11 / 97.0097.2597.19

EvoTreeNAD ranks first under both metrics, with mean Best and Final-epoch accuracies of 97.25% and 97.19%, compared with 96.94% and 96.84% for UCT. With agents, training recipe, and budget fixed, top-percentile routing discovers architectures with higher full-fidelity accuracy than the UCT variant.

2. Mechanism and comparison with prior work

EvoPrompting combines code-level variation with evolutionary selection, while AIDE refines candidate programs in a solution tree (Section 2, "Related Work"). EvoTreeNAD's contribution is to use top-percentile descendant-family evidence to reassess earlier lineage choices and to condition subsequent architectural variation on the selected lineage's design and evaluation history (Sections 3.2, "Architecture Genealogy and Family Values", and 3.3, "Evolutionary Iteration"). Maximum-family routing instead remains equivalent to greedy continuation despite retaining alternatives (Proposition 1).

We ran the official AIDE implementation twice on CIFAR-10 with OSS20B, the same 42k/8k split, and a 10-hour discovery budget per run. We preserved AIDE's original search procedure and validation-based solution selection. Each selected program was retrained on the full 50k training set with its own training recipe, for the number of epochs chosen by AIDE and for 500 epochs.

AIDE architectures and full-fidelity test accuracy:

AIDE runSelected architectureParameters (M)AIDE-selected epochsFinal test at selected epochs (%)Final test at 500 epochs (%)
1ResNet34Cifar21.2828095.6196.73
2ResNet18Cifar11.1743093.2895.64

The EvoTreeNAD architectures reported in the controlled comparison achieve a mean epoch-500 test accuracy of 97.19%. All three outperform both AIDE solutions at epoch-500 evaluation.

Under Reward-Top1 reporting, all three EvoTreeNAD architectures also outperform both AIDE solutions, with a mean epoch-500 test accuracy of 96.92% (see CIFAR-10 results with Reward-Top1 selection (100 iterations) in our response to Reviewer vxaA).

The UCT comparison tests the routing choice within EvoTreeNAD, whereas the AIDE comparison tests the complete discovery pipeline against a separate system.

AuthorsOpenReview ↗

Response to Reviewer 9Wug (2/2)

3. Task coverage and parameter recommendations

The evaluation covers CIFAR-10/100 and six 2D/3D medical-image tasks (Appendix D.2, "MedMNIST-v2 Tasks and Data", Table 7). Scaling EvoTreeNAD depends on informative evaluation feedback at an affordable computational cost (Section 6, "Discussion and Conclusion").

As a family grows, maintaining its top-percentile value a fixed amount above a threshold requires a nonvanishing fraction of high-reward members (Appendix A.2, "Properties of Family-Value Rules", Proposition 2). Under its variation and convergence assumptions, Theorem 2 gives an exact relation between limiting top-percentile family values and long-run high-reward production rates.

  • Percentile fraction τ\tau. A larger τ\tau can reduce the influence of a single reward by averaging more family members, but also includes lower-reward variants. An additional comparison uses GPT-4.1 as the Idea Agent and OSS20B as the Code Agent, with one 300-iteration discovery run per routing setting. On CIFAR-10, EvoTreeNAD discovers competitive architectures at both τ=0.1\tau=0.1 and τ=0.2\tau=0.2. Architectures selected by discovery reward achieve at least 97.5% final-epoch test accuracy, outperforming those discovered by UCT, full-family mean, and greedy continuation. We recommend 0.1-0.2 as a mechanism-informed starting range to combine focus on high-reward outcomes with support from multiple members as families grow.
  • Child capacity. W(s)W(s) sets the maximum number of children at node ss (Section 3.3, "Evolutionary Iteration"). At a fixed budget, wider branching allocates more trials to siblings and fewer to depth. We recommend a larger empty-root capacity to explore more initial designs and modest non-root capacities to leave budget for deeper refinement (Appendix B.1, "Hyperparameters", Table 6). The family-value relation in Theorem 2 holds for node-dependent fixed, finite child capacities (Appendix A.1, "Formal Setup").
  • History length. Both agents use 1-3 recent ancestors (Appendix B.1, "Hyperparameters", Table 6), while family values aggregate rewards from the node and all its descendants. Limiting the prompt history therefore does not truncate the evidence used for routing.

4. Discovery and full-fidelity cost

Table 10 in Appendix E.5 ("Agent Configurations and Cost") includes Code-Agent retries in the attempt count and reports these averages over three independent 100-iteration runs per agent configuration.

CIFAR-10 discovery cost per 100-iteration run:

Idea / Code AgentsAttemptsDiscovery GPU-daysIdea tokens (M)Code tokens (M)Agent API fees (USD)
OSS20B / OSS20B138.70.350.451.570
GPT-4.1 / OSS20B144.00.370.301.610.84

Discovery-run timing includes LLM-request waiting and failed attempts. The experiments in Table 10 use one A100 GPU per discovery run and two V100 GPUs for the local OSS20B service (Appendix D.1, "Discovery Protocol").

Each CIFAR-10 candidate has a training-time budget of 10.5 minutes and at most 5,000 optimization steps (Section 3.3, "Evolutionary Iteration", and Appendix B.2, "Discovery-Stage Evaluation"). Full-fidelity CIFAR-10 training uses 500 epochs (Appendix D.3, "Full-Fidelity Evaluation Protocol", Table 8) and takes approximately 5-10 hours per architecture and training seed. Our standard protocol uses three independent discovery runs per task, with three reward-selected candidates per run for full-fidelity evaluation.

We will add the UCT variant to Table 2. The AIDE comparison and parameter-selection recommendations will be included in the appendix and referenced from the main text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.