- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper treats scientific optimization as a search over the full search policy: selecting the starting design from a library of historical designs and evolving the optimizer program. They are searched by LLM agents, and the final policy is tested on eight unseen gravitational-wave detector topologies under a matched time budget.
Critical strengths
The where/how decomposition is clean. The limitations section shows the dependence on the historical experience library.
Critical weaknesses
-
The claim that the search policy is "left to evolution" is not supported. The supplement's prompts already contain the shape of the final policy, its constants, and related hints, so it is unclear what evolution actually discovered.
-
The baselines have no detail settings or initialization, and standard baselines for expensive black-box optimization such as Bayesian optimization and CMA-ES are missing.
-
The title's scope is much broader than the evidence. All experiments are on a synthetic domain and a problem family, and there is no second task to support the general claim.
Questions for authors
-
How many runs per method? Please report seed-level variance, especially on T2 where the margin over GEPA is 0.5% and over Sophia is 1.0%.
-
Please give the full baseline configurations: learning rates, initialization, hardware, budget, and tuning protocol, and add a Bayesian-optimization or CMA-ES baseline.
-
The introduction lists four factors but the method keeps two, both stages allocate budget, and the pipeline is sequential rather than jointly optimized. Why exactly where and how, and how does this relate to multi-start and algorithm-selection practice?
6 comments
Additional Responses to Reviewer wTme
Q3. Cross-Domain Generalizability
Thank you for highlighting this limitation. We have clarified the generalizable components of ADAPT and introduced a second scientific optimization task to evaluate its transferability.
ADAPT provides a reusable framework for historical experience construction, initialization selection, and optimizer evolution. Its core workflow is domain-independent. Applying ADAPT to a new scientific problem requires adapting the domain-specific data, retrieval and optimization interfaces, and prompts while preserving the overall evolution framework.
To validate its transferability, we introduced a second case study on aerodynamic airfoil optimization with 21 continuous variables, comprising 20 free-form deformation parameters and one angle of attack. The task minimizes aerodynamic drag under lift, area, and thickness constraints using a physics-based simulator. We constructed a library of 1,000 historical designs and evaluated ADAPT on eight held-out targets using 32 retrieved initializations. All methods were evaluated across the same 10 random seeds, with identical random initializations for their random components and matched evaluation budgets. ADAPT achieves a mean penalized objective of 0.171 ± 0.091, reducing the penalized objective by 66.2% relative to randomly initialized trust-constr and by 53.8% relative to the strongest agentic evolution baseline. This result extends our evaluation beyond gravitational-wave detector design and provides further evidence of the framework's applicability across scientific domains.
We have incorporated this experiment into Section 4.6 of the revised manuscript and clarified the distinction between reusable framework components and domain-specific adaptations.
Q4. Repeated-Run Reliability
Thank you for highlighting this concern. We have repeated all methods across the same 10 random seeds with identical random starting points and matched evaluation budgets. The revised tables now report seed-level variability for each topology.
On T2, ADAPT achieves a mean loss of 0.408 ± 0.025, compared with 0.422 ± 0.012 for GEPA and 0.480 ± 0.068 for Sophia. Although individual methods may perform better on specific runs or topologies, ADAPT achieves the lowest overall mean loss of 0.208 ± 0.098, compared with 0.568 ± 0.232 for GEPA and 0.739 ± 0.302 for Sophia. These results demonstrate consistent overall performance across repeated runs, without assuming uniform superiority on every topology.
We have updated Tables 1–3 in the main manuscript with the complete 10-seed results. The additional experiments provide stronger evidence of the effectiveness and robustness of ADAPT.
Q5. Policy Decomposition and Methodological Contributions
Thank you for this insightful comment. We have clarified the rationale for the policy decomposition and highlighted the methodological contributions of ADAPT beyond established optimization approaches.
The four factors identified in the Introduction are organized into two complementary decisions. The selection policy determines where to search by combining historical retrieval, initialization selection, and resource allocation across candidate trajectories. The optimizer determines how to search through update rules, numerical safeguards, and the use of the remaining computational budget. Both stages share a single evaluation budget but operate at different decision levels.
ADAPT adopts a sequential evolution strategy. Simultaneous joint optimization would introduce a large, highly coupled search space, substantially increasing computational cost and optimization complexity, particularly for expensive scientific evaluations. The Selection Evolver first identifies a promising initialization policy. The Optimizer Evolver then develops an optimization procedure conditioned on the selected starts. This decomposition reduces search complexity while retaining the complete search procedure as the optimization target. We have clarified this design choice in the revised manuscript.
ADAPT advances multi-start optimization and algorithm selection through feedback-driven policy evolution. It unifies historical scientific experience, adaptive initialization, resource allocation, and executable optimizer evolution within a single framework. This integration offers a new direction for AI-driven optimization and, to our knowledge, represents its first application as a unified approach to scientific optimization under physical constraints and limited computational budgets.
I appreciate the authors' efforts in addressing the comments. I have some more questions:
- What are the values of the baseline constants injected in the Selection Evolver prompt, and how does ADAPT compare against a same-budget, one-shot implementation of the prompt-suggested shortlist-and-race configuration without any evolution?
- Please recompute the aggregate Mean±Std column under a single stated convention (Table 1 baseline Std values equal the across-topology std of the per-topology means, whereas ADAPT's 0.013 does not), and reconcile the "lowest loss on all eight topologies" claim with OpenEvolve's lower T2 loss (0.321 vs. 0.408).
Response to Reviewer wTme
Q1. Evolutionary Discovery and Prompt Constraints
Thank you for raising this important concern. We have clarified what is prescribed by the scientific constraints and what is learned through evolution.
Scientific optimization requires explicit constraints to ensure valid and efficient exploration. Our prompts define the scientific objective, executable interfaces, evaluation budgets, and permissible search operations. They also suggest possible directions, including candidate screening, staged probing, and numerical safeguards, without prescribing the final algorithms. Within this search space, LLMs propose and refine executable policies based on measured scientific feedback.
The Selection Evolver determines the combination of four high-quality and four diversity-preserving candidates, the progressive 64-to-8-to-2-to-1 selection strategy, and the associated probing and budget-allocation rules. The Optimizer Evolver identifies a shorter momentum memory with β₁ = 0.7, a quadratic time-dependent update schedule, and explicit numerical state protection. These specific rules, parameter values, and their final implementations are selected through evolution rather than fixed by the prompts.
Figures 2 and 5 trace these successive changes from the reference implementations to the final algorithms, while Figure 4 presents the resulting executable optimizer. We have further added error bars from 10 random seeds to assess the stability of the evolution trajectories. These results clarify that ADAPT performs constrained, feedback-driven algorithm evolution, with scientific guidance defining the search boundaries and measured performance determining the resulting policies.
Q2. Baseline Configurations and Additional Comparisons
Thank you for this valuable suggestion. We have added Bayesian Optimization and CMA-ES as additional baselines and repeated all experiments across the same 10 random seeds with identical random starting points. All methods use matched scientific objectives and evaluation budgets of 4 hours per topology on eight NVIDIA H100 GPUs.
ADAPT achieves a mean loss of 0.208 ± 0.098, compared with 4.765 ± 0.080 for Bayesian Optimization and 4.534 ± 0.094 for CMA-ES, corresponding to reductions of 95.6% and 95.4%, respectively. ADAPT also outperforms the other classical and agentic baselines under matched evaluation conditions.
We have updated Tables 1–3 in the main manuscript with the repeated-run results and added complete baseline configurations to the supplementary material, including initialization procedures, learning rates, hyperparameters, hardware, evaluation budgets, and tuning protocols.