acceptodds
Under review as a conference paper at ICLR 2027

Agentic High-Dimensional Bayesian Optimization with Hypothesis- and Evidence-Guided Search

Abstract

High-dimensional Bayesian optimization (HDBO) seeks sample-efficient optimization when the number of variables is large relative to the evaluation budget. Recent LLM-based and agentic BO methods incorporate task knowledge and adapt search decisions during a run, but have primarily been evaluated on low- and moderate-dimensional problems. We ask whether this paradigm can transfer to the higher-dimensional regime. Our experiments show that these methods do not remain reliable in the high-dimensional regime, where the challenge is not only where to evaluate, but also which modeling assumption and search geometry to use when the objective's useful structure is unknown. We therefore introduce HERA, a Hypothesis- and Evidence-guided Research Agent, which manages revisable search hypotheses, and PRISM, which executes specialized optimizers. HERA remains competitive with strong classical HDBO methods on metadata-free synthetic functions. Across diverse real-world tasks, it achieves the best mean final objective on most benchmarks, outperforming strong classical and recent LLM-based or agentic methods. Further analyses characterize how structural evidence, task metadata, and decision frequency shape its behavior and performance.

Primary area:
optimization
Keywords:
LLM Agents, Black-box optimization, Bayesian optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_tAauOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors have used an agentic AI framework (labelled HERA) to dynamically control a suite of Bayesian optimisation strategies (labelled PRISM) to achieve robust performance against a suite of high dimensional BO tasks. They demonstrate that each individual high-dimensional BO strategy performs well on some tasks but not others. Similarly, previous agentic-controlled BO methods perform less well.

Critical strengths

The article presents a framework that performs well in experiments.

The framework is simple and clear, and therefore easily adoptable and extendable

Critical weaknesses

The article does not explain the differences between the current proposition and previous agentically-controlled BO

The article does not make it clear how the agent is set up and prompted. Code is provided, but I really want to see some more details in the main article.

This article bolts together existing tools for a solution. I'm not averse to that - no research is truly original - but I can't ascertain if there is anything interesting in the combination; the message I read from the article was simply that reasonably generic agentic control of a suite of existing methods helps improve/robustly performance.

Questions for authors

What is the special thing that makes this article interesting? Why is it substantially different from previous closely related efforts? Is it simply that you have access to a better LLM?

Reviewer_UgZ3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes HERA–PRISM, positioning the LLM as a strategy-level controller for high-dimensional Bayesian optimization rather than a routine generator of high-dimensional candidates. HERA integrates task context, optimization feedback, and structural diagnostics to select, reconfigure, or restart numerical backends and determine the number of evaluations per search block. PRISM handles within-block candidate generation, model updates, and shared observation histories, while supporting a one-shot semantic initialization charged to the common evaluation budget. Across four metadata-free synthetic tasks and eight real-world tasks, HERA remains competitive with strong numerical baselines on the synthetic suite and achieves better mean final objectives than the strongest evaluated numerical and LLM-based or agentic comparators on six and seven real-world tasks, respectively.

Critical strengths
  1. Clear separation between strategy control and numerical execution: Typed tools, candidate validation, and shared observation histories separate high-level strategy interventions from per-evaluation model updates, allowing different search strategies to reuse previous evaluations.

  2. Broad evaluation with diverse comparators: The main experiments use a common evaluation budget, shared initial designs, and ten seeds to compare numerical and agentic methods, with additional same-portfolio heuristic and UCB scheduling baselines on the synthetic tasks.

Critical weaknesses
  1. Confounded controller comparison: In Section 4.2, HERA additionally supports reconfiguration, restarts, and variable block lengths, whereas the scheduling baselines only select backends in fixed five-evaluation blocks, so Figure 3 does not isolate reasoning benefits from differences in control capabilities.

  2. Unestablished optimization benefit of structural diagnostics: Section 4.4 reports similar final performance on most tasks after removing sparsity, subspace, and interaction diagnostics, supporting changes in strategy use but not a clear sample-efficiency gain from this central module.

  3. Confounded initialization and online adaptation: The metadata ablation in Section 4.4 affects both initial-solution generation and subsequent decisions, leaving unclear whether the major gains in Figure 4 arise from better initialization or continued online strategy management.

Questions for authors
  1. Matched control capabilities: Does HERA retain its advantage over the same-portfolio heuristic and UCB schedulers in Figure 3 when restricted to backend selection with fixed five-evaluation blocks and no additional reconfiguration or restarts?

  2. Value of structural evidence: Could the authors conduct paired continuation runs from identical observation histories using original, masked, and shuffled structural diagnostics, with all other settings matched, to test whether diagnostic-driven strategy changes improve subsequent optimization?

  3. Gains beyond initialization: Could the authors provide fixed numerical baselines with the same HERA-generated semantic initial point under an identical total evaluation budget to quantify the online strategy-management gains in Figure 4 beyond initialization?

1 comment

Dear Reviewer UgZ3,

As the discussion phase comes to an end, please acknowledge the author's response and consider leaving them a comment on whether it has resolved some or all of your concerns.

Thanks!

Reviewer_PjTzOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how language models can help solve optimization problems with many variables and very few attempts to find a good solution.

The authors introduce HERA, a system that uses a language model to choose between five optimization methods. HERA can change methods, adjust their settings, and decide how long to use each one. The actual search is handled by another system called PRISM. HERA can also suggest a starting solution based on the task description.

The authors test their approach on four synthetic problems and eight real-world problems. HERA performs well against standard optimization methods and other language-model-based systems. It achieves the best average result among all tested methods on five of the eight real-world problems. The paper also studies which parts of the system help and how often the language model needs to make decisions.

Critical strengths
  1. I like the idea of letting the language model decide how to search, while leaving the actual optimization to numerical methods. Different methods work well for different problems, so being able to switch between them makes sense. HERA also keeps all previous observations when switching methods, which avoids wasting useful information. This is a practical approach to optimization problems with many variables and limited evaluations (Section 3).

  2. The experiments are a strong part of the paper. The authors test several types of problems and compare HERA with both standard optimization methods and other language-model systems. HERA performs well even on synthetic tasks where it does not receive task descriptions. The authors also report cases where their approach does not help. For example, removing some information about the problem's structure barely changes the final results on most tasks. This makes the evaluation more convincing because it shows both the benefits and the limitations of the method (Section 4 and Appendices E-H).

Critical weaknesses
  1. It is not clear which part of HERA is responsible for its biggest improvements. HERA can use the task description to suggest a good starting point before the main search begins. On tasks such as Rover and Ant, much of the improvement happens very early. The paper does not clearly show how much extra benefit comes from HERA's later decisions. Removing task descriptions does not answer this question because it changes both the starting point and the later decisions. Also, removing the extra information about the problem's structure has little effect on most final results. A comparison with a standard optimizer using exactly the same starting points would help explain what HERA's ongoing decisions add (Section 4.4 and Appendices F and H).

  2. The comparison with simpler methods does not fully explain why HERA performs better. HERA can change optimizer settings, restart methods, and decide how long each method runs. The simpler controllers have fewer choices. Therefore, the results show that the complete HERA system can work better, but they do not tell us whether the improvement comes from better reasoning or simply from having more control. I would like the authors to be more careful when using these results to argue that simpler controllers cannot make good decisions (Section 4.2 and Appendix D).

Questions for authors
  1. How much of HERA's improvement comes from its suggested starting point? For Rover, Ant, and HalfCheetah, could you report the result after this starting point and how much further the search improves? I would especially like to see a comparison where a standard optimizer starts with exactly the same observations as HERA, including its suggested point, and receives the same evaluation budget. This would help me understand whether HERA's later decisions provide a meaningful advantage.

  2. Could you explain more clearly why HERA performs better than the simpler controllers? Is the improvement mainly due to choosing better optimization methods, changing their settings, restarting them, or deciding how long to run them? If the experiments cannot separate these effects, I think the conclusions should be limited to the complete systems being compared.

  3. Could you clarify the definitions of the synthetic problems? The main text describes Rosenbrock as a 10-dimensional problem extended to 300 dimensions, but Table 3 describes a full 300-variable problem. Which one was actually used? I have a similar question about whether Ackley uses 10 or 200 variables after the mixing step. This matters because the paper compares methods that make different assumptions about the structure of the problem.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.