acceptodds
Under review as a conference paper at ICLR 2027

Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch

Abstract

Bayesian optimization (BO) is the standard tool for sample-efficient gradient-free optimization through uncertainty-aware search driven by generic statistical priors. Richer domain priors can improve BO, but encoding them through, e.g., tailored kernels is difficult. LLMs can help bring these priors into the optimization, drawing them directly from natural language, code, and documentation. However, existing LLM-based BO methods either insert the LLM into a fixed role, surrogate, acquisition proxy, or configuration interface, or hand it broad control, sacrificing the systematic exploration that makes BO reliable. We introduce agentic Bayesian optimization: a paradigm in which an LLM agent controls the BO loop while a Bayesian backend holds the probabilistic surrogate. The agent selects evaluations and can revise the strategy during the run by tightening bounds, proposing targeted evaluations, or reframing the problem as evidence or instructions change. We instantiate this idea in Sara, a surrogate-augmented autoresearch agent, and lenz, a modular BO backend that the agent can inspect and modify through a structured interface. Across synthetic and real-world benchmarks, Sara preserves the reliability of standard BO without prior knowledge, outperforms LLM-based baselines, and uses natural-language priors to improve beyond standard BO. Furthermore, in dynamic settings, Sara can reconfigure the full optimization problem on the fly as requirements change; a capability not available in standard BO.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Bayesian optimization, autoresearch, black-box optimization, agentic optimization, natural-language priors

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_PHgoOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes agentic Bayesian optimization, where an LLM agent runs the BO loop and a GP backend does the modeling and acquisition. Before each evaluation, the agent can probe the surrogate, ask it for proposals, or change the bounds, acquisition function, objectives or constraints. Then it decides which point to evaluate. The authors frame this as a metalevel decision process, with LLM-as-optimizer methods and standard BO as special cases. They implement the idea as an agent called Sara and a BoTorch-based backend called lenz. lenz keeps the full trial history, so the agent can reformulate the problem mid-run without losing data. The experiments cover three setting: the first has no prior knowledge, using synthetic functions, the second gives the agent a short natural-language description, using hyperparameter tuning and reaction-yield tasks, and the third is a cost-aware AutoML task with a wall-clock deadline. Sara is compared with standard BO and with LLM-based optimizers. It roughly matches standard BO when there is no context, does clearly better when context is available, and beats the LLM-based methods throughout.

Critical strengths
  • The core design that keeps the GP as a tool the agent calls, instead of asking the LLM to act as the surrogate, fixes the main weakness of LLM-as-optimizer methods, and the results support this. Sara beats every LLM-only baseline by a wide margin on every suite.
  • The evaluation handles LLM contamination more carefully than existing works. The authors shift optima, rename parameters, run each agent in an anonymous sandbox, and add a memorization probe on LCBench.
Critical weaknesses
  • The paper never isolates what the agent adds on top of a well-configured GP. The central claim is about run-time policy control. However, every comparison mixes the LLM's warm start, lenz's own BO defaults and the agent's actual decisions. I believe two baselines could separate these effects. The first is lenz on autopilot, where a script calls suggest and observe with no agent. The second is an LLM warm start followed by plain BO, such as Ax seeded with the LLM's opening points, or πBO with an LLM-elicited prior. Without the second baseline, the gains on the prior-informed tasks could come mostly from a few good first points, which is an effect LLAMBO and Centaur already show early in their runs. Without the first, I can't tell whether Sara's no-context win over Ax on the reaction tasks comes from the agent or from lenz's settings
  • The prior-informed results are probably inflated, for two reasons. First, the reaction benchmarks are easy for an LLM by construction. The authors simulated them and placed each optimum where textbook chemistry says it should be. For example, the Grignard optimum is cold and dry, and the Heck optimum is hot with little water. The prompt then gives the reaction's name. An LLM that knows basic chemistry can therefore guess the right region from the name alone, before it sees any data. The objective is also close to separable, so tuning one variable at a time works well, and the authors admit this in G.1. A real reaction dataset such as Shields et al. (2021) would be a much fairer test.
  • The baselines also leave out the strongest competitors. Each baseline lacks one of Sara's two advantages. Ax has a GP but never sees the problem description, while LLAMBO and Centaur see the description but have no GP. Hybrid methods such as BORA and LLINBO have both, and Table 1 lists them as Sara's closest relatives, but none of them is in the experiments. JAHS has a similar gap. Only Sara can read the training code, which shows which operation index is a convolution. The baselines see only the numbers 0 to 4. So the paper shows that Sara beats methods that are missing either the GP or the context. It doesn't show that Sara beats methods that have both.
  • The mid-run reformulation is shown as a single qualitative trajectory. A small quantitative comparison would make the claim stronger, for example against restarting multi-objective BO from scratch, or against Ax reusing the same trials. I'd also soften the claim in the abstract that this is "a capability not available in standard BO." Any BO library that keeps its data can switch from a constrained problem to a multi-objective one. What's new is that the agent makes the switch from a natural-language instruction.
  • It would also help to report the cost per run next to Ax's in the main text. Figure 23 shows up to about 200k new tokens per run, which is fine for wet-lab experiments but matters for cheaper ones.
  • These presentation issues need clarifying. The Metrics paragraph says yield objectives report the raw best value, but Table 2 shows regret, and the GP-sample rows are negative and unlabeled. Finally, there are a few typos: "exposed" (l. 256), "the the" (l. 1279), "versionn" (l. 1499), "significiant" (l. 1549), "accessable" (l. 2388) and "durration" (Figure 14).

Shields et al., "Bayesian reaction optimization as a tool for chemical synthesis", Nature, 2021.

Questions for authors
  • Could you add a lenz-autopilot baseline and an LLM-warm-start-plus-BO baseline on the reaction and LCBench tasks?
  • In the full-system runs, how often does Sara take a suggestion unchanged, edit it, or propose a point of her own? How often does she reconfigure the problem, and does any of this correlate with final regret?
  • Did the full Sara also recognize the synthetic functions, especially Ackley-10? If so, how much of her lead over Ax there comes from recognition?
  • The original LLAMBO ranks sampled candidates with an LLM-based surrogate, but Appendix D describes only direct proposal. Which version did you run?
  • Why is there no hybrid LLM–BO baseline? Those methods seem like the most natural comparison for the prior-informed tasks.
5 comments

We thank the reviewer for the detailed and constructive review. Since the review raises many points, our response is split across four comments: W1 here, W2 to W4 in the second, W5 to Q2 in the third, and the remaining questions in the fourth. New results are in W1 (controls isolating the agent’s contribution) and W3 (LLINBO as a hybrid baseline).

W1: The paper never isolates what the agent adds on top of a well-configured GP. [...]

Thank you, this is a very helpful suggestion. To clarify the setup: lenz and Ax use the same defaults for the model, ie, a Matérn GP with the same hyperpriors and a logEI-based acquisition functions as defaults. The only difference is that Ax additionally winsorizes the output. As suggested, we also ran lenz in autopilot against Ax on Ackley-10D and Hartmann-6D and observe similar performance and convergence behavior.

TaskMethodFinal regretAUC
Ackley-10DAx4.4153 [3.9955, 4.5093]9.8765 [9.4313, 11.3058]
Ackley-10DLenz autopilot with Ax Sobol init4.0674 [3.7740, 4.2684]10.2020 [9.7190, 11.3863]
Hartmann-6DAx0.0207 [0.0049, 0.0489]0.4828 [0.3284, 0.6917]
Hartmann-6DLenz autopilot with Ax Sobol init0.0192 [0.0031, 0.0853]0.5403 [0.3870, 0.6867]

(Median [25th, 75th percentile] over ten seeds; lower is better.)

We would also frame the comparison of Sara (no context) and Ax on the reaction tasks as essentially the same performance when averaged over the tasks. A pattern we do see is that Sara tends to make more suggest calls restricted to a region around the incumbent towards the end of a run (see also Figure 5), which lets Sara fine-tune the solution. This likely explains the slightly better final performance on the two yield tasks, since these have a single global optimum for which fine-tuning an incumbent is a good strategy.

We nevertheless agree that the ablation can isolate the agent’s contribution more cleanly. We therefore ran the suggested controls on the reaction and LCBench tasks: lenz on autopilot, and Sara’s first five evaluations followed by either lenz or Ax (Sara prefix + lenz/Ax), which separates the effect of good first points from the agent’s decisions during the run. All three reach a lower final median than Sara on all four tasks, as shown in the table below. Autopilot is considerably worse on the yield tasks. The Sara prefix recovers most of this gap on Dionis and the yield tasks, but not on Covertype, and still falls short on the yield tasks. The first points therefore explain a large part of the benefit, but not all of it. To understand where the remaining benefit comes from, we examined Sara’s behavior further and saw the local fine-tuning around the incumbent described above. To test this, we designed an additional, deliberately simple baseline that switches to local refinement around the incumbent after 50% of the evaluations. We devised it on Covertype and kept it fixed for the other three tasks. This heuristic matches or exceeds Sara’s median on all four tasks, which confirms where the remaining gain comes from. We want to be clear that this baseline was derived and constructed from Sara’s own behavior. The controls one would specify in advance, autopilot and a warm start followed by BO, both fall short, whereas the agent arrives at this strategy on its own without being told what to do.

BaselineCovertypeDionisHeckBuchwald
lenz autopilot−0.46−0.83−8.09−3.72
Sara prefix + Ax−0.54−0.18−2.02−0.57
Sara prefix + lenz−0.43−0.04−1.00−0.75
Sara prefix + winsorized lenz + local refinement after 50% of trials−0.19+0.23+0.08+1.62

(Final median minus Sara's final median in percentage points over ten seeds; positive means better than Sara.)

Finally, Centaur+, which uses the LLM for a warm start, improves markedly early on but plateaus at a worse final value than Sara. The bash-only and evaluate-only ablations additionally separate the surrogate backend from reasoning in context. We will include all new results in the final version.

W2: The prior-informed results are probably inflated [...] A real reaction dataset such as Shields et al. (2021) would be a much fairer test.

Thank you for the comment, and we agree with the observation. Note, however, that this knowledge is shared by all LLM baselines. They receive the same description and use the same LLM, so if the LLM could pin down the optimum from the name alone, all LLM-based baselines would one-shot it. They do not. LLAMBO and Centaur find the right regime quickly but fail to refine it, whereas Sara can use the probabilistic backend to fine-tune the solution towards the maximum yield. Leveraging such textbook knowledge is exactly what we wanted to highlight. Embedding it in standard BO is not trivial, while an LLM in the loop can use it directly, and combining it with a BO backend gives the best final performance. We also want to note that LCBench is a real benchmark where the prior is only generic (model family and parameter scales) and there is no textbook answer to recall as also shown by the memorization probes in Figure 11.

We agree that a real reaction dataset would be an interesting test. We did not include one because it is not central to our contribution. We want to show that the agent can effectively derive and use priors, not that it is necessarily a good chemist. The simulated tasks give us a controlled setting with known optima to effectively compare performance.

W3: The baselines also leave out the strongest competitors. [...]

We agree that a comparator combining a GP with language-model context is important. We added LLINBO-Justify [R1] on Covertype, Dionis, Heck and Buchwald, using the same base LLM as Sara (Opus 4.8 on high reasoning effort). LLINBO compares an LLM proposal against a GP-UCB alternative using its acceptance rule. We evaluated two initialization conditions: LLINBO with its own five LLM-generated opening points, and LLINBO supplied with each Sara run’s exact first five evaluated configurations and outcomes. Both conditions use ten seeds per task and the same total number of max. evaluations. The table reports best observed validation accuracy (LCBench) or yield (reactions), in percent, as median [25th, 75th percentile]; higher is better.

TaskSaraLLINBOLLINBO: Sara prefix
Covertype74.20 [74.07, 74.25]73.57 [73.45, 73.65]73.56 [73.48, 73.65]
Dionis85.22 [85.07, 85.29]84.87 [84.75, 85.05]85.07 [84.94, 85.20]
Heck87.12 [81.51, 87.49]83.85 [81.51, 85.71]85.12 [84.64, 85.81]
Buchwald90.15 [89.98, 91.75]89.14 [88.30, 89.53]89.39 [89.04, 89.76]

Sara achieves the highest median on all four tasks, though the interquartile ranges overlap on Dionis and Heck. Giving LLINBO Sara’s first five evaluations improves its median on three of the four tasks, which shows that the starting points resulting from Sara’s agentic multi-step reasoning are also helpful in this framework. It does not close the gap, however, suggesting that Sara’s advantage is not due to initialization alone. We thank the reviewer for suggesting this baseline and will add these results to the final version!

W4: The mid-run reformulation is shown as a single qualitative trajectory. [...] I'd also soften the claim in the abstract [...]

With this example we mainly wanted to showcase how the optimization can be steered effectively by a human (or another agent), and we therefore opted for a qualitative showcase. The key aspect is that the reformulation is done automatically by the agent from a high-level natural-language instruction, and no human has to operate the BO library. This covers both new information or context (“I asked my colleague and he said that for these types of models values around 0.1 usually work best”) and changed requirements as in the example. Having the agent do this reconfiguration through the BO backend also adds a level of robustness, since all existing observations are carried over consistently instead of being re-specified by hand. One could of course let an LLM operate a standard BO library on behalf of the human, but this would essentially arrive at what we propose. We agree that the wording in the abstract does not make this distinction clear, and we will rephrase it in the final version. We see this example as a demonstration of a capability and not as a performance claim, so we would keep it as a qualitative showcase.

W5: It would also help to report the cost per run next to Ax's in the main text. [...]

Yes, that is a good idea! We were under space constraints and therefore moved the discussion of computational cost to the appendix. We will state in the main text that running an optimization with an LLM in the loop comes with additional token costs (up to roughly 200k new tokens per run) and a small latency from deliberation. For applications where a single evaluation is expensive, such as wet-lab experiments or training runs, this overhead is small relative to the cost of an evaluation. You are right that the tradeoff should be explicit, and we will report the cost next to Ax’s.

W6: These presentation issues need clarifying. [...]

Thank you very much for catching these, we will fix the typos and clean up the metrics description in the final version. For clarification, we do not know the true optima for the GP samples and therefore report the best value there. We will label this in the main text and make the metric description consistent with Table 2.

Q1: Could you add a lenz-autopilot baseline and an LLM-warm-start-plus-BO baseline on the reaction and LCBench tasks?

Yes, we ran both (see W1) and will include the results in the final version.

Q2: In the full-system runs, how often does Sara take a suggestion unchanged, edit it, or propose a point of her own? How often does she reconfigure the problem, and does any of this correlate with final regret?

Thank you for the question. We characterize the agent’s behavior at the level of tool usage. Figure 5 shows how the different lenz calls evolve over normalized trial progress, and the full per-seed sequences of tool calls are provided in the appendix in Figures 24 and 25. These show that Sara combines the different backend calls in varying ways across seeds and tasks, and that she tends to shift towards local refinement around the incumbent in the later stages of a run. As discussed in W1, this refinement can be critical for reaching the best final performance.

To quantify this, we audited the ten Covertype runs, the task from W1 on which Sara still reaches a slightly higher median than the post-hoc heuristic. 25% of the evaluated configurations are lenz suggestions taken unchanged, while 75% are edited or proposed by Sara herself. In all ten seeds, the configuration that first reached the final best value was not an unchanged suggestion. Regarding reconfiguration, the only full reconfiguration of the problem is the mid-run reformulation example. There are also smaller reconfigurations at the level of the BO backend, namely changing the acquisition function and changing the bounds from which the next proposal is generated, which are visible as set-acqf and set-bounds calls in the traces. Sara made such a change persistently in 4/10 seeds for the bounds and in 4/10 seeds for the acquisition function for the Covertype runs. In addition, she used non-persistent changes for individual suggest calls, e.g., suggest --around the incumbent in 9/10 seeds, which allow local evaluations without changing the backend configuration, as well as acquisition override for an individual suggest call in 6/10 seeds.

Note, however, that these numbers are descriptive. Since Sara’s actions depend on all outcomes observed so far, attributing final regret to individual decisions is difficult, as it already is in standard BO. These numbers are also specific to Opus 4.8 at high reasoning effort with our system prompt. As discussed in Section G.4, the way Sara interacts with lenz differs between models, e.g., GPT 5.5 follows the trial-by-trial instruction more closely, so we would expect these counts to change with the underlying LLM.

Reviewer_fjLgOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Sara, an AI agent that uses Bayesian optimization to find good solutions. It works with a tool called lenz, which keeps track of previous experiments, makes predictions, and suggests what to try next.

Unlike a regular Bayesian optimizer, Sara can use information from written descriptions, inspect predictions, choose its own experiments, and change the search strategy while the experiments are running.

The authors test the system on mathematical benchmarks, machine learning hyperparameter tuning, simulated chemical reactions, and neural network architecture search. The results show that Sara performs about as well as standard Bayesian optimization when no useful background information is available. When useful information is provided, Sara often performs better. The paper also shows how Sara can change the optimization goal without losing earlier results.

Critical strengths

1. The paper combines AI reasoning with Bayesian optimization in a useful way.

I like how the authors separate the work between Sara and lenz. Sara uses background knowledge and decides what to try, while lenz handles predictions and uncertainty. This lets Sara use what it already knows without giving up the benefits of Bayesian optimization.

I also find the ability to change the optimization problem during a run useful. For example, Figure 6 shows Sara changing from finding the best model under a compute limit to exploring the trade-off between model quality and compute. It can make this change without throwing away earlier experiments. This could be useful in real research, where requirements often change.

2. The experiments provide good evidence that the system works.

The authors test Sara on different problems, repeat experiments across ten seeds, and compare it with several existing methods. I especially appreciate that they test new random GP functions, since language models might already know the standard mathematical benchmarks.

The experiments also help explain why the system works. For example, Table 8 shows that giving Sara access to both training code and runtime predictions reduces the average test error to 9.14%, compared with 11.42% when it has neither. Using only one of these sources gives smaller improvements.

These results suggest that combining background knowledge with numerical predictions is genuinely useful.

Critical weaknesses

1. I am not sure whether the LLAMBO baseline was implemented in the same way as the original paper.

Appendix D describes LLAMBO as giving the language model previous results and asking it to suggest the next configuration.

However, the original LLAMBO method does more than this. It generates possible configurations, uses a language model to predict their performance, and chooses the next configuration using expected improvement (Liu et al., ICLR 2024).

It is unclear whether the authors used this full method or a simpler version. This might just be a missing detail in the explanation, rather than a problem with the experiments.

I think this matters because LLAMBO is an important baseline. The authors should make clear exactly how they implemented it so that the comparison can be properly judged.

2. It is difficult to tell how much of Sara's improvement comes from the agent making decisions throughout the search.

One important idea in this paper is that Sara can keep guiding the optimizer instead of only providing a good starting point.

However, some of the improvements might come simply from using the language model to choose better starting configurations. For example, Ax begins with five Sobol evaluations, while Sara can use background knowledge from the first evaluation.

The authors include a version of Centaur that starts with a language-model suggestion, which is helpful. However, Centaur uses a different optimization method.

I would like to see a comparison where both methods start with the same language-model suggestions and use the same Bayesian optimization backend. One would then continue with normal Bayesian optimization, while the other would let Sara keep making decisions.

The existing experiments show that Sara benefits from having optimization tools. However, this additional comparison would help show how much benefit comes specifically from giving Sara control throughout the search.

Questions for authors

1. Could you clarify how LLAMBO was implemented?

Did you use the full method from the original paper, including generating candidates, predicting their performance, and selecting the next point using expected improvement?

If you used a simpler version, could you explain what was changed and how that affects the comparison?

2. How much improvement comes from Sara continuing to guide the search?

Have you compared Sara with a regular Bayesian optimizer that starts with the same language-model suggestions and uses the same lenz backend?

This would help distinguish the benefit of choosing good starting points from the benefit of letting the agent guide the search throughout the experiment.

4 comments

We thank the reviewer for the thorough review and their helpful questions. We address the main weaknesses and questions below and in the next comment, including some critical clarifications on the LLAMBO baseline.

Regarding W1 and Q1:

Thank you for raising this, and you are right. The results in the paper use a direct-proposal version of LLAMBO, in which the LLM receives the problem context and the trial history and directly proposes the next configuration, as described in Appendix D. This variant is called LLAMBO-light in the LLINBO paper [R1], where it is introduced as a lightweight alternative to the costly end-to-end method. Interestingly, in their results this light version outperforms the end-to-end LLAMBO version on several benchmarks (their Figure 3). It is nevertheless a simplification of the original method, which additionally predicts the performance of sampled candidates with the LLM and selects the next point using expected improvement, and we should have stated this difference explicitly.

As part of this rebuttal, we tried to rerun the full end-to-end LLAMBO version with Opus 4.8 (high reasoning effort) for a fair comparison across baselines. However, we ran into the issue of full LLAMBO quickly flatlining, which we attribute in part to collapsed predictive distributions. We saw that the repeated LLM predictions for a candidate are nearly identical, so the LLM-based surrogate provides almost no uncertainty estimate. To increase numeric stability, we also moved to a logEI scoring, which prevented an underflow issue we were having with these collapsed distributions, but this did not prevent the quick flatlining either. It is also considerably more expensive: on Heck, a single end-to-end LLAMBO run processed about 21.9M tokens, compared to about 0.13M for LLINBO (more on this below) and about 0.36M for Sara. Since Sara runs as a single persistent conversation, 0.31M of its tokens are cheap prompt-cache reads (approx 0.04M excluding them, cf. Fig. 23). Note that these are preliminary results with a different LLM than in the original paper. We will therefore keep the direct-proposal version as the baseline and update the legends to LLAMBO-light and the description in Appendix D in the final version.

To give a stronger LLM-based comparison beyond LLAMBO, we additionally ran LLINBO-Justify [R1] on Covertype, Dionis, Heck and Buchwald, with the same base LLM as Sara. LLINBO builds on LLAMBO-light as its LLM proposer and combines it with a GP, which accepts or rejects each LLM proposal against a GP-UCB alternative. It is therefore a direct and more recent successor of the baseline in question, and in their paper it outperforms both LLAMBO-light and end-to-end LLAMBO. We ran it in two settings: with its own five LLM-generated initial points, and with the exact first five evaluations of each Sara run, so that both methods start from the same data. The table reports the best observed validation accuracy (LCBench) or yield (reactions) in percent, as median [25th, 75th percentile] over ten seeds; higher is better.

TaskSaraLLINBOLLINBO: Sara prefix
Covertype74.20 [74.07, 74.25]73.57 [73.45, 73.65]73.56 [73.48, 73.65]
Dionis85.22 [85.07, 85.29]84.87 [84.75, 85.05]85.07 [84.94, 85.20]
Heck87.12 [81.51, 87.49]83.85 [81.51, 85.71]85.12 [84.64, 85.81]
Buchwald90.15 [89.98, 91.75]89.14 [88.30, 89.53]89.39 [89.04, 89.76]

Sara reaches the highest median on all four tasks, though the interquartile ranges overlap on Dionis and Heck. Giving LLINBO Sara’s first five evaluations improves its median on three of the four tasks but does not close the gap. This also relates to your second question below: even with the same starting points and a GP in the loop, Sara’s continued guidance of the search still makes a difference. We will add these results to the final version. Thank you!

Regarding W2 and Q2:

Thank you, this is a very good question, and we ran exactly this comparison. First, lenz uses the same defaults as Ax for the model and the acquisition function, with the only difference that Ax additionally winsorizes the output. Running lenz autonomously gives comparable performance to Ax in both final regret and convergence speed, as shown in the table below, and we will add this comparison to the paper. Differences in performance are therefore due to having the agent in the loop and not to differences in the implementation of the BO primitives.

TaskMethodFinal regretAUC
Ackley-10DAx4.4153 [3.9955, 4.5093]9.8765 [9.4313, 11.3058]
Ackley-10DLenz autopilot with Ax Sobol init4.0674 [3.7740, 4.2684]10.2020 [9.7190, 11.3863]
Hartmann-6DAx0.0207 [0.0049, 0.0489]0.4828 [0.3284, 0.6917]
Hartmann-6DLenz autopilot with Ax Sobol init0.0192 [0.0031, 0.0853]0.5403 [0.3870, 0.6867]

(Median [25th, 75th percentile] over ten seeds; lower is better.)

Second, you rightly note that Centaur+ is not a clean test of this, since it also uses a different optimizer. To separate the starting points from the continued guidance with the same backend, we compare against “Sara prefix + lenz”. It uses Sara’s first five evaluations and then continues with lenz autonomously, with no agent. We include the same control with Ax as the backend (Sara prefix + Ax), as well as lenz autopilot, ie, standard BO without LLM-chosen starting points.

BaselineCovertypeDionisHeckBuchwald
lenz autopilot−0.46−0.83−8.09−3.72
Sara prefix + Ax−0.54−0.18−2.02−0.57
Sara prefix + lenz−0.43−0.04−1.00−0.75
Sara prefix + winsorized lenz + local refinement after 50% of trials−0.19+0.23+0.08+1.62

(Final median minus Sara's final median in percentage points over ten seeds; negative means worse than Sara.)

Sara prefix + lenz recovers a large part of the gap on Dionis and the yield tasks, but not on Covertype, so good starting points are an important factor. It still reaches a lower median than Sara on Covertype and the yield tasks, which indicates that the agent continuing to guide the search adds further benefit. To understand where this remaining benefit comes from, we examined Sara’s behavior and saw that she tends to shift to local refinement around the incumbent in later stages (see Figure 5 and the traces in the appendix). We confirmed this with an additional diagnostic baseline that hard-codes this late local refinement on top of Sara prefix + lenz, which closes the gap to Sara (see also our response to Reviewer PHgo, W1). Importantly, the agent, however, arrives at this strategy on its own without being told what to do. We will add these comparisons to the paper.


References

[R1] Chang, Chih-Yu, et al. "Llinbo: Trustworthy llm-in-the-loop bayesian optimization." arXiv preprint arXiv:2505.14756 (2025).

Thank you for the detailed responses and new experiments.

The LLAMBO clarification resolves my first concern. Please label the baseline as LLAMBO-light clearly throughout the paper. The new LLINBO results are also a useful addition.

The matched-start experiments address my second concern. They show that good initial points explain much of Sara's advantage, although continued agent guidance still helps on some tasks.

The fixed-rule baseline is particularly informative: it matches or outperforms Sara on most tasks without continued agent control. However, it combines winsorization and local refinement, so their individual effects remain unclear. Also, Sara's prompt already encourages local refinement, so I would avoid claiming that the agent discovered this strategy independently. Please also report variation across seeds for the new controls.

Overall, the new results strengthen the paper and address my main questions. I still see a useful contribution in combining language-based reasoning with Bayesian optimization, especially when requirements change. However, the results do not establish a clear advantage over simpler search strategies. Therefore, I'd like to keep my score.

Thank you for the follow-up and for engaging with the new results. With the discussion deadline approaching, we wanted to respond to your remaining points below.

Winsorization vs. local refinement. The ablations we posted already separate the two. Ax differs from lenz only by winsorizing the outcomes, so comparing "Sara prefix + Ax" with "Sara prefix + lenz" isolates the effect of winsorization. It does not help on three of the four tasks (−0.54 vs. −0.43 on Covertype, −0.18 vs. −0.04 on Dionis, −2.02 vs. −1.00 on Heck) and helps slightly on Buchwald. This indicates that the gain of the fixed-rule baseline comes from local refinement.

Whether the agent discovered the strategy independently. You are right, and we will correct our wording. The system prompt names fine-tuning the incumbent as one use of the remaining budget, alongside reopening the full space to check for a better basin. How to weigh the two, and when and where to refine, is left to the agent, and importantly we do not prescribe any schedule or trigger point for it. We will describe it this way in the paper and will not claim that the agent arrived at the strategy without any guidance.

On the advantage over simpler strategies. We agree with your reading of the results and will state it plainly in the paper. Good starting points explain much of Sara's advantage, and on these four tasks a heuristic such as the fixed local-refinement rule can recover the rest. We would add two remarks. First, all matched-start ablations, including the fixed-rule baseline, begin from Sara's first five evaluations, which the agent chose over several rounds of reasoning about the problem context. On the 6D reaction tasks with a budget of 40 evaluations, five well-placed points are already a strong warm start, so these ablations inherit a substantial part of the agent's contribution and are not agent-free alternatives. Interestingly, the LLINBO comparison points in the same direction. Replacing its own LLM-generated starting points with Sara's improved its median on three of the four tasks, which already indicates a small advantage of an agent that chooses its initial points over several rounds of reasoning in one continuous context. Without that prefix, lenz on autopilot falls well short. Second, the fixed rule itself was derived from Sara's traces and selected on Covertype. Both Sara and this rule reach higher medians than existing baselines such as LLINBO on all four tasks, so the rule is not a weak baseline.

In the final version, we will also label the baseline as LLAMBO-light throughout, report the median and interquartile range over seeds for all new ablations, and make sure to use appropriate wording on local refinement as described above.

Lastly, we would like to take this opportunity to restate, also with the upcoming discussion in mind, that we see the main contribution of the paper in the general framework of agentic BO, in which an LLM agent controls the optimization loop while a principled Bayesian backend provides the surrogate and the BO primitives. All three reviews, including yours, name this core idea as a strength, and one explicitly expects future work to build on the framework. Sara and lenz are one instance of the framework, and the specific behaviors discussed above reflect this instance and our design choices. Already in this form, the instance matches standard BO when no context is available, improves on it when context is given, and compares favorably with LLM-based baselines, including the hybrid method LLINBO. The ablations from this discussion make it clearer where these gains come from, and we will make sure to state this explicitly in the paper. We believe the framework is relevant for autonomous research pipelines and other practical applications, and that it can be a promising basis for further research.

Reviewer_VugAOpenReview ↗
Rating
4(+2)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This work proposes the agentic bayesian optimization framework, which equips an LLM agent with subroutines from bayesian optimization. The agent can interact with the bayesian optimization backend and based on this interaction commit to evaluating a configuration. Experiments are performed on synthetic functions, 3 LCBench tasks, 4 Yield tasks, and one of the three JAHS-Bench-201 tasks.

Critical strengths

The core idea of an LLM agent controling the optimization process with access to subroutines from Bayesian optimization is novel and of interest to many ICLR attendees. I expect future work to build on this framework.

Critical weaknesses

The evaluation is weak on several fronts, the small amount and unclear choice of benchmarks is the most critical. Especially the choice of running on only one of the three JAHS-Bench tasks and not noting so in the main paper may look like cherry picking. More details below.

  • Besides the synthetic test functions, only LCBench (3 benchmarks), Yield (4 benchmarks), and one task from JAHS-Bench are considered. On LCBench, for example, contains a lot more benchmarks, why only focus on these 3 and how were they chosen? On JAHS Bench, why only run on CIFAR-10 and not the other 2, and why not explicitly say so in the main paper and simply omit it?
  • The effect of the prior in the LCBench benchmarks is unclear. What if the prior is misleading? How would the methods perform if no prior is given?
  • Sensitivity to prompt and interface design is listed as a key limitation, the authors report how performance is dependent on this, but do not present any results on this.
  • I have some open questions, answers to which may elicit additional baselines or experiments
Questions for authors
  1. What does "budget" refer to? Does your, or any other, method have access to lower-cost evaluations (similar to how multi-fidelity approaches may have)? What do the maximum budgets correspond to?
  2. How was the "fixed wall-clock" deadline chosen for JAHS-Bench, how does the comparison look like for different values for this? Why not show Figure 14a in the main paper instead of 14c, it seems more informative to me and also favours your method?
  3. Why does the line for Sara only start at 12h (Figure 14a)?
  4. What effect does the base LLM have on the results of the LLM-based baselines?
  5. Your agent decides when to stop optimization. How is this treated in the baselines?
  6. Why does your problem setting cover multi-objective and constrained settings? Do our experiments cover these?

Post-rebuttal update: The authors have largely addressed my soundness concerns around their evaluation. I remain convinced of the novelty and high interest to many ICLR attendees and recommend acceptance.

In addition to the changes the authors promised, I suggest they should add two points they make in their rebuttal to their discussion on limitations: 1. Not studying robustness against misleading context and 2. a point around their statement on "To end up with the system prompt and tool descriptions provided in the paper, we iterated on them during development until we reached satisfactory performance".

I read the other reviews, but did not carefully check the corresponding rebuttals or points raised.

4 comments

We thank the reviewer for the thorough review and their helpful questions. We address the main weaknesses and questions below and in the following two comments with new results in W1 and W2.

W1: Besides the synthetic test functions, only LCBench (3 benchmarks), Yield (4 benchmarks), and one task from JAHS-Bench are considered. On LCBench, for example, contains a lot more benchmarks, why only focus on these 3 and how were they chosen? On JAHS Bench, why only run on CIFAR-10 and not the other 2, and why not explicitly say so in the main paper and simply omit it?

Thank you for the comment. We realize that this was not stated clearly in the paper, neither in the main body nor in the appendix. Given that we were already quite space constrained and benchmarking with an LLM in the loop can be time consuming and expensive in terms of tokens, we decided to focus on the most interesting LCBench benchmarks. To decide where to spend this budget, we first compared Ax and Sobol on the LCBench tasks, since Sobol sampling was already very competitive on many of them. Dionis and Covertype showed the largest gaps in this screen and were selected for that reason. Airlines was added as a third task. Importantly, the selection was made independently of our method’s performance. On JAHS, we focused on CIFAR-10, which we should have stated explicitly. Below, we can additionally report archived experiments on all three JAHS-Bench-201 datasets. As in the paper these cover ten seeds per method and dataset, under a 48-hour simulated-training-time budget with four asynchronous workers, at most 80 launches, and evaluations at 200 epochs. The table reports the best observed validation accuracy in percent, as median [25th, 75th percentile] across ten seeds; higher is better.

JAHS taskSobolAxSara: code access, no runtime history
CIFAR-1086.82 [85.22, 87.72]85.63 [82.45, 88.92]89.26 [89.23, 90.73]
Fashion-MNIST93.95 [93.67, 94.27]93.81 [93.14, 94.29]94.78 [94.43, 95.10]
Colorectal histology94.81 [94.34, 95.28]95.52 [95.33, 95.75]95.75 [95.64, 96.05]

Note that in these runs Sara had access to the benchmark source code, which Ax and Sobol cannot use by design. The comparison should therefore be read as Sara with this additional context against black-box baselines, not as a like-for-like black-box comparison. In the final version, we will describe this selection process in the appendix and state it explicitly in the main body with a pointer to the appendix.

W2: The effect of the prior in the LCBench benchmarks is unclear. What if the prior is misleading? How would the methods perform if no prior is given?

Thank you for the question. To clarify, we do not provide an explicit prior as in piBO or interactive BO, but only implicit priors in the form of natural language context (e.g., parameter 1 is the learning rate, parameter 2 is the layer depth, the task is a funnel-shaped NN), from which the agent has to derive its own prior to steer the optimization. The setting without a prior corresponds to our no-context variants, where we do not specify what the tuning knobs correspond to. To make this comparison explicit, we ran Sara with and without context on Covertype and Dionis in a paired rerun, using Opus 4.8 with high reasoning effort, ten runs per condition, and 80 evaluations per run. Since these are fresh runs, the values with context differ slightly from the Sara results reported elsewhere. The table reports the best observed validation accuracy in percent, as median [25th, 75th percentile]:

TaskContextNo context
Covertype74.08 [73.63, 74.20]73.56 [73.23, 73.73]
Dionis85.21 [84.87, 85.28]80.19 [73.70, 84.54]

Sara with context reaches a higher median on both tasks, with a large gap on Dionis. These results are also reflected in the aggregate performance over the three considered LCBench tasks, as reported e.g. in Figure 18 d) (blue lines), and for various base LLMs, including GPT models and open-weight models, in Figure 22 d). We generally agree that robustness to misleading context is an interesting question, but we have not investigated it. Our hypothesis is that the probabilistic backend could help to recover more quickly than pure-LLM approaches, because it is grounded in the actual observations and can help falsify the derived hypothesis. Making this recovery reliable might require additional changes to the system prompt. We see how to further robustify the framework against misleading priors, with an LLM agent as the main orchestrator of the optimization loop, as an interesting direction for future work.

W3: Sensitivity to prompt and interface design is listed as a key limitation, the authors report how performance is dependent on this, but do not present any results on this.

To end up with the system prompt and tool descriptions provided in the paper, we iterated on them during development until we reached satisfactory performance. However, as we also state in the paper, there are likely modifications that make the system prompt more performant or more token efficient. In Table 10, we list the core directives together with the agent behavior we observed during development that motivated each of them. We agree that this is qualitative and that we do not present a controlled study of prompt sensitivity. What we do show is that the resulting system works well under the same system prompt across models, ranging from (at the time) frontier models to open-weight models in Section G.2. As the models change, these directives may need to change as well. Instead of deriving the best system prompt and interface which may overfit to a specific LLM model, the core objective of this paper was to state the general framework of agentic BO, which we believe can emerge as a core component in future black-box optimization, and to provide an already useful instantiation of it to practitioners. To give additional guidance on how to modify the system prompt and the overall framework for a specific use case, we also included the discussion on recommendation for practitioners.

Q1: What does "budget" refer to? Does your, or any other, method have access to lower-cost evaluations (similar to how multi-fidelity approaches may have)? What do the maximum budgets correspond to?

Budget essentially refers to the number of evaluations relative to the total number of allowed evaluations. Especially for the synthetic benchmarks, we had to find a way to visualize the aggregate performance of all methods across all benchmarks, which have different maximum numbers of evaluations depending on the dimensionality of the problem, as shown in the per-problem result figures in Figure 10. We therefore decided to show progress as the percentage of used evaluations relative to the total number of evaluations. Importantly, all baselines always have the same maximum number of allowed evaluations to ensure a fair comparison. None of the methods, ours or the baselines, has access to lower-cost evaluations. All of them treat evaluations as single-fidelity black-box queries. We see how this framing is slightly confusing, and we will make it more explicit in the final version.

Q2: How was the "fixed wall-clock" deadline chosen for JAHS-Bench, how does the comparison look like for different values for this? Why not show Figure 14a in the main paper instead of 14c, it seems more informative to me and also favours your method?

We chose 48h as a deliberately tight deadline, since this is the regime where deciding what to run matters most. We have not evaluated other deadlines. We would expect the gap to shrink for longer horizons, as all baselines can run more experiments. At the tight 48h deadline, the “pure GP” based baselines cannot gather enough information to fully inform their surrogate for an efficient search. This shows that having an agent in the loop that leverages a surrogate over the predicted runtimes yields substantially better performance. All baselines share the same deadline, and CarBO uses the same surrogate to estimate runtimes, so we believe the comparison is still valid. We chose Figure 14c out of preference, as it better highlights the final performance. Figure 14a would need a short additional explanation of the different “starts” of the baselines (see your next question), so we picked the plot with the same main takeaway that would prompt readers to look at the full results in the appendix. We do agree that Figure 14a is more informative and are happy to move it to the main body.

Q3: Why does the line for Sara only start at 12h (Figure 14a)?

The start of the line marks the point where all seeds have at least one completed trial. With Sara in the “full information” setting, one seed chose an initial batch of configs whose first result only arrives after approx. 12 simulated hours. Therefore, the plotted median and IQR only start there. We will state this more clearly in Appendix F. Thank you!

Q4: What effect does the base LLM have on the results of the LLM-based baselines?

We did not ablate the base LLM for the LLM-based baselines, so we cannot say how their performance changes with a different model. To keep the comparison controlled, we used the same LLM (Opus 4.8) for Sara and all LLM-based baselines in every cross-baseline comparison. For Sara, we additionally ablate the base LLM, including GPT models as well as different open-weight models in Figure 22.

Q5: Your agent decides when to stop optimization. How is this treated in the baselines?

By default, all baselines run for the full specified maximum number of evaluations, and all methods are compared at the end of this budget. When Sara stops early, we carry her best value forward to the end of the budget. Stopping early can therefore never help Sara in the comparison, since she only gives up evaluations that the baselines still use. In our experiments, when Sara stops early, she does so close to the end of the budget (at the earliest after 72 of 80 evaluations on the LCBench tasks). For Sara in practical applications, early stopping can be beneficial. If the agent is certain that there is nothing left to optimize, it may be desirable to run fewer than the maximum number of evaluations (which could e.g., correspond to full training runs) to save resources. We include the stopping decision for completeness of the agentic BO framework, but do not investigate it further here, and we will state this explicitly in the paper.

Q6: Why does your problem setting cover multi-objective and constrained settings? Do our experiments cover these?

Multi-objective and constrained problems are common in practice, and we include them to demonstrate the flexibility of the full framework. The mid-run reformulation in the main paper in Section 6.2 includes a reformulation to a constrained MO problem. In Appendix E.1 in Figure 10 f), we additionally include a multi-objective benchmark on GP samples of different dimensionalities. As part of the synthetic benchmarks, we also include constrained Hartmann as a classic constrained BO problem.

Dear reviewer VugA, please let us know if our comments address you points and whether you have any additional questions that we can answer before the end of the response period (today). Thank you.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.