acceptodds
Under review as a conference paper at ICLR 2027

Bayesian Optimization with Rich Auxiliary Information via LLMs

Abstract

Bayesian Optimization (BO) is widely used for optimizing expensive black-box functions, yet it typically reduces each expensive experiment to an input and its objective value, discarding much of the information the experiment produces. Such auxiliary information can include training curves in hyperparameter optimization, expert notes and images in scientific experimentation, or known facts about the specific optimization problem. Leveraging the ability of large language models (LLMs) to process diverse and unstructured information, we introduce two methods: AuxBO-Evolve, which uses auxiliary information to evolve beliefs over the location of the optimum, and AuxBO-Vicinity, which uses it to locally guide the acquisition function. Across synthetic tasks, hyperparameter optimization benchmarks, and a nuclear fusion task with unstructured scientist-written logs, our methods consistently improve optimization performance and outperform standard BO and existing LLM-based approaches. We further find that LLMs provide more effective probabilistic guidance when modeling likely maximizer locations rather than objective values pointwise. Overall, our results demonstrate that exploiting rich information beyond standard interactions can substantially improve BO performance.

Primary area:
optimization
Keywords:
Bayesian optimization, LLMs, black-box optimization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Hs3AOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how LLMs can enable standard BO to leverage auxiliary information commonly available in optimization tasks. It proposes AUXBO-EVOLVE and AUXBO-VICINITY, which use LLMs to propose promising configurations and rank candidate configurations, respectively. The methods are evaluated on AutoML and scientific optimization tasks.

Critical strengths

The paper has a clear motivation and addresses a common setting in optimization tasks.

Critical weaknesses
  1. Limited novelty. LGBO, proposed in Unleashing LLMs in Bayesian Optimization: Preference-Guided Framework for Scientific Discovery (ICLR 2026), also uses LLMs to identify preferred configurations, which are then used to raise the GP mean. The approaches appear similar, while LGBO offers a simpler formulation. The methodological advances over LGBO need to be clarified.
  2. Insufficient experimental evaluation. The paper lacks discussion of and comparisons with recent LLM-assisted BO baselines. Moreover, the proposed methods involve multiple components, but the ablation studies are insufficient to establish the necessity of each component.
Questions for authors
  1. Could the authors provide component-wise ablation studies to assess the contributions of the key design choices?
  2. Could the authors evaluate the methods in high-dimensional settings and combinatorial optimization problems?
  3. Could the authors expand both the methodological discussion and empirical comparisons to include additional BO baselines, such as BOPRO (ICLR 2025), CAKE (NeurIPS 2025), LGBO (ICLR 2026), and LABO (ICML 2026)?
  4. How sensitive are the methods to the prompt design and the choice of LLM backbone?
4 comments

Thank you for the review. First, we request you to please refer to our general official comment on “Clarification of novelty and contributions”. Here is our answers to your concerns:

  1. We discuss ablations in our general comment on “Clarification of the experimental design and findings”. The submitted paper already includes several component-level studies: as discussed, we view synthetic standard functions as an ablation study to isolate the effect of auxiliary feedback from the task prior. In the HPO bench, we do ablations with/without auxiliary feedback on the synthetic and a π\piBO comparison isolating the effect of prior knowledge from auxiliary feedback.

    Additionally we perform ranking-quality ablations for AuxBO-Vicinity in Appendix Figure 6; search space dimension ablation in Figure 9 (new); and UCB, PI, and posterior-variance comparisons in Appendix Figure 7 showing robustness across acquisition functions. These cover the major components of our algorithms. Also, we would be happy to run an additional targeted ablation if you identify a specific component not covered by the existing studies.

  2. As per our understanding, HPOBench is a combinatorial optimization problem. We also now include higher dimensional settings in our general comment “AuxBO-Evolve and AuxBO-Vicinity at higher dimensions.”

  3. Among the suggested baselines, BOPro seems to be an LLM reasoning paper only inspired by BO and there is not a clear way to use it for BO tasks. So, we omit it. All others (LGBO, LABO, CAKE) are other LLM+BO algorithms that try to exploit the prior knowledge of LLMs and do not have any study of auxiliary information. But, we acknowledge that many of these algorithms can be combined with auxiliary information. Hence, we supplied these algorithms with auxiliary information and found that they’re still underperforming AuxBO methods in the synthetic tasks. For HPOBench, so far we only added LGBO and you can see the updated HPOBench plot for results.

Now, Appendix A10 contains CAKE, LABO, and LGBO results for the HPOBench task. And Figure 5 with nuclear fusion results in the main paper is also updated with these baselines. AuxBO-Evolve outperforms all the baselines in both environments.

We add results from ablating the LLM of choice in the fusion task in Appendix A11. We rerun our method and baselines with qwen3-8b model. The results and conclusions are the same. AuxBO-Evolve performs the best independent of choice of LLM.

Reviewer_Hs3AOpenReview ↗

Thank you for your response

Thank you for the authors’ response. I still believe that this work lacks novelty, and the organization of the paper also requires substantial revision. I maintain my score.

Reviewer_qbiDOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper aims to use auxiliary information for more effective BO. The idea is to use an LLM with the auxiliary information to build a prior on the optimal solution which is then multiplied to the acquisition function (AF) for selecting the next design.

Critical strengths
  1. The paper solves an interesting and important problem of incorporating auxiliary information into BO
  2. The paper does a good job acknowledging the existing literature on LLM-BO approaches
  3. Overall the paper is easy to read
Critical weaknesses
  1. The main concern is the novelty. It is not clear what is new in the paper. AUXBO-Evolve is essentially the well known piBO (Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization) approach. The authors state that biggest difference is that the prior can be called a posterior since it uses the auxiliary information, but that is is also true for any method in human-AI collaborative BO like piBO where expert knowledge that constructs the prior is data/knowledge that we condition on. Yet the modeling approach is exactly the same.

  2. this is also true for the theory which follows directly given the posterior term is decayed by 1/t like piBO.

  3. The authors also propose AUXBO-VICINITY which is a heuristic that weights different point importance based on the LLM rank. Yet, the need for this method is not clear given that it underperforms AUXBO-Evolve by far. It also underperforms other benchmarks.

  4. How feasible it is to build the posterior over the argmax by asking LLMs in high dimensions? For each sample you need to ask for multiple points, build a GP, then find the posterior maximizer. In high dimensions the number of call grows exponentially making the method very hard to use. Having more auxiliary information will also make the cost higher here.

  5. The results in the main paper do not show a clear improvement of the method compared to what exists. For example Fig 4 shows that AUXBO_evolve is similar to LLINBO and LLINBO is better in the early start which is usually where using LLM can be helpful as it can warm start the BO process.

  6. The paper will benefit from much further experimentation and benchmarks. The area of LLM-BO approaches has become very active and many other benchmarks could be added including ones that the authors already cited in Sec 2.

Questions for authors
  1. I wonder if AUXBO-VICINITY could be related to a statistical method such as a kernel based regression where the LLM decides the weights. This can help strengthen the method.
2 comments

We thank you for the review.

  1. We request you to please refer to our general official comment on “Clarification of novelty and contributions”.

  2. While our result is inspired by π\piBO, it does not follow directly from its analysis. Hvarfner et al. compare prior-weighted EI with standard EI, using the fixed base prior and the (1/t)(1/t) exponent to show that the multiplicative distortion converges to one. Our proof instead uses a counting argument specific to the more restrictive finite, independent-arm setting to show that every arm is sampled. In fact, the assumption in line 613 that the evolving weights are uniformly bounded away from zero is sufficient; exponential decay is not required for the proof and appears in the theorem only to match the algorithm used empirically. Thus, although the acquisition rules are related, the theorem’s sufficient conditions and proof technique differ.

  3. Please refer to our general official comment “AuxBO-Evolve and AuxBO-Vicinity at higher dimensions.”

  4. We believe that we do not need to sample arg⁡max⁡\arg\max queries scaled by the search space volume as the arg⁡max⁡\arg\max belief of the LLM is by nature sparse and we are not trying to estimate a quantity uniformly well in a search space. In addition, GP building cost is the same, as our GP procedure does not depend on LLM sampling.

  5. We thank you for pointing this out. There was a mistake with LLINBO, please refer to our general comment “Correction to Figure 4”.

  6. We refer you to our general comment “Clarification of the experimental design and findings.” In addition, we now include LGBO, LABO and CAKE as additional baselines. Please refer to point 3 in our response to Reviewer Hs3A.

Regarding the kernel question.

Thank you for this observation. AuxBO-Vicinity indeed admits a kernel-regression interpretation. For each feedback event, the LLM ranks LL neighboring candidates, and we convert this ranking into labels Lpj,iL p_{j,i}. Ignoring the numerical-stability term, the ratio in Eq. (12) can be written as

qjaux(x)qj0(x)=∑iKℓ(x,zj,i)(Lpj,i)∑iKℓ(x,zj,i),\frac{q_j^{\mathrm{aux}}(x)}{q_j^0(x)}=\frac{\sum_i K_\ell(x,z_{j,i})(L p_{j,i})}{\sum_i K_\ell(x,z_{j,i})},

which is a Nadaraya–Watson kernel-regression estimate of the LLM-derived preference labels. Its logarithm defines the local correction j(x)j(x). Equation (13) then applies a second, regularized Nadaraya–Watson-type smoother to aggregate these corrections across feedback events, with shrinking the correction toward zero where nearby feedback is scarce. Thus, AuxBO-Vicinity can be viewed as a nested kernel-regression construction that converts LLM rankings into a smooth, persistent preference field used to modulate the acquisition function. We thank the reviewer for suggesting this useful statistical interpretation.

Thank you for your feedback. This is very helpful.

That said, I still believe that the novelty is a bit limited compared to \piBO. The only key change is asking the LLM for multiple maximizers which are then used for building \pi using a mixture normal with a common \sigma^2.

The AUXBO-Vicinity is interesting, but it is not superior to the benchmarks in many of the tests done. Perhaps focusing on its advantages in higher dimensions (and adding more simulations along that line) can be helpful for the paper.

Reviewer_oRgoOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes using LLMs to exploit auxiliary observations (training curves, diagnostics, scientist logs) that standard Bayesian optimization (BO) discards. It introduces two methods on top of a GP surrogate. AuxBO-Evolve has the LLM propose likely maximizer locations, fits a Gaussian mixture to them, and multiplies the acquisition function by this belief raised to β/t, following πBO but with the belief updated from the history. AuxBO-Vicinity asks the LLM to rank the L nearest unevaluated candidates after each evaluation and turns those rankings into persistent local corrections on the acquisition function through an RBF memory. The authors also make two claims: LLMs are better used to model maximizer distributions than pointwise objective values (Figure 2), and auxiliary feedback helps beyond the LLM’s prior knowledge (HPOBench and the DIII-D nuclear fusion task). Theorem 3.1 gives a zero-simple-regret result for AuxBO-Evolve under posterior-variance acquisition in a discrete independent-arms setting.

Critical strengths
  • The problem is well motivated. Discarding auxiliary signals such as learning curves and expert logs is a real gap, and the framing of LLMs as an interface to unstructured information is useful.

  • The maximizer-distribution idea is clean. Modulating the acquisition by a decaying LLM belief reuses the πBO machinery and inherits its robustness to misspecified beliefs.

  • The real-world test function is valuable. Using 281 historical DIII-D shots with unstructured operator logs is a realistic test of whether the method works without a curated feedback format.

Critical weaknesses
  1. Algorithmic novelty and the argmax prior (Contributions 1 and 2)

The argmax-prior construction and the exponential decay of the acquisition modulation are adapted from πBO, so the main algorithmic change for AUXBO-EVOLVE is the source of the prior: an LLM-elicited, history-conditioned belief in place of a user-specified one. This is a useful extension, but it would be stronger if the paper made that contribution clearly and stated where it departs from πBO. Figure 2 shows that the argmax prior captures the optimum better than a pointwise function-value prior on one SVM task. To support the claim that maximizer modeling is the better design, the comparison should be extended to the other problems in the paper, including the HPOBench tasks and the synthetic functions, and should report how consistently the advantage holds.

  1. Relationship between AUXBO-VICINITY and AUXBO-EVOLVE

AUXBO-VICINITY has a local-refinement structure that resembles TuRBO [1] in a high-level way. The paper does not yet explain why AUXBO-VICINITY trails AUXBO-EVOLVE on the presented problems. Appendix A.2 suggests that ranking quality is one factor, but the paper would benefit from a more direct analysis of when each variant should be preferred. For example, does AUXBO-VICINITY become competitive in higher-dimensional domains, where a global maximizer belief is harder to estimate and local corrections may be more informative, as TuRBO's motivation would suggest? Experiments that vary dimensionality (currently 2-4D for the synthetic functions), or that characterize the locality of the auxiliary feedback, would help readers understand how the two settings differ and which one fits a given problem.

  1. Scope of the real-world evaluation (Contribution 3)

The current experiments make it hard to assess how well AUXBO handles unstructured auxiliary information in practice. Most test functions provide exact gradients, which is a much more structured and informative signal than scientist-written notes. An LLM given gradients can reason in a fairly direct way, whereas the reasoning required for free-text logs may be quite different. On the real-world tasks, the improvements over baselines are modest in Figures 4 and 5, and Figure 4 does not significantly show the effect of unstructured information. Adding more scientific test problems would strengthen the empirical case. It would also help to examine how the LLM uses the auxiliary information, for example by analyzing the reasoning traces or the beliefs it forms. The paper already shows that stronger, more informative feedback (the gradient-based tasks) yields larger gains. Explaining why the gains are smaller in the scientific setting would make the results more informative for readers and would help identify where future work should focus.

[1] Eriksson, D., Pearce, M., Gardner, J. R., Turner, R., & Poloczek, M. (2019). Scalable Global Optimization via Local Bayesian Optimization. NeurIPS 2019.

Questions for authors

Please check the weakness section.

1 comment

We thank you for taking time to review our work. These are our answers to your concerns -

  1. Our main contribution is on establishing mechanisms to utilize auxiliary information. Our algorithm AuxBO-Evolve differs from π\piBO in updating the arg⁡max⁡\arg\max distribution and we provide a general mechanism for constructing this distribution from an LLM. These differences are mentioned in the paper in section 3.3 and we also have π\piBO as a baseline. We further request you to please refer to our general official comment on “Clarification of novelty and contributions”.

    Figure 2. Quantitative results over a broader set of HPOBench tasks are already reported as regret curves in Figure 4 of the experiments section. The upper panel of Figure 2 serves a different purpose: it lets the reader see the priors the LLM produces. We chose SVM because it has only two hyperparameters (C,γ)(C, \gamma), so the ground-truth landscape and both priors can be drawn as heatmaps on the same grid.

    The LLM priors are elicited from a description of the model and its search space alone, with no information about the dataset. To give a fuller picture, we now provide the ground-truth landscapes of all 29 HPOBench SVM datasets (check Appendix A8), which can be compared directly against the priors shown in Figure 2. Standard synthetic functions do not induce a useful prior from the LLM.

  2. TuRBO and AuxBO-Vicinity

    While TuRBO and AuxBO-Vicinity both focus on exploiting local structure, they differ in significant ways. TuRBO uses multiple boxes in search space and fits local GPs. Our vicinity method on the other hand, uses only a single GP model across the whole parameter space. The modulation term Ht(x)H_t(x) in the Vicinity algorithm is constructed locally, however it does not separate the search space into pieces.

    We thank reviewer oRgo for suggesting analysis to higher dimensions. Please refer to our general official comment regarding this point.

  3. Please refer to our general official comment on “Correction to Fig 4” (updated results on the HPO) and “Clarification of the experimental design and findings”, which provide discussions on the significance of the results, and explains gradient improvements are more pronounced because it is the only case where the auxiliary information can be isolated from the contribution from the task prior. Finally, the fusion task provides very unstructured information as feedback (examples are shown in Appendix A3), showcasing a realistic case.

Comments on the paper

2 comments

Correction to Figure 4.

We identified a minor inconsistency in the initialization protocol. To accommodate methods that require at least one observation, all methods were intended to receive one uniformly random warm-start evaluation. However, LLINBO was mistakenly run without this warm start, allowing its LLM component to select the first point and giving it an unfair early advantage. This explains the unusually low early regret noted by qbiD. After rerunning LLINBO with the same random warm start, all methods begin comparably, and AuxBO-Evolve achieves statistically significant gains over LLINBO and the other baselines. We will replace Figure 4 and correct the corresponding protocol description.

Clarification of novelty and contributions.

We thank the reviewers for the opportunity to clarify the paper’s central contributions.

(1) Our primary contribution is the systematic study of evaluation-side auxiliary information in BO.

Standard BO reduces each experiment to (x,y)(x,y), discarding rich, usually left-out information produced alongside the objective—such as training curves and diagnostic metrics in HPO, gradients, and unstructured scientist-written experimental logs. We instead study the setting (x,y,yaux)(x,y,y_{\mathrm{aux}}), using an LLM as a general interface for interpreting heterogeneous yauxy_{\mathrm{aux}} without requiring a custom probabilistic model for each feedback type. To our knowledge, this is the first general LLM-based framework specifically designed to exploit such heterogeneous information. Our primary novelty is therefore the problem setting and the demonstration that this routinely discarded information can substantially improve BO, not a new method to combine an LLM with a GP.

(2) Exploiting this information requires a mechanism designed for it.

Merely providing yauxy_{\mathrm{aux}} to an existing LLM-based optimizer is insufficient: in Figure 4, LLINBO and LLAMBO obtain little benefit from the additional information. This motivates our algorithms. AuxBO-Evolve is strongly inspired by π\piBO, and we intentionally retain its acquisition-modulation and annealing mechanism. However, it differs from π\piBO in two important ways. First, whereas π\piBO assumes that a maximizer distribution is supplied by the user, we provide a concrete mechanism for constructing this distribution from an LLM: we repeatedly sample likely maximizer locations from the LLM and place Gaussian components around the resulting proposals to form a mixture distribution over xx. Second, π\piBO’s distribution remains fixed throughout optimization, whereas we re-elicit this distribution after every iteration using the accumulating objective and auxiliary observations. Consequently, our πt(x⋆∣Dtaux)\pi_t(x^\star\mid D_t^{\mathrm{aux}}) is an evolving posterior belief conditioned on the observed optimization history, rather than a fixed prior. Our contribution is the LLM-based construction and continual updating of the maximizer distribution. The static π\piBO comparison demonstrates that this adaptation is effective at using auxiliary information.

Clarification of the experimental design and findings.

A fundamental challenge in evaluating auxiliary information on real-world tasks is that an LLM can benefit from two sources of improvement: (1) its pre-existing knowledge of the optimization domain and (2) the auxiliary observations received during optimization. For example, in HPO, the LLM already knows plausible learning rates and common relationships among hyperparameters before seeing any observations. Removing yauxy_{\mathrm{aux}} isolates source (1)—as in our π\piBO baseline—but isolating source (2) is generally impossible on a real-world task. Interpreting a learning curve or scientist-written note necessarily requires domain knowledge, which can simultaneously provide a useful zero-shot prior over promising configurations. We discuss this issue in Section 4.2.

The synthetic-function experiments were designed specifically to resolve this identifiability problem. We hide the function name and transform and anonymize its coordinates, so the LLM knows only that it is optimizing an unknown function and cannot rely on memorized knowledge of its landscape. The purpose of standard synthetic function experiments is to isolate source (2) in the absence of source (1). The large improvement in this setting demonstrates that auxiliary information alone can improve optimization significantly.

This distinction also offers a potential explanation for oRgo’s observation that the gains are larger on the gradient-based synthetic tasks. The difference may not arise only because gradients are a stronger and more informative feedback type. On real-world tasks, source (1) can already provide LLM-based methods with a substantial advantage, leaving less room for source (2) to yield an additional improvement. On the anonymized synthetic tasks, source (1) is unavailable by construction, so the effect of auxiliary feedback is more directly visible.

On HPOBench, the benefit of source (2) beyond source (1) is smaller but consistent. The reported curves aggregate several tasks, each method is evaluated over 50 independent runs, and the non-overlapping 95% bootstrap confidence bands show that the improvement is statistically significant.

We agree that unstructured auxiliary information that is not artificially created must also be evaluated directly, which is why we constructed the DIII-D fusion experiment. The raw operator logs shown in Appendix Table 1 are free-form scientist-written records with no common schema; they contain operational events, equipment issues, and scientist observations. They cannot be represented easily in any numerical search space. This is therefore a deliberately challenging test of unstructured auxiliary feedback, not an extension of the gradient experiment. AuxBO-Evolve achieves the lowest median cumulative regret and clearly improves over standard GP-UCB. Because the long log histories require a larger, higher-context, and hence costlier model, this experiment currently contains 10 episodes per method. We can expand the number of episodes if the reviewers recommend doing so. We would also be happy to evaluate a reviewer-suggested benchmark containing unstructured auxiliary feedback (not artificially created); such benchmarks are less common than standard (x,y)(x,y) BO benchmarks.

AuxBO-Evolve and AuxBO-Vicinity at higher dimensions

We note test performance on increasing dimensions on Styblinski Tang and Rosenrock (10D, 20D and 40D) synthetic function optimization tasks. We notice that the Vicinity method outperforms Evolve as we move to higher dimensions. This happens because in Evolve, the neighbourhood around what LLM thinks is the best point shrinks relative to the full parameter space. This makes the LLM influence term πt\pi_t sparser. On the other hand, the Vicinity method’s influence term is denser through the parameter space. Additionally, it is also able to utilize the local structure and can hence identify better local optima. Regardless, both methods outperform GP-UCB. Note that to have more precise values around the previous experiment’s neighbourhoods, we used the sample_around_best setting in BoTorch. Please refer to Appendix A9 for the plot.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.