acceptodds
Under review as a conference paper at ICLR 2027

Large Language Models as Prior Elicitors for Sample-Efficient Black-Box Optimization

Abstract

Many scientific and engineering applications require optimizing complex decisions whose outcomes can only be evaluated through costly simulations, experiments, or high-fidelity models, giving rise to black-box optimization (BBO) under tight evaluation budgets. Existing BBO methods typically learn from objective evaluations collected on the target task and therefore rely heavily on costly observation-driven exploration. Large language models (LLMs), with broad cross-domain knowledge, offer a potential opportunities to alleviate this dependence. This study introduces a different perspective on using LLMs for BBO: rather than using LLMs to assist sampling, we elicit embedded domain knowledge of LLMs to explicitly form a probabilistic distribution over objective functions. Specifically, we develop an LLM-enhanced Bayesian optimization framework that first elicits a probabilistic prior over the objective function from LLMs and then incorporates this prior into Bayesian optimization. The key idea is to develop an agentic preference elicitation method, where an LLM agent is established to analyze the problem context and to generate pairwise preferences between candidate solutions, thereby deriving a probabilistic distribution through Bayesian preference learning before expensive evaluations are collected. Theoretically, we show that the elicited distribution contracts predictive uncertainty and information gain and, under an explicit alignment condition, yields a no-larger finite-budget GP-LCB simple-regret bound than the reference prior. We further establish controlled sensitivity to corrupted preferences and show that an initially biased prior can be corrected as sufficiently informative objective observations accumulate. Experiments on six real-world benchmarks demonstrate improved sample efficiency, with the largest gains under limited evaluation budgets.

Primary area:
optimization
Keywords:
Bayesian Optimization, Black-box Optimization, Large Language Model, LLM Agent

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_CYXoOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper incorporates LLM-generated pairwise preferences into a GP prior to improve the sample efficiency of black-box optimization. The method is evaluated on six datasets ranging from 11D to 25D.

Critical strengths

The paper evaluates the method on benchmarks spanning multiple domains.

Critical weaknesses
  1. Limited novelty. LABO (ICML 2026) models the optimization problem’s input–output relationship using the Kennedy–O’Hagan (KOH) formulation and dual-fidelity modeling. The proposed approach also shares similarities with Direct Preference Optimization (DPO). LGBO (ICLR 2026) has explored incorporating LLM preferences into GP priors. More detailed methodological and empirical comparisons are needed.
  2. Insufficient justification for the “agentic” claim. The approach appears to rely primarily on prompt engineering, without clearly demonstrating autonomous planning, task execution, tool use, or feedback-driven iterative decision-making. A relevant comparison is Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch .
Questions for authors
  1. Can the method be evaluated on other BBO applications, such as machine learning hyperparameter optimization?
  2. Can the authors evaluate performance in higher-dimensional settings, which are common in scientific optimization?
  3. Can the prior itself evolve throughout optimization?
  4. The ablation studies are incomplete. Can the authors directly assess the necessity of the agentic preference-generation mechanism, compare different LLMs, and evaluate prompt sensitivity?
  5. Compare against more LLM-assisted BO baselines, such as BOPRO (ICLR 2025).
8 comments
AuthorsOpenReview ↗

Reviewer CYXo-Part 1 (Weakness 1)

Weakness 1

Limited novelty. LABO (ICML 2026) models the optimization problem’s input–output relationship using the Kennedy–O’Hagan (KOH) formulation and dual-fidelity modeling. The proposed approach also shares similarities with Direct Preference Optimization (DPO). LGBO (ICLR 2026) has explored incorporating LLM preferences into GP priors. More detailed methodological and empirical comparisons are needed.

Response

We respectfully disagree with the reviewer's assessment of limited novelty. The cited studies address substantially different methodological questions and do not establish that our contribution lacks originality. Our central contribution is an agentic framework for eliciting domain knowledge from pretrained LLMs and transforming it into an informative function-space prior before any target-objective evaluations. Specifically, a coding agent autonomously constructs a reusable, task-specific preference generator, whose pairwise judgments are converted into a GP prior with learned mean and covariance through Bayesian preference learning. This provides a systematic mechanism for incorporating LLM domain knowledge into BBO, rather than merely using LLMs to guide individual search decisions.

First, LGBO (Yuan et al., 2026) is the most closely related work, but its methodology differs fundamentally from ours. LGBO repeatedly queries an LLM using evolving optimization histories to identify promising points or regions, which are incorporated through region-lifted updates that shift the GP mean while preserving its covariance. In contrast, our agent constructs an executable preference-generation mechanism before any objective evaluations, and the resulting pairwise comparisons are aggregated probabilistically to learn both the mean and covariance of a GP prior. The distinction therefore lies in how domain knowledge is elicited, how it is represented probabilistically, and how it is incorporated into optimization.

Second, LABO (Chen et al., 2026) addresses a different problem through multi-fidelity modeling. It treats LLM-generated numerical predictions as low-fidelity observations and employs the Kennedy–O'Hagan (KOH) formulation to combine them with real evaluations through a cross-fidelity relationship and discrepancy model. Our approach instead constructs a GP prior from ordinal comparisons without requiring calibrated numerical predictions or estimating a low-to-high-fidelity relationship. The shared objective of modeling an unknown input–output relationship does not imply methodological equivalence; this is a general principle underlying surrogate-based optimization.

Third, the comparison with DPO (Rafailov et al., 2023) is not methodologically justified. DPO uses preference data to optimize the policy parameters of a language model. Our framework does not optimize or fine-tune the LLM; it uses agent-generated comparisons to infer a probabilistic prior over an external black-box objective. The mere use of pairwise comparisons does not make these approaches methodologically equivalent.

Importantly, our manuscript already provides direct empirical comparisons with LGBO across all six benchmarks, using matched initialization and evaluation budgets over 20 seeds. Our method achieves substantially better average ranks at 10, 20, and 200 post-initialization evaluations (1.17 vs. 4.00, 1.00 vs. 4.33, and 2.08 vs. 4.08, respectively), while using considerably fewer LLM tokens (0.52M vs. 16.92M). Furthermore, our theoretical analysis characterizes finite-budget regret improvement under an explicit alignment condition, sensitivity to preference errors, and asymptotic convergence.

In summary, the novelty of our work lies not in the isolated use of LLMs, preferences, or GPs, but in a new mechanism for autonomously eliciting domain knowledge and converting it into a reusable probabilistic prior for BBO. None of the cited studies implements this complete mechanism. We therefore maintain that the methodological contribution is distinct and supported by both theoretical and empirical evidence. We have further clarified these distinctions in the revised Related Work section.

AuthorsOpenReview ↗

Reviewer CYXo-Part 2 (Weakness 2)

W2

Insufficient justification for the “agentic” claim. The approach appears to rely primarily on prompt engineering, without clearly demonstrating autonomous planning, task execution, tool use, or feedback-driven iterative decision-making. A relevant comparison is Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch.

Response

We respectfully disagree with the reviewer's characterization of our approach as primarily relying on prompt engineering. Our framework employs a coding agent that autonomously constructs and validates task-specific computational procedures, rather than simply prompting an LLM to generate preferences.

As detailed in Section 3.1 and Appendix B.1, the agent is implemented using the Codex CLI coding-agent environment. Given the optimization problem and static scenario information, it autonomously analyzes the underlying system, identifies relevant domain mechanisms, selects approximation methods, and implements executable analysis code. It further performs computational checks, such as boundary tests and sensitivity analyses, to assess the reliability of its estimates and revise the procedure or abstain from uncertain comparisons when necessary. Importantly, our prompt specifies neither task-specific models nor predefined scoring rules; these are developed by the agent through its own analysis and tool execution. The resulting artifact is a reusable preference-generation program, rather than a collection of direct LLM judgments. These operations explicitly involve autonomous method selection, task execution, tool use, and feedback-based validation.

The suggested comparison with Agentic Bayesian Optimization (Brunzema et al., 2026) also concerns a different research objective. That work investigates agentic optimization, where an LLM agent controls the optimization process, analyzes accumulated observations, and adaptively modifies search strategies. In contrast, our work investigates agentic domain-knowledge elicitation: the agent autonomously transforms problem descriptions and domain knowledge into an executable approximation mechanism, which is subsequently used to construct a GP prior before any expensive objective evaluations. Accordingly, the feedback used by our agent comes from its own computational analyses and validation procedures, rather than observations collected during optimization. These are distinct applications of agentic capabilities, and autonomous modification of the optimizer is not a prerequisite for agentic knowledge elicitation.

We therefore maintain that the agentic nature of our framework should be assessed by its autonomous modeling decisions, executable actions, and use of computational feedback, rather than whether it operates within the BO loop. To make these mechanisms more transparent, we will further clarify the agent's execution and validation workflow in the revised manuscript.

AuthorsOpenReview ↗

Reviewer CYXo-Part 3 (Q1, Q2)

Q1

Can the method be evaluated on other BBO applications, such as machine learning hyperparameter optimization?

Response

We thank the reviewer for this suggestion. Our framework is not restricted to any particular application domain. Its applicability primarily depends on whether the problem context provides domain knowledge that can be translated into informative preferences. In principle, the framework can also be applied to hyperparameter optimization (HPO), where knowledge about model architectures, training procedures, and hyperparameter interactions may inform prior construction.

However, our primary focus is on expensive scientific and engineering BBO problems, where rich domain knowledge is available but objective evaluations are costly. We have already evaluated the framework on six benchmarks spanning four distinct application domains: traffic control, EV charging, water-network operation, and materials screening. These experiments demonstrate its effectiveness across heterogeneous optimization problems, rather than within a single application.

We agree that HPO represents an interesting additional application. We therefore consider it a valuable direction for future investigation.

Q2

Can the authors evaluate performance in higher-dimensional settings, which are common in scientific optimization?

Response

We thank the reviewer for raising this important question about high-dimensional optimization. While our original six benchmarks span 11–25 dimensions, we agree that further evaluation is needed to assess performance in substantially higher-dimensional settings.

We therefore conducted an additional experiment on HPA101-2, a 108-dimensional human-powered aircraft design problem (Namura, 2025). We construct the preference-informed GP prior using 1,000 agent-generated hard preferences from 2,048 support candidates, with 512 inducing points. No target-task objective evaluations are used during prior construction. We assess preference quality, the elicited distribution, and optimization performance.

(1) Quality of agent-generated preferences. We evaluate the preference generator on a test set containing 10,000 candidate pairs.

Table 1.

Test pairsCoveragePreference accuracy
10,00081.02%87.45%

The agent-generated preferences achieve 87.45% accuracy with 81.02% coverage, indicating that informative pairwise relationships can be elicited in this high-dimensional problem.

(2) Quality of the elicited distribution. We compare the ranking accuracy of the preference-informed GP prior with conventional GP posteriors trained on direct objective observations, using 5,000 held-out candidate pairs.

Table 2.

Predictive distributionObjective evaluationsRanking accuracy
Preference posterior072.62%
GP posterior (20)2055.67%
GP posterior (50)5057.46%

GP results use observations from GP-LCB trajectories and data-estimated RBF lengthscales, averaged over 20 seeds.

Without using target-task evaluations, the elicited distribution achieves 72.62% ranking accuracy, substantially outperforming conventional GP posteriors trained with 20 or 50 objective observations.

(3) Optimization performance. We further incorporate the elicited distribution into GP-LCB and compare it with Vanilla BO (GP-LCB) over 40 paired seeds. Both methods share 10 initial objective evaluations and receive 200 additional evaluations. We report mean best-so-far objective values (lower is better).

Table 3.

Additional evaluationsOurs (GP-LCB)Vanilla BO (GP-LCB)Relative reduction
1042.01 ± 6.6760.88 ± 26.92+30.99%
2038.60 ± 4.7244.39 ± 8.42+13.04%
5034.66 ± 3.5935.35 ± 3.51+1.95%
10033.04 ± 2.5832.77 ± 2.71−0.82%
20031.41 ± 1.5631.32 ± 1.66−0.29%

Relative reduction = (Vanilla BO − Ours) / Vanilla BO × 100%; positive values favor our method. The two methods use their respective tuned GP-LCB configurations.

Our method reduces the mean objective value by 30.99% after 10 additional evaluations and 13.04% after 20 evaluations. The advantage diminishes as more observations become available, with both methods achieving comparable final performance.

Overall, these results demonstrate that our framework can elicit informative preferences, construct a useful probabilistic prior, and improve early-stage optimization in a 108-dimensional search space using only 1,000 hard preferences. This provides further empirical evidence of its applicability to high-dimensional problems, although broader scalability across dimensions and problem classes remains to be investigated. This experiment has been supplemented in the revised manuscript in Appendix E.5

Reviewer_Ek35OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes using LLMs not to directly suggest optimization points, but to elicit prior knowledge about the unknown objective function. An LLM-based agent generates pairwise preferences between candidate solutions, which are converted through Bayesian preference learning into a Gaussian-process prior. This informative prior is then used in Bayesian optimization to improve sample efficiency, especially under tight evaluation budgets. The method is evaluated on six real-world black-box optimization benchmarks and includes theoretical analysis of when the elicited prior can improve finite-budget regret.

Critical strengths

The idea of using an LLM to construct the prior for Bayesian optimization is quite interesting. Here, the author uses an LLM as a noisy labeler to build a statistical GP prior using preference feedback. The work is supported by some theoretical analysis and several real-world benchmarks.

Critical weaknesses

I am not fully convinced that preference feedback is the right type of feedback to expect from LLMs. One of the main motivations for using preference feedback is that humans are sometimes better at making pairwise comparisons than at reporting accurate numerical values. However, whether this also holds for LLMs is not clearly motivated or discussed in the paper. Hence, the choice of preference feedback for constructing the prior model seems somewhat under-justified.

Questions for authors

1, There are some typos, Because Eq. equation 6, existence of both "zero GP prior" and "zero-GP prior". Please check the draft more carefully.

2, Could the authors further motivate why preference feedback is the better choice for collecting feedback from LLMs? Appendix E.4 already shows that the Agent-score baseline is competitive and even performs better in some settings, so it is not very clear why preference feedback should be preferred. Although the authors argue that preference feedback avoids the need for score-to-objective calibration, it is still unclear whether LLMs are actually better at making pairwise comparisons than directly reporting scalar scores. A more direct comparison of the accuracy and quality of the prior models built from preference feedback and direct scores would help support this choice.

3, It would also be useful to understand how LLM-generated preference feedback compares with feedback from human experts. Even if this comparison is only conducted on a subset of the benchmarks, it could help show how much difference it makes to use LLMs instead of human experts for building the prior. This would make the practical motivation for using LLM preference feedback more convincing.

6 comments
AuthorsOpenReview ↗

Response to Reviewer Ek35-Part1 (Weakness)

W

I am not fully convinced that preference feedback is the right type of feedback to expect from LLMs. One of the main motivations for using preference feedback is that humans are sometimes better at making pairwise comparisons than at reporting accurate numerical values. However, whether this also holds for LLMs is not clearly motivated or discussed in the paper. Hence, the choice of preference feedback for constructing the prior model seems somewhat under-justified.

Response

We thank the reviewer for this comment. We clarify our motivation for preference elicitation and provide additional experimental evidence.

First, the role and acquisition of preferences in our setting differ from those in conventional preference-based optimization. Classical preferential BO typically considers latent utility functions for which direct numerical evaluations are unavailable, relying on pairwise comparisons as the primary feedback (González et al., 2017). Preferences have also been used to assist BO when the objective is directly evaluable (Huang et al., 2023). However, obtaining such feedback from human experts requires repeated human involvement, which can limit scalability when many comparisons are needed (Kayal et al., 2025). In contrast, we consider objectively evaluable but computationally expensive optimization problems. Our framework uses a coding agent to construct a reusable, task-specific preference generator before objective evaluations. This procedure automatically produces pairwise comparisons informed by domain knowledge, without requiring human annotation of individual pairs or access to true objective values. These comparisons are then used to construct an informative prior for subsequent BO with numerical objective feedback.

Second, our motivation for preference elicitation is that relative comparisons can provide useful information even when numerical objective estimates are unreliable. Previous studies have demonstrated advantages of LLM-based comparative judgments over direct numerical scoring in certain evaluation tasks (Liusie et al., 2024; Licht et al., 2025). More specifically, upon examining the agent-generated procedures in our experiments, we found that the agent constructs multiple approximate scoring mechanisms under plausible assumptions about unspecified problem conditions, which is not specified in our prompt. Although these approximations may not accurately predict the true objective values, they can reveal consistent relative rankings between candidate solutions across different scenarios. The agent thus derives preference from these rankings across scenarios. We therefore use preferences to capture such ordinal information without requiring the approximate scores to be calibrated to the true objective values.

Third, we conducted additional experiments comparing preference-based and score-based elicitation across six benchmarks. We first evaluate agent-generated preferences and score-induced rankings against true objective orderings on 5,000 sampled pairs per benchmark. We then construct preference-derived and score-derived GP priors from the retained comparisons and candidate scores, respectively. Both priors are conditioned on the same 20 objective observations and evaluated on a separate set of 5,000 held-out pairs.

Table 1. Comparison of agent-generated preferences and scalar scores. Raw accuracies are computed over definite preferences and non-tied scores, respectively. GP accuracies use identical held-out pairs after conditioning on the same 20 observations. Results are based on one fixed random design.

BenchmarkPref. Acc. (%)Score Acc. (%)Pref.-GP Acc. (%)Score-GP Acc. (%)
TSC-C360.3658.5458.2857.26
TSC-C866.2962.3267.5373.73
EVC-Caltech99.8395.7486.7878.80
EVC-JPL99.7695.4081.1870.74
COF80.6277.1469.7865.58
Net397.8694.2292.3885.38

The elicited preferences achieve conditional accuracies of 60.36%–99.83%, exceeding score-based rankings through selective retention of informative comparisons. More importantly, preference-derived GPs achieve higher held-out ranking accuracy on five of six benchmarks, with gains of 1.02–10.44 percentage points. These results support preference elicitation as an effective interface for constructing informative priors, although its advantages are task-dependent and scalar models remain competitive. We have included these experiments in revised Appendix E.4 and clarified the motivation in the main text.

AuthorsOpenReview ↗

Response to Reviewer Ek35-Part2 (Q1, Q2)

Q1

There are some typos, Because Eq. equation 6, existence of both "zero GP prior" and "zero-GP prior". Please check the draft more carefully.

Response

We thank the reviewer for pointing out these typos and inconsistencies. We have carefully proofread the manuscript and corrected them throughout.

Q2

Could the authors further motivate why preference feedback is the better choice for collecting feedback from LLMs? Appendix E.4 already shows that the Agent-score baseline is competitive and even performs better in some settings, so it is not very clear why preference feedback should be preferred. Although the authors argue that preference feedback avoids the need for score-to-objective calibration, it is still unclear whether LLMs are actually better at making pairwise comparisons than directly reporting scalar scores. A more direct comparison of the accuracy and quality of the prior models built from preference feedback and direct scores would help support this choice.

Response

We thank the reviewer for this constructive suggestion. The competitive performance of Agent-score in revised Appendix E.4 further justified our choice of preference feedback.

Upon examining the agent-generated procedures, we found that, without specified requirements, the agent constructs multiple approximate scoring mechanisms under plausible assumptions about unspecified problem conditions and synthesizes their assessments into pairwise preferences. This allows us to extract relative rankings without relying on potentially inaccurate numerical estimates of the true objective.

To further validate this choice, we conducted two additional experiments across six benchmarks. First, we compared the accuracy of agent-generated preferences and score-induced rankings on 5,000 candidate pairs per benchmark. Preferences achieved accuracies of 60.36%–99.83%. Their higher accuracy reflects selective retention of reliable comparisons. Second, we compared preference-derived and score-derived GP models on 5,000 held-out pairs after conditioning both on the same 20 objective observations. The preference-derived GP achieved higher ranking accuracy on five of the six benchmarks, with improvements of 1.02–10.44 percentage points.

More broadly, our central goal is to develop a mechanism for eliciting domain knowledge from LLMs and constructing a prior for the objective. While both preference- and score-based elicitation yield competitive optimization performance, our results suggest that preferences provide a more reliable basis for prior construction, as they selectively retain informative comparisons and achieve higher held-out ranking accuracy on five of the six benchmarks. These findings support preference elicitation as an effective approach to constructing informative GP priors, although its advantages are task-dependent. We have added the detailed experimental results to Appendix E.4 and clarified this motivation in the main text.

AuthorsOpenReview ↗

Response to Reviewer Ek35-Part3 (Q3 and References)

Q3

It would also be useful to understand how LLM-generated preference feedback compares with feedback from human experts. Even if this comparison is only conducted on a subset of the benchmarks, it could help show how much difference it makes to use LLMs instead of human experts for building the prior. This would make the practical motivation for using LLM preference feedback more convincing.

Response

We thank the reviewer for this valuable suggestion. We agree that comparing LLM-generated preferences with human expert judgments would provide useful insights into their relative quality and their effectiveness in constructing BO priors.

Our motivation for using an LLM agent, however, is scalability rather than superior judgment accuracy. In our setting, the optimization objectives are explicitly defined but expensive to evaluate through simulation. Without access to these evaluations, reliably comparing candidate solutions may require substantial domain-specific reasoning and analysis, making large-scale manual preference elicitation burdensome. Our coding agent instead translates domain knowledge into executable, task-specific scoring procedures, which can be reused to automatically generate large numbers of pairwise preferences without repeated human annotation or expensive objective evaluations. This substantially reduces the marginal cost of preference generation.

We acknowledge that a controlled comparison with human experts would be valuable. We do not claim that LLM-generated preferences are inherently more accurate or lead to better BO priors than expert judgments. We will clarify this motivation and explicitly acknowledge the absence of such a comparison as a limitation in the revised manuscript.

References

  • González et al. (2017). Preferential Bayesian Optimization. ICML, PMLR 70:1282–1291.
  • Huang et al. (2023). Augmenting Bayesian Optimization with Preference-based Expert Feedback. ICML 2023 Workshop on The Many Facets of Preference-Based Learning.
  • Kayal et al. (2025). Bayesian Optimization from Human Feedback: Near-Optimal Regret Bounds. ICML, PMLR 267:29449–29468.
  • Licht et al. (2025). Measuring Scalar Constructs in Social Science with LLMs. EMNLP, pp. 32144–32171. https://doi.org/10.18653/v1/2025.emnlp-main.1635
  • Liusie et al. (2024). LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models. EACL, pp. 139–151. https://doi.org/10.18653/v1/2024.eacl-long.8
Reviewer_4oUTOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The papers aims to use LLM for more efficient BO that leverages problem context. The idea is to use the LLM to build a prior on the unknown backbox objective, where the Gaussian process learned from data is then conditioned on this prior to incorporate the LLM knowledge within BO.

Critical strengths

The paper is well written and easy to read. The approach of using paired comparisons to build a prior is novel to the best of my knowledge.

Critical weaknesses
  1. The paper's motivation is not well justified. The paper starts with this motivation: "Despite their effectiveness in enhancing observations, a fundamental challenge of BBO is largely unexplored: how can we effectively uncover the implicit input–output relationship of the expensive objective function?" However, BBO aims at optimization and not understanding and this is why we do not do pure exploration in BBO. Indeed, this statement directly misaligns (and in most BBO cases conflicts) with the underlying goal of BBO.

  2. I suggest making Fig 1 simpler. The idea is rather simple but adding all the math early on can make that figure hard to understand

  3. A central question is how will this method perform in high-dimensions. If d is slightly large then the number of needed comparison exponentially grows. This is also leads to an exponential growth is cost induced. Indeed, if the LLMs goal is to build a good prior for the entire function, this will become practically infeasible even for a small d. Do you consider asking the LLM to focus on promising areas (where f is believed to be small) ?

  4. Given that Sec 3.2 lists existing approaches including the inducing variable approximation, I am not sure if it helps the main story or can be better positioned in the appendix.

  5. How and in which step are the kernel hyperparameters estimated ?

  6. The theory is a strength of the paper. That said, the criteria for tighter bound is very hard to check in practice.

  7. The area of LLM assisted BO has become very active in the past years with many papers in this area. It is not clear why the authors avoided comparison with some of the common benchmarks like LLAMBO (Large language models to enhance Bayesian optimization) or LLMP (LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language) or even papers they cite themselves like LLINBO.

  8. Further a simple benchmark is using a strong LLM as the main optimizer where all context and historical data are provided. This can highlight the strength of taking a hybrid approach. The newer LLMs have already shown strength compared to BO methods in higher dimensions as standalone optimizers.

  9. The literature on Human-AI Collaborative BO is highly relevant. It is suggested to highlight the distinction with this set of literature and novelty. For instance a well known approach is PiBO (Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization) and it similarly uses a prior belief to improve BO. Now, there it is assumed that an expert can provide this prior but similarly the LLM can act as the expert. It is worth comparing with piBO's approach of defining the acquisition function.

Questions for authors

Please see the section above for questions.

8 comments
AuthorsOpenReview ↗

Response to Reviewer 4oUT-Part1 (W1, W2)

W1

The paper's motivation is not well justified. The paper starts with this motivation: "Despite their effectiveness in enhancing observations, a fundamental challenge of BBO is largely unexplored: how can we effectively uncover the implicit input–output relationship of the expensive objective function?" However, BBO aims at optimization and not understanding and this is why we do not do pure exploration in BBO. Indeed, this statement directly misaligns (and in most BBO cases conflicts) with the underlying goal of BBO.

Response

We thank the reviewer for this important comment. The goal of BBO is to identify high-quality solutions under limited evaluation budgets. Our intended point is that a fundamental difficulty in BBO arises from the black-box nature of the objective function: the relationship between decision variables and objective values is not explicitly available. Solving BBO, including learning the black box and identifying promising solutions, relies mainly on observations. Our motivation is to elicit informative domain knowledge from LLMs to reduce the required expensive observations.

In BO, a probabilistic model of the objective supports decisions about where to evaluate next. Representing beliefs over functions does not require accurately reconstructing the entire objective landscape: partial information about relative solution quality can already help guide search. This motivates our central question: how can domain knowledge embedded in pretrained LLMs be converted into an informative prior that improves the sample efficiency of BO?

Our contribution is an automated mechanism for constructing such a prior. A coding agent translates problem context and domain knowledge into an executable preference generator, and Bayesian preference learning converts its pairwise comparisons into an explicit GP prior. This prior-construction stage uses no target-objective evaluations. Subsequent evaluations are selected through the usual optimization acquisition function, and their outcomes update the model. Thus, the framework supplies additional information for allocating the limited evaluation budget, without introducing a separate stage of expensive evaluations for global function reconstruction.

We assess this contribution directly through optimization outcomes. Our theoretical analysis concerns finite-budget simple-regret bounds under an explicit alignment condition, while our experiments compare simple regret or best-so-far objective values under matched initialization and evaluation budgets. Across six benchmarks, following the same 10-point initialization, our method achieves average ranks of 1.17 and 1.00 after 10 and 20 additional evaluations, respectively, compared with 2.67 and 3.17 for vanilla GP-LCB (Table 1 in the manuscript). These results support the value of the elicited knowledge for early-stage optimization.

We have revised the Introduction to make this optimization-centered motivation explicit.

W2

I suggest making Fig 1 simpler. The idea is rather simple but adding all the math early on can make that figure hard to understand

Response

Thanks for pointing this out. We have simplified the figure in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer 4oUT-Part2 (W3)

W3

A central question is how will this method perform in high-dimensions. If d is slightly large then the number of needed comparison exponentially grows. This is also leads to an exponential growth is cost induced. Indeed, if the LLMs goal is to build a good prior for the entire function, this will become practically infeasible even for a small d. Do you consider asking the LLM to focus on promising areas (where f is believed to be small) ?

Response

We thank the reviewer for raising this question.

First, our existing optimization experiments cover dimensions beyond several representative settings in prior LLM-assisted BO studies. The following Table 1 summarizes the dimensions of selected benchmark problems used in these studies. Our six benchmarks span 11–25 dimensions. These experiments therefore provide evidence in a nontrivial dimensional regime for GP-based BO.

Table 1. Dimensions of representative LLM-assisted BO benchmarks.

StudyProblem dimension
Liu et al. (2024)2–8
Suwandi et al. (2025)2–14
Chang et al. (2025)2–6
Yuan et al. (2026)3–7; 14
Ranković and Schwaller (2025)6–10
Kobalczyk et al. (2025)5–8
Ours11–25

Second, we conducted an additional experiment on a 108-dimensional aircraft-design problem (HPA101-2; Namura, 2025). We construct a GP prior using only 1,000 agent-generated hard preferences from 2,048 support candidates, without target-objective evaluations. We examine three aspects:

(1) Preference quality. On 10,000 test pairs, the agent-generated preferences achieve 87.45% accuracy with 81.02% coverage (Table 2), demonstrating that informative comparisons can be generated in 108 dimensions.

Table 2. Preference quality.

Test pairsCoverageAccuracy
10,00081.02%87.45%

(2) Prior quality. On 5,000 held-out pairs, the elicited prior achieves 72.62% ranking accuracy without any objective evaluations, outperforming conventional GP posteriors trained with 20 or 50 observations (Table 3).

Table 3. Predictive ranking accuracy.

Predictive distributionEvaluationsAccuracy
Preference posterior072.62%
GP posterior (20)2055.67%
GP posterior (50)5057.46%

(3) Optimization performance. We compare our prior-informed GP-LCB with Vanilla GP-LCB over 40 paired seeds, sharing 10 initial evaluations and 200 additional evaluations (Table 4).

Table 4. Mean best-so-far objective (lower is better).

Additional evaluationsOurs (GP-LCB)Vanilla GP-LCBRelative reduction
1042.01 ± 6.6760.88 ± 26.92+30.99%
2038.60 ± 4.7244.39 ± 8.42+13.04%
5034.66 ± 3.5935.35 ± 3.51+1.95%
10033.04 ± 2.5832.77 ± 2.71−0.82%
20031.41 ± 1.5631.32 ± 1.66−0.29%

Our method reduces the mean objective by 30.99% after 10 additional evaluations, with both methods achieving comparable final performance. These results demonstrate that informative prior knowledge can improve early-stage optimization even in a 108-dimensional space. The experiment has been added to Appendix E.5.

Third, the number of comparisons and LLM computational cost need not grow exponentially with dimension. Our coding agent constructs a reusable executable preference generator, avoiding repeated LLM queries for individual pairs. Sparse GP preference learning further avoids enumerating the search space through inducing-variable approximations. Although approximating arbitrary high-dimensional functions remains challenging, our goal is to extract useful structure rather than reconstruct the entire landscape. Effectiveness therefore depends on task structure and domain knowledge.

We also appreciate the suggestion to focus comparisons on promising regions, which represents a valuable direction for future work.

AuthorsOpenReview ↗

Response to Reviewer 4oUT-Part3 (W4, W5, W6)

W4

Given that Sec 3.2 lists existing approaches including the inducing variable approximation, I am not sure if it helps the main story or can be better positioned in the appendix.

Response

We thank the reviewer for this helpful suggestion. We have therefore streamlined Section 3.2 to focus on the preference-learning formulation and the resulting GP prior, while providing the detailed inducing-variable construction and Laplace approximation in Appendix B.2.

W5

How and in which step are the kernel hyperparameters estimated ?

Response

The reference-kernel hyperparameters are determined during prior construction, before preference learning and any objective evaluations. Specifically, we use a unit-amplitude isotropic RBF kernel. Each input dimension is first standardized using the preference support candidates, and the lengthscale is set to the median nonzero pairwise Euclidean distance among a reproducibly sampled subset of up to 2,048 standardized candidates. Thus, the kernel hyperparameters are determined from input geometry, without using preference labels or objective observations. With the reference kernel fixed, we perform sparse Bayesian preference learning using a diagonal Laplace approximation to construct the preference-informed GP prior. The reference-kernel hyperparameters remain fixed during subsequent BO iterations, while the GP posterior is updated using objective observations. We have clarified these steps in Appendix B.2 of the revised manuscript.

W6

The theory is a strength of the paper. That said, the criteria for tighter bound is very hard to check in practice.

Response

Thanks for pointing this out. The alignment condition in Theorem 1 is difficult to verify in practice. However, we emphasize that it is a sufficient theoretical condition for obtaining a tighter finite-budget regret bound, rather than a requirement for applying our method. To provide empirical insight, Appendix E.2 presents post-hoc alignment diagnostics, showing improved posterior ranking accuracy on five of the six benchmarks.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Responses to Reviewers

We thank the AC and reviewers for their constructive feedback. Below, we summarize our responses to the four major concerns.

  1. For concerns about motivation and novelty (Reviewers 4oUT and CYXo):

    Our central contribution is a new mechanism for extracting domain knowledge from pretrained LLMs and converting it into informative priors for sample-efficient BBO. The fundamental challenge motivating our work is that conventional BBO relies heavily on expensive objective evaluations to learn the unknown input–output relationship. Rather than reconstructing the entire objective landscape, we seek to extract useful structural knowledge before any target-objective evaluations to improve optimization efficiency. Specifically, a coding agent autonomously constructs reusable, task-specific comparison procedures, and Bayesian preference learning converts their outputs into a GP prior with learned mean and covariance. This differs fundamentally from existing approaches such as LGBO's online preference guidance and LABO's numerical multi-fidelity modeling. Our theoretical analysis further characterizes finite-budget regret improvement under an explicit alignment condition, sensitivity to preference errors, and asymptotic convergence. We therefore emphasize that the contribution is a distinct knowledge-to-prior framework, rather than a simple combination of existing LLM and BO components.

  2. For concerns about the motivation and necessity of preference-based knowledge elicitation (Reviewers 4oUT and Ek35):

    Our framework is not simply a replacement of human experts with LLMs in existing preferential or human–AI collaborative BO. Conventional preferential BO primarily uses comparisons as optimization feedback, while expert-informed BO often assumes externally provided expert knowledge. In contrast, our setting has explicitly defined but expensive objectives, and the central challenge is how to autonomously obtain reliable prior knowledge without objective evaluations. Our coding agent addresses this by constructing executable comparison mechanisms rather than directly requesting LLM judgments. Additional experiments support this design: preference-derived GPs achieve higher held-out ranking accuracy than score-derived GPs on five of six benchmarks, with improvements of 1.02–10.44 percentage points. Moreover, direct LLM prompting achieves only 55.67% accuracy among definite judgments, consuming 6.39 million tokens for 600 comparisons. These results highlight both the difficulty and computational cost of direct preference elicitation, motivating our reusable agentic mechanism.

  3. For concerns about insufficient comparisons with existing LLM-assisted BO methods (Reviewers 4oUT and CYXo):

    Our original experiments already include CAKE and LGBO, two representative approaches to LLM-guided kernel design and preference-guided BO, respectively, alongside established BBO baselines. Across six benchmarks and 20 seeds, our method achieves average ranks of 1.17 and 1.00 after 10 and 20 additional evaluations, compared with 4.00 and 4.33 for LGBO. To broaden the empirical evaluation, we further conducted comparisons with LLAMBO (four benchmarks), LLMP (six benchmarks), and a standalone LLM optimizer (six benchmarks). These additions cover complementary paradigms, including LLM-assisted surrogate modeling, numerical prediction, and direct LLM optimization, providing broader evidence for the effectiveness of our prior-elicitation approach.

  4. For concerns about high-dimensional scalability (Reviewers 4oUT and CYXo):

    Our original six benchmarks already span 11–25 dimensions, exceeding the dimensionality of several representative LLM-assisted BO studies. To further address scalability, we added a 108-dimensional aircraft-design benchmark. Using only 1,000 agent-generated hard preferences, our framework achieves 87.45% preference accuracy (81.02% coverage) and constructs a GP prior with 72.62% held-out ranking accuracy without any objective evaluations. More importantly, across 40 paired optimization seeds, our method reduces the mean best-so-far objective by 30.99% after 10 additional evaluations compared with vanilla GP-LCB. These results directly demonstrate that informative priors can be constructed and effectively used for early-stage optimization in substantially higher-dimensional settings.

Overall, we believe these revisions directly address the principal concerns regarding originality, methodological justification, empirical validation, and scalability. The combination of a distinct knowledge-elicitation mechanism, theoretical characterization, and substantially expanded experimental evidence provides strong support for the novelty and practical value of our framework for sample-efficient BBO. We hope the Area Chair and reviewers will take these substantive additions into account in their final assessment.

AuthorsOpenReview ↗

Follow-up on Rebuttal and Invitation for Further Discussion

We have carefully addressed the major concerns through detailed responses, additional experiments, and substantial revisions. However, we have not yet received follow-up feedback from the reviewers. We would greatly appreciate further assessment of our responses and welcome further discussion.

AuthorsOpenReview ↗

Post-Rebuttal Summary and Request for AC Consideration

Dear Area Chair,

Thank you for overseeing the discussion. We would like to summarize the post-rebuttal assessments and request your independent evaluation of the remaining novelty disagreement.

Reviewers 4oUT and Ek35 have both raised their ratings to Weak Acceptance after considering our responses and additional experiments. These positive reassessments support the originality and value of our broader contribution: an agentic framework that autonomously extracts domain knowledge from LLMs through executable, reusable comparison procedures and converts the elicited knowledge into an explicit probabilistic prior distribution over the unknown objective function before expensive evaluations, thereby reducing BO's reliance on observation-driven learning. Reviewer 4oUT explicitly acknowledges novelty, while Reviewer Ek35 confirms that the additional evidence has largely addressed the main concern and strengthened the motivation for our approach.

For Reviewer CYXo, we provided detailed methodological comparisons and added experiments on agentic preference elicitation, high-dimensional optimization, and additional baselines. The reviewer’s follow-up nevertheless maintains that our approach amounts to “merely swapping in a different mathematical model” relative to LGBO, without explaining why the identified methodological differences are insufficient to constitute a contribution.

Our subsequent comment, Clarification of Methodological Novelty Relative to LGBO, directly addresses this characterization through a comparison table and supporting evidence. Specifically, our coding agent constructs a reusable, executable comparison procedure without accessing target-objective evaluations, and its outputs are converted into a GP prior. We explain why this knowledge-elicitation mechanism cannot be obtained simply by replacing LGBO’s mathematical model. We also requested clarification of the specific technical basis for the reviewer’s assessment; this subsequent explanation has not yet received a further response.

We respectfully ask the AC to assess whether the stated novelty objection, considered alongside our detailed response and additional evidence, provides sufficient grounds for a clear-rejection recommendation, and to give due consideration to the two positive post-rebuttal reassessments. We believe our work makes a meaningful contribution to the community by advancing the use of LLM domain knowledge for sample-efficient black-box optimization.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.