- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CoRe-BO, a method that leverages scientific knowledge from large language models to guide high-dimensional Bayesian optimization. It maps LLM-generated variable importance scores and estimates of the number of influential variables into a hierarchical Dirichlet prior, allowing variables to compete for a shared Gaussian process sensitivity budget. As observations accumulate, posterior inference and KL-constrained online reflection refine variable relevance, while predictive likelihoods adaptively calibrate the influence of semantic guidance. The authors provide a regret analysis for a fixed-prior UCB variant and report improved optimization performance over several baselines on six high-dimensional scientific tasks and three additional AutoML tasks in the appendix.
Critical strengths
-
Scientific variables often have clear physical or chemical meanings, making LLM-generated relative importance well motivated. The competitive relevance prior translates these semantic judgments into GP sensitivity allocations, directly connecting domain knowledge with structural learning in high-dimensional optimization.
-
The method separately models overall sensitivity, relative importance, and relevance breadth. Posterior inference, bounded online revision, and adaptive trust allow experimental evidence to correct inaccurate semantic guidance, reducing dependence on the initial LLM judgments.
-
The authors provide a regret analysis for a fixed-prior UCB variant, characterizing the trade-off between reduced search complexity and the cost of semantic misspecification. Although the analysis does not cover the full adaptive algorithm, it offers a valuable theoretical explanation of the design.
-
The method demonstrates faster convergence and better final performance on six high-dimensional scientific tasks, with means and standard errors reported over 5 random seeds. Extensive ablations examine the contributions of the shared sensitivity budget, semantic calibration, online reflection, and adaptive trust. The appendix further includes AutoML evaluations and analyses of robustness to different LLM configurations.
Critical weaknesses
-
The core novelty of this work is to use LLM to judge the importance of each feature, but there already exists several similar approaches. For example, Physics-informed, dual-objective optimization of high-entropy-alloy nanozymes by a robotic AI chemist, Matter 2025, uses GPT-4 proposed statistical analysis as prior knowledge, and Language-Based Bayesian Optimization Research Assistant (BORA), IJCAI25, similarly uses LLM-proposed prior knowledge. Although these approaches are not exactly the same, they still somehow hurt the key novelty of this project. In addition, BORA is not used as a baseline in this work.
-
The authors didn't provide the code of their work in supplementary materials.
-
The authors used 6 real scientific problems to evaluate the BO algorithms, with no synthetic functions. However, they did not illustrate how the algorithms are evaluated on these problems. Usually there is only a tabular data with each published scientific paper, so I wonder if the authors used the tabular dataset to train an oracle model, and query it as ground truth, or they searched the table for the nearest point from the suggestion point, or probably they did the wet experiments themselves, which is technically not impossible.
-
The authors used LLM to score the feature importances, which is a common approach. However, there is a paper in ICLR26 talking about the inconsistency of scoring and comparison (TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them): for two good items a and b, maybe a is better than b, but LLM may possibly score a as 8 and b as 9, because 8 and 9 are both relatively good scores. Therefore, the scoring system from the LLM needs to be further evaluated.
-
The author's approach directly change the lengthscale of each feature in the kernel function. In BO, sometimes the lengthscale may be optimized to a extremely large or extremely small value, both of which may negate the author's effort to modify them. The paper should contain discussion on this problem.
Questions for authors
Please see the weaknesses.
7 comments
Response to Reviewer bDDL — Part 2 of 2
W4: LLM scoring consistency
Thank you for raising this concern. We would like to clarify that CoRe-BO is designed to accommodate inconsistent LLM scores, and our experiments show that it remains robust to initial score perturbations.
(1) How CoRe-BO handles imperfect scores. While TrustJudge seeks to improve the consistency of LLM judgments of generated responses, we aim to make high-dimensional BO robust to inconsistencies and other imperfections in LLM outputs. To this end, CoRe-BO provides an integrated framework to harness LLM guidance. Considering the potential inconsistencies in LLM scores, CoRe-BO uses these informative yet coarse judgments to initialize a prior over variable relevance, rather than directly treating them as objective-value predictions. Relevance remains a latent random quantity under this prior and is inferred through posterior sampling from experimental observations. As optimization proceeds, data-driven posterior inference and LLM revision refine the importance structure, while adaptive trust regulates the strength of LLM guidance.
(2) Robustness evidence. To assess robustness to inconsistent and stochastic LLM scores, we perturb the initial importance scores on Ferroelectric using nearby-score swaps affecting approximately 20% of variables or Gaussian noise with standard deviations of 0.5, 1, and 2. We then run CoRe-BO with other settings unchanged; results below are mean ± SD (higher is better).
| Perturbation | AUC | Final |
|---|---|---|
| None | 569.9±12.5 | 597.1±6.7 |
| Gaussian 0.5 | 562.8±19.3 | 594.6±9.9 |
| Gaussian 1 | 557.1±17.8 | 586.7±10.9 |
| Gaussian 2 | 565.4±13.9 | 596.4±5.2 |
| Swaps | 558.9±15.0 | 587.1±9.4 |
Mean AUC and final objective values decrease by at most 2.25% and 1.75%, respectively, supporting robustness to initial score perturbations. Appendix G.3 also shows similar final performance across five Superhard runs despite differences in LLM guidance sampled at temperature 0.7. We have added the above discussion and perturbation results, including additional experiments on Superhard, to Appendix G.4.
W5: LLM guidance on lengthscale
Thank you for raising this point. In our framework, LLM judgments shape a prior over variable relevance (inverse squared lengthscales, ), while relevance remains latent and is refined from experimental observations. To continually leverage LLM guidance during optimization, we let the LLM revise its judgments online and adapt its influence using predictive evidence. The LLM's influence increases when it informs an effective relevance structure, as illustrated by the rising trust on Ferroelectric in Figure 11(d). Removing semantic prior injection (Figures 4(a) and 6, Data-only) or online revision (Figures 4(b) and 12) reduces performance, supporting the contribution of LLM guidance on the relevance structure throughout optimization.
Regarding lengthscale behavior, we note that the shared relevance budget constrains excessive relevance, thereby mitigating overly small lengthscales. The LLM-informed Dirichlet model encourages relevance to concentrate on a few influential variables, leaving most dimensions with low relevance and correspondingly large lengthscales that suppress their influence. Figure 3(a) confirms this allocation pattern in the learned posterior.
Response
The rebuttal clarifies the evaluation protocol and establishes meaningful methodological differences from BORA. I also acknowledge the shuffled-prior and matched no-budget ablations as useful evidence. However, I still have remained concerns about whether the proposed construction provides sufficient additional value over simpler ways of incorporating the same LLM-derived relevance information into established ARD or sparsity-inducing priors. The existing comparisons only partially resolve this question. Furthermore, the fixed-kernel regret analysis does not establish a specific advantage of the competitive prior, and the oracle-based evaluations leave the broader applicability of the gains uncertain. For now, I will keep my assessment unchanged.
Response to Reviewer bDDL — Part 1 of 2
We extend our sincere gratitude for dedicating your valuable time to reviewing our paper. In the following, we provide a comprehensive response to your comments.
Thank you for highlighting these relevant studies. We would like to clarify that CoRe-BO focuses on high-dimensional BO, offering an interpretable and stable framework that connects LLM guidance with relevance structure learning for sample-efficient experimentation. We list the key differences and provide an empirical comparison below.
(1) Distinct research problem. CoRe-BO addresses the difficulty of discovering a useful relevance structure from sparse observations in high-dimensional BO, evaluated on 29–150D scientific tasks. The Matter study focuses on jointly optimizing reaction rate and substrate affinity in five-component alloys. BORA focuses on incorporating contextual knowledge into scientific optimization and overcoming search stagnation, with evaluations up to 15D. Neither work investigates how to harness LLM capabilities specifically for high-dimensional BO.
(2) Methodology differences. The Matter study augments acquisition functions with physics-based predictors and uses GPT-4 to suggest elemental proportion adjustments in subsequent searches. BORA uses LLM hypotheses to generate or select candidates under controlled interventions. Rather than using LLM outputs to propose or select candidates or bias acquisition scores, CoRe-BO introduces a hierarchical Dirichlet prior that makes variables compete for a shared GP sensitivity budget. LLM judgments about relative importance and the number of influential variables calibrate its allocation mean and concentration hyperprior; posterior inference and KL-bounded semantic updates adaptively refine the relevance structure. Neither work develops this competitive relevance prior or the associated semantic calibration.
(3) Empirical comparison. We evaluated BORA on all six benchmarks over 5 seeds and report Superhard and Ferroelectric results below. We do not include the Matter framework as a direct baseline because it requires task-specific physics predictors that are unavailable for these benchmarks. Final best objective values are reported as mean ± SD (higher is better).
CoRe-BO achieves higher mean performance on both tasks, supporting the effectiveness of using LLM guidance for adaptive relevance learning rather than direct candidate generation or selection in high-dimensional BO. We have added descriptions of both approaches in Appendix E and included a comparison with BORA on all six datasets in Figure 5.
Thank you for pointing this out. We have updated the Supplementary Material to include the code implementation of CoRe-BO.
Thank you for highlighting the need to clarify the evaluation protocol. In our experiments, each task uses a fixed computational oracle to evaluate candidate designs. Perovskite, Superhard, Formation stability, and Superconductor use random-forest (RF) oracles fitted to tabulated materials data; Ferroelectric uses an RF distilled from the released FerroAI model to enable extensive evaluation on CPU-only machines; and Nutrition uses an explicit objective computed from a food–nutrient table. At each BO iteration, the selected candidate is evaluated by the corresponding oracle, and the returned objective value is used to update the BO surrogate. We have revised Appendix B to provide further details on oracle construction and evaluation.