acceptodds
Under review as a conference paper at ICLR 2027

Which Variables Matter? LLM-Informed Competitive Relevance Learning for High-Dimensional Bayesian Optimization

Abstract

High-dimensional design problems are common in scientific discovery and formulation optimization, where effective designs must be identified under tight experimental budgets. However, existing high-dimensional BO methods often struggle to uncover useful structural biases from sparse observations alone. To address this challenge, we exploit the pronounced hierarchy of variable relevance and propose CoRe-BO, a framework for LLM-informed competitive relevance learning. Rather than asking the LLM for precise relevance scores or direct kernel modifications, CoRe-BO uses coarse judgments about variable importance to initialize a task-aware relevance prior, with variables competing for a shared sensitivity budget. As experiments accumulate, this structure is continually adapted through bounded LLM reflection and posterior inference, progressively concentrating modeling and search on influential variables. A regret analysis characterizes how the quality of LLM relevance guidance affects search complexity and the cost of imperfect priors. Experiments on six scientific optimization tasks with 29–150 variables show consistent gains in both convergence and final optimization performance over high-dimensional and LLM-assisted BO baselines. Our results suggest that CoRe-BO provides a practical and theoretically grounded approach to integrating LLMs into high-dimensional scientific optimization.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Bayesian Optimization, AI for Science

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_bDDLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoRe-BO, a method that leverages scientific knowledge from large language models to guide high-dimensional Bayesian optimization. It maps LLM-generated variable importance scores and estimates of the number of influential variables into a hierarchical Dirichlet prior, allowing variables to compete for a shared Gaussian process sensitivity budget. As observations accumulate, posterior inference and KL-constrained online reflection refine variable relevance, while predictive likelihoods adaptively calibrate the influence of semantic guidance. The authors provide a regret analysis for a fixed-prior UCB variant and report improved optimization performance over several baselines on six high-dimensional scientific tasks and three additional AutoML tasks in the appendix.

Critical strengths
  1. Scientific variables often have clear physical or chemical meanings, making LLM-generated relative importance well motivated. The competitive relevance prior translates these semantic judgments into GP sensitivity allocations, directly connecting domain knowledge with structural learning in high-dimensional optimization.

  2. The method separately models overall sensitivity, relative importance, and relevance breadth. Posterior inference, bounded online revision, and adaptive trust allow experimental evidence to correct inaccurate semantic guidance, reducing dependence on the initial LLM judgments.

  3. The authors provide a regret analysis for a fixed-prior UCB variant, characterizing the trade-off between reduced search complexity and the cost of semantic misspecification. Although the analysis does not cover the full adaptive algorithm, it offers a valuable theoretical explanation of the design.

  4. The method demonstrates faster convergence and better final performance on six high-dimensional scientific tasks, with means and standard errors reported over 5 random seeds. Extensive ablations examine the contributions of the shared sensitivity budget, semantic calibration, online reflection, and adaptive trust. The appendix further includes AutoML evaluations and analyses of robustness to different LLM configurations.

Critical weaknesses
  1. The core novelty of this work is to use LLM to judge the importance of each feature, but there already exists several similar approaches. For example, Physics-informed, dual-objective optimization of high-entropy-alloy nanozymes by a robotic AI chemist, Matter 2025, uses GPT-4 proposed statistical analysis as prior knowledge, and Language-Based Bayesian Optimization Research Assistant (BORA), IJCAI25, similarly uses LLM-proposed prior knowledge. Although these approaches are not exactly the same, they still somehow hurt the key novelty of this project. In addition, BORA is not used as a baseline in this work.

  2. The authors didn't provide the code of their work in supplementary materials.

  3. The authors used 6 real scientific problems to evaluate the BO algorithms, with no synthetic functions. However, they did not illustrate how the algorithms are evaluated on these problems. Usually there is only a tabular data with each published scientific paper, so I wonder if the authors used the tabular dataset to train an oracle model, and query it as ground truth, or they searched the table for the nearest point from the suggestion point, or probably they did the wet experiments themselves, which is technically not impossible.

  4. The authors used LLM to score the feature importances, which is a common approach. However, there is a paper in ICLR26 talking about the inconsistency of scoring and comparison (TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them): for two good items a and b, maybe a is better than b, but LLM may possibly score a as 8 and b as 9, because 8 and 9 are both relatively good scores. Therefore, the scoring system from the LLM needs to be further evaluated.

  5. The author's approach directly change the lengthscale of each feature in the kernel function. In BO, sometimes the lengthscale may be optimized to a extremely large or extremely small value, both of which may negate the author's effort to modify them. The paper should contain discussion on this problem.

Questions for authors

Please see the weaknesses.

7 comments
AuthorsOpenReview ↗

Response to Reviewer bDDL — Part 1 of 2

We extend our sincere gratitude for dedicating your valuable time to reviewing our paper. In the following, we provide a comprehensive response to your comments.

W1: Comparisons with prior work

Thank you for highlighting these relevant studies. We would like to clarify that CoRe-BO focuses on high-dimensional BO, offering an interpretable and stable framework that connects LLM guidance with relevance structure learning for sample-efficient experimentation. We list the key differences and provide an empirical comparison below.

(1) Distinct research problem. CoRe-BO addresses the difficulty of discovering a useful relevance structure from sparse observations in high-dimensional BO, evaluated on 29–150D scientific tasks. The Matter study focuses on jointly optimizing reaction rate and substrate affinity in five-component alloys. BORA focuses on incorporating contextual knowledge into scientific optimization and overcoming search stagnation, with evaluations up to 15D. Neither work investigates how to harness LLM capabilities specifically for high-dimensional BO.

(2) Methodology differences. The Matter study augments acquisition functions with physics-based predictors and uses GPT-4 to suggest elemental proportion adjustments in subsequent searches. BORA uses LLM hypotheses to generate or select candidates under controlled interventions. Rather than using LLM outputs to propose or select candidates or bias acquisition scores, CoRe-BO introduces a hierarchical Dirichlet prior that makes variables compete for a shared GP sensitivity budget. LLM judgments about relative importance and the number of influential variables calibrate its allocation mean and concentration hyperprior; posterior inference and KL-bounded semantic updates adaptively refine the relevance structure. Neither work develops this competitive relevance prior or the associated semantic calibration.

(3) Empirical comparison. We evaluated BORA on all six benchmarks over 5 seeds and report Superhard and Ferroelectric results below. We do not include the Matter framework as a direct baseline because it requires task-specific physics predictors that are unavailable for these benchmarks. Final best objective values are reported as mean ± SD (higher is better).

DatasetCoRe-BOBORA
Superhard296.1±0.3284.3±16.7
Ferroelectric597.1±6.7525.4±21.5

CoRe-BO achieves higher mean performance on both tasks, supporting the effectiveness of using LLM guidance for adaptive relevance learning rather than direct candidate generation or selection in high-dimensional BO. We have added descriptions of both approaches in Appendix E and included a comparison with BORA on all six datasets in Figure 5.

 

W2: Code availability

Thank you for pointing this out. We have updated the Supplementary Material to include the code implementation of CoRe-BO.

 

W3: Task evaluation protocol

Thank you for highlighting the need to clarify the evaluation protocol. In our experiments, each task uses a fixed computational oracle to evaluate candidate designs. Perovskite, Superhard, Formation stability, and Superconductor use random-forest (RF) oracles fitted to tabulated materials data; Ferroelectric uses an RF distilled from the released FerroAI model to enable extensive evaluation on CPU-only machines; and Nutrition uses an explicit objective computed from a food–nutrient table. At each BO iteration, the selected candidate is evaluated by the corresponding oracle, and the returned objective value is used to update the BO surrogate. We have revised Appendix B to provide further details on oracle construction and evaluation.

AuthorsOpenReview ↗

Response to Reviewer bDDL — Part 2 of 2

W4: LLM scoring consistency

Thank you for raising this concern. We would like to clarify that CoRe-BO is designed to accommodate inconsistent LLM scores, and our experiments show that it remains robust to initial score perturbations.

(1) How CoRe-BO handles imperfect scores. While TrustJudge seeks to improve the consistency of LLM judgments of generated responses, we aim to make high-dimensional BO robust to inconsistencies and other imperfections in LLM outputs. To this end, CoRe-BO provides an integrated framework to harness LLM guidance. Considering the potential inconsistencies in LLM scores, CoRe-BO uses these informative yet coarse judgments to initialize a prior over variable relevance, rather than directly treating them as objective-value predictions. Relevance remains a latent random quantity under this prior and is inferred through posterior sampling from experimental observations. As optimization proceeds, data-driven posterior inference and LLM revision refine the importance structure, while adaptive trust regulates the strength of LLM guidance.

(2) Robustness evidence. To assess robustness to inconsistent and stochastic LLM scores, we perturb the initial importance scores on Ferroelectric using nearby-score swaps affecting approximately 20% of variables or Gaussian noise with standard deviations of 0.5, 1, and 2. We then run CoRe-BO with other settings unchanged; results below are mean ± SD (higher is better).

PerturbationAUCFinal
None569.9±12.5597.1±6.7
Gaussian 0.5562.8±19.3594.6±9.9
Gaussian 1557.1±17.8586.7±10.9
Gaussian 2565.4±13.9596.4±5.2
Swaps558.9±15.0587.1±9.4

Mean AUC and final objective values decrease by at most 2.25% and 1.75%, respectively, supporting robustness to initial score perturbations. Appendix G.3 also shows similar final performance across five Superhard runs despite differences in LLM guidance sampled at temperature 0.7. We have added the above discussion and perturbation results, including additional experiments on Superhard, to Appendix G.4.

 

W5: LLM guidance on lengthscale

Thank you for raising this point. In our framework, LLM judgments shape a prior over variable relevance (inverse squared lengthscales, ρi=ℓi−2\rho_i=\ell_i^{-2}), while relevance remains latent and is refined from experimental observations. To continually leverage LLM guidance during optimization, we let the LLM revise its judgments online and adapt its influence using predictive evidence. The LLM's influence increases when it informs an effective relevance structure, as illustrated by the rising trust on Ferroelectric in Figure 11(d). Removing semantic prior injection (Figures 4(a) and 6, Data-only) or online revision (Figures 4(b) and 12) reduces performance, supporting the contribution of LLM guidance on the relevance structure throughout optimization.

Regarding lengthscale behavior, we note that the shared relevance budget constrains excessive relevance, thereby mitigating overly small lengthscales. The LLM-informed Dirichlet model encourages relevance to concentrate on a few influential variables, leaving most dimensions with low relevance and correspondingly large lengthscales that suppress their influence. Figure 3(a) confirms this allocation pattern in the learned posterior.

Reviewer_bDDLOpenReview ↗

Response

The rebuttal clarifies the evaluation protocol and establishes meaningful methodological differences from BORA. I also acknowledge the shuffled-prior and matched no-budget ablations as useful evidence. However, I still have remained concerns about whether the proposed construction provides sufficient additional value over simpler ways of incorporating the same LLM-derived relevance information into established ARD or sparsity-inducing priors. The existing comparisons only partially resolve this question. Furthermore, the fixed-kernel regret analysis does not establish a specific advantage of the competitive prior, and the oracle-based evaluations leave the broader applicability of the gains uncertain. For now, I will keep my assessment unchanged.

Reviewer_eQDTOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This manuscript describes a Bayesian Optimization (BO) method called CoRe-BO for high-dimenstional BO. It introduces two main methodological novelties: the use of a fixed relevance budget for assigning importance to variables and the use of LLMs via sparse relevance updates and an adaptive LLM trust calibration mechanism to redirect the search to relevant subspaces. While other methods have proposed LLM guidance in BO, CoRe-BO provides a new architecture that proposes to better exploit LLMs’ capabilities.

The method is tested on 6 scientific optimization tasks (with up to 150 variables) and 3 additional ML predictor creation tasks. The method is reported to have significantly better performance than a number of traditional BO and LLM-assisted BO prior methods. It has competivie running time with other LLM-assisted BO methods.

Critical strengths
  1. The method indroduces several well-defined and interesting methodological advances. The competitive budget, which does seem to contribute significantly to performance, is one. The LLM use for semantic prior calibration via Dirichlet priors is another. A third is allowing the LLM to actively query BO trace observations in several ways, rather than simplying providing the LLM with the traces. A forth is the adaptive trust calculation that determines how much importance to place on the LLM updates based on how much the LLM updates have improved the ongoing optimization relative to not having used them.

  2. The method is extensively tested on 9 tasks: 6 “optimal mixture” tasks (main text) and 3 ML predictor creation tasks (Appendix I). The performance improvements in all tasks is quite strong, with the method improving faster early and also always achieving the best final performance. The running time analysis suggests that it does this with smaller total wall-clock time than other LLM-based BO methods (though not as quickly as non-LLM BO methods).

  3. The paper is very clearly written and logically argued, with extensive experimental results. There are ablation studies that show that each component of CoRe-BO is contributing to its final performance. Ablation studies include using LLMs for only each part of the priors, eliminating LLM updates, eliminating LLM calibration, eliminating LLM use, no shared budget constraint, and several others (Appendix F). Alternative LLMs are also tested and show that broadly the method is performant with alternative LLMs.

Critical weaknesses
  1. Theorem 1 is never related to the experimental results. This isn’t a major weakness since Theorem 1 isn’t on any critical path to showing that the method works in practice. But at present, the theory aspect of the paper (which is not extensive) feels disconnected from the main story of the empirical results.
Questions for authors
  1. The prompt in Appendix D asks the LLM to pre-select a query type to examine once the observations become available. Why is the LLM asked to propose the next query (VERIFY_DIM, COMPARE_DIM, VERIFY_GROUP) before it sees the effect of its initial choices? I see that this could help identify where it thinks it’s least certain, but it’s not clear how this is used.

  2. Is the LLM given definitions of VERIFY_DIM, COMPARE_DIM, and VERIFY_GROUP? The included example prompts tell the LLM that those functions exist, but don’t explain to it what they do? Is there a resonable analysis that could show that the LLM makes good choices about these queries? (I.e. a post-hoc analysis asking whether the LLM asked the most informative question of the observations?)

  3. How accurate is the LLM’s estimation of the range of the number of relevant dimensions and does that range change over time?

4 comments
AuthorsOpenReview ↗

Response to Reviewer eQDT — Part 1 of 2

Thank you for your careful review. We appreciate your recognition of the novelty of our method and the strength of our experimental evaluation. We respond to your questions point by point below.

W1: Connection between theory and experiments

Thank you for highlighting the need to clarify this connection. Our theoretical formulation enables a sound and interpretable analysis of how semantic relevance shapes search complexity and how its misalignment with the objective incurs a cost. Several experimental findings support this interpretation:

  • Structured and aligned guidance. Theorem 1 characterizes the trade-off between search complexity and semantic alignment error. Figure 3 shows that CoRe-BO identifies a selective relevance structure aligned with objective sensitivity and focuses search on a few influential variables. These findings are consistent with a favorable balance between search complexity and semantic alignment, jointly helping explain the performance gains.
  • The cost of misalignment. Shuffling relevance values across variable identities slows early optimization (Figure 4(a)), consistent with the cost of semantic misspecification identified by the theory.
  • Correcting guidance and calibrating its influence. This misalignment cost motivates online reflection to correct guidance and adaptive trust to calibrate its influence, with ablations supporting the effectiveness of both mechanisms (Figure 4(b)).

We have revised the Experiments section to explain how these findings provide empirical support for the mechanism and trade-off characterized by Theorem 1.

 

Q1: Query pre-selection and usage

Thank you for your comment. The 20 initial designs and their objective values can form a large numerical context, particularly in settings with more than 100 dimensions. During semantic initialization, we therefore ask the LLM to propose a query to select observations from these designs that it expects to be useful for revising its beliefs. During online revision, subsequent queries are informed by the previously selected evidence. We have clarified this design rationale in Appendix D.

AuthorsOpenReview ↗

Response to Reviewer eQDT — Part 2 of 2

Q2: Query definitions and informativeness

Thank you for highlighting this omission. Our experimental system prompt already defined the query types, and we have now added these definitions to Appendix D. For example, the prompt defines VERIFY_DIM as assessing whether the available empirical evidence gives a reason to reconsider the relevance currently assigned to one variable. The system prompt also clarifies that queries retrieve existing evidence rather than request new experiments; the LLM then uses this evidence to produce sparse relevance updates during reflection.

To assess whether the LLM selects informative questions about the available observations, we examine whether its queries improve belief revision and subsequent optimization. The LLM does not expose an explicit, calibrated distribution over its evolving relevance beliefs, making direct estimation of query information gain difficult. We therefore evaluate query utility through a controlled ablation: we replace only the LLM-selected query dimensions with randomly selected dimensions, retaining the query types, evidence retriever, and remaining algorithm components.

We quantify the benefit of query-guided belief revision using cosine similarity between the revised relevance allocation m\mathbf m and an objective-derived sensitivity reference p\mathbf p. Higher similarity indicates closer agreement between the revised beliefs and the relative influence of variables on the objective. To construct this reference, we vary one input variable at a time while holding the others fixed and denote the resulting average squared change in the objective by viv_i. Each variable's share of the total measured sensitivity is pi=vi∑jvjp_i=\frac{v_i}{\sum_j v_j}.

The table reports initial alignment and alignment after each reflection. Both strategies share the same initial relevance belief, generated alongside the first LLM query during deterministic semantic initialization (temperature=0).

Observations availableSuperhard: LLM queriesSuperhard: random queriesFerro: LLM queriesFerro: random queries
00.310.310.790.79
200.57±0.0510.42±0.120.83±0.0940.81±0.058
300.62±0.0670.43±0.190.87±0.0550.81±0.074
400.71±0.0950.41±0.150.96±0.0280.82±0.083
500.65±0.0710.43±0.180.91±0.0520.84±0.088
600.68±0.0930.43±0.170.93±0.0270.84±0.080
700.69±0.0910.46±0.190.93±0.0290.84±0.077
800.70±0.0920.49±0.200.94±0.0250.83±0.078
900.71±0.0950.48±0.230.96±0.0280.85±0.082

LLM-selected queries yield higher mean alignment at every reflection checkpoint on both tasks. Randomizing query targets also reduces AUC and the final best objective value relative to the default configuration:

DatasetMetric ↑LLM QueryRandom queries
SuperhardAUC293.1±1.2287.2±3.5
SuperhardFinal296.1±0.3294.5±0.5
FerroAUC569.9±12.5535.3±24.7
FerroFinal597.1±6.7548.0±19.7

Together, these results support the practical value of LLM query selection: it guides relevance revisions toward objective sensitivity and improves subsequent optimization.

 

Q3: Range of relevant dimensions

Thank you for this question. We ask LLMs to estimate the effective-dimension range [dL,dU][d_L,d_U] only once at initialization. We combine this range with the allocation mean m0\mathbf m_0 derived from LLM importance scores to calibrate a hyperprior over the concentration parameter α\alpha via Dirichlet geometry. We find that estimating this range is more challenging for LLMs than assessing relative variable importance, plausibly because it requires more precise judgments about effect magnitudes. We therefore allow α\alpha to be continually refined from observations.

To assess the accuracy of the LLM's effective-dimension estimates, we use the objective-derived sensitivity reference p\mathbf p defined in our response to Q2 and compute defforacle=1∑ipi2d_{\mathrm{eff}}^{\mathrm{oracle}}=\frac{1}{\sum_i p_i^2}. This effective dimension equals kk when kk variables are equally influential and the others have no effect. It becomes smaller as sensitivity concentrates on fewer variables, providing an objective-based reference for the number of influential variables. The resulting reference is 12.7±3.112.7\pm3.1 on Ferroelectric and 6.0±1.26.0\pm1.2 on Superhard.

Intern-S1 produces identical estimates at the decoding temperature of 0 used in our main experiments. We additionally queried it 4 times at temperature 0.7. The mean lower and upper bounds were [8.6,15.8][8.6,15.8] on Ferroelectric and [6.4,8.2][6.4,8.2] on Superhard. Overall, these estimates are reasonably accurate. The former broadly agrees with the objective-based reference, while the latter is slightly higher. Both suggest a relatively small number of influential variables. We appreciate the reviewer’s suggestion to examine this aspect.

Thank you for the responses. I think they continue to support my positive score.

AuthorsOpenReview ↗

Thank you for your response

We sincerely appreciate your constructive suggestions, which have helped us clarify the connection between our theoretical analysis and experimental results and better articulate our design choices. Thank you again for your support of our work.

Reviewer_235VOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+2)
Read review
Paper summary

The paper uses LLM feedback in assessing which variables are influential for high dimensional Bayesian optimization, by learning a prior based on LLM's response on questions pertaining to which variables are likely to matter more and important. Authors support the method with experimental results across a range of tasks and comparison with baselines, where the method outperforms the baselines. Authors also provide trade-off dilemma arising in the decision making using regret guarantees for GP-UCB as a reference.

Critical strengths

The paper is well written, easy to follow and provides a mathematically sound way of integrating LLM feedback in deciding variable importance in the prior design. It is exciting to see a principled approach for integrating LLM feedback in BO loop. As a tool this can be very effective in sample efficiency, and potentially a lot of otherwise high dimensional systems can become solvable using GP surrogates and BO. This is important because the benefits of GP in modeling of epistemic uncertainty are not captured by other parametric probabilistic models, and dimensionality is the key reason that restricts the use of GPs, which can hopefully be alleviated by this work and its future developments.

Critical weaknesses
  1. Ablation studies for LLM choices: While the ablation analysis covers a range of possibilities, LLM ablation is not tested across all tasks. This is important to ensure that benefit of this approach is not heavily correlated with LLM choice. Furthermore, qualitative analysis of whether different LLMs vary significantly in their responses and the consequent variable selection matters to the users.

  2. As a followup, it is important that a user is provided with sufficient information on how to choose the right model when integrating in their BO loop. The paper currently lacks any initial feedback/report on the LLM selection (without fully executing the BO loop), that can help a user decide what to choose or not. I believe this is crucial to establish relevance of this work, otherwise it is impractical to apply this to a new problem, and observe inefficient performance, swap around the LLM until one gets reasonable results. It is not expected that we know everything about the model performance in advance, but attempts must be made to provide some initial diagnosis and the limitations of such diagnosis must be made clear.

Questions for authors
  1. Is it possible to provide a more exhaustive LLM ablation for all tasks? What is the response variance among different models like? Do all models lead to same set of variable selection or do they differ in practice?
  2. What is the justification for choosing 150D as the highest dimension for experiments, when there have been documented cases of BO application with GP surrogates with even higher dimension (ref. TuRBO paper considers a 200D ackley function, and more, see ref [1] below) ?

[1] Hvarfner, Carl, Erik Orm Hellsten, and Luigi Nardi. "Vanilla Bayesian optimization performs great in high dimensions." arXiv preprint arXiv:2402.02229 (2024)

7 comments
AuthorsOpenReview ↗

Response to Reviewer 235V — Part 1 of 2

We sincerely thank you for taking the time to review our paper carefully and for your thoughtful and constructive comments. We address your questions point by point below.

W1&Q1: LLM ablations and cross-model variability

Thank you for this question. We answer it from three perspectives: optimization performance across LLMs, a qualitative comparison of their initial variable selections, and quantitative agreement in variable selections before and after optimization.

(1) Performance across LLMs. We have extended the LLM ablation to the other four datasets—Perovskite, Formation stability, Superconductor, and Nutrition—complementing the Ferroelectric and Superhard results in Figures 4(c) and 13. Final best objective values are reported below as mean ± SD (higher is better).

LLMPerovskiteFormation stabilitySuperconductorNutrition
Intern-S17.67±0.094.41±0.03120.94±5.860.87±0.01
Intern-S1-mini7.51±0.134.34±0.02113.44±4.200.83±0.03
Qwen3-Instruct7.45±0.134.17±0.34115.40±1.070.79±0.04
Qwen3-Thinking7.60±0.984.28±0.48119.10±6.710.86±0.01

CoRe-BO maintains strong final performance across all four LLM choices. Together with the existing results, this supports our method's robustness to LLM choice.

(2) Qualitative initial selection. For a qualitative comparison, we list each LLM's ten highest-rated variables at initialization on Ferroelectric.

LLMInitial top-10 variables
Intern-S1Ti,Zr,Nb,Ba,Ta,Sr,La,Pb,K,Na
Intern-S1-miniTi,Zr,Nb,Ba,Ta,Sr,La,K,Na,Bi
Qwen3-InstructTi,Zr,Nb,Ba,Ta,Sr,La,Pb,Bi,O
Qwen3-ThinkingTi,Zr,Nb,Ba,Ta,Sr,La,Pb,K,O

The four models' responses share seven top-10 variables, indicating agreement in their initial judgments.

(3) Initial and final selection agreement. We further compare initial and final top-10 agreement across all six tasks using Jaccard similarity, J(A,B)=∣A∩B∣∣A∪B∣J(A,B)=\frac{|A\cap B|}{|A\cup B|}, where AA and BB are two models' top-10 variable sets. Initial values average the six model-pair Jaccard similarities under deterministic LLM scoring (temperature=0). Final top-10 sets are ranked by posterior mean relevance; we average their six pairwise similarities within each seed and report mean ± sample SD across 5 seeds.

DatasetLLM Top-10 JaccardFinal BO Top-10 Jaccard
Perovskite0.820.89±0.027
Superhard0.430.83±0.033
Formation stability0.440.81±0.030
Superconductor0.790.89±0.025
Ferroelectric0.740.82±0.051
Nutrition0.720.78±0.022

Every model pair initially shares at least five variables. After optimization, agreement increases on all six tasks, indicating that different LLM backbones lead to more consistent final variable selections. These results support the role of the full CoRe-BO framework in reducing sensitivity to LLM choice. We have added the above results to Appendix G.1.

 

W2: LLM choice

Thank you for this comment. We would like to clarify that CoRe-BO is a general framework whose design is not tailored to any particular LLM's output characteristics. Our goal is to support a broad range of LLMs. The following mechanisms enhance CoRe-BO's robustness to variability in LLM outputs and reduce reliance on a particular model:

  • Flexible relevance prior. Our LLM-informed CoRe prior encodes the common scientific structure of a few dominant variables, while posterior inference progressively refines relevance beliefs as experimental evidence accumulates.
  • Continual revision. LLM online reflection uses accumulating evidence to revise its judgments and correct biases arising from domain knowledge gaps.
  • Adaptive influence. Predictive evidence dynamically adjusts LLM influence, strengthening useful guidance and downweighting misleading judgments.

We test this robustness across four LLMs and six tasks, examining optimization performance and providing variable selection analysis (W1; Appendix G.1). Further experiments with varied prompt wording and stochastic decoding reinforce this robustness to differences in LLM output quality (Appendices G.2–G.3). Together, these studies support CoRe-BO's generality and its ability to maintain strong performance without relying on a carefully selected LLM.

AuthorsOpenReview ↗

Response to Reviewer 235V — Part 2 of 2

Q2: Experimental dimensionality

Thank you for raising this question. We would like to clarify that our 150D setting is substantial for data-limited scientific optimization, where conventional BO is typically most effective with tens of variables [1]. While prior work evaluates BO on higher-dimensional benchmarks, including the 200D Ackley function (Eriksson et al., 2019) and policy-optimization tasks with up to 6,392 dimensions (Hvarfner et al., 2024), their input variables lack clear scientific semantics. This makes them less suited to evaluating CoRe-BO, which uses the semantic meaning of variables and objectives to inform relevance learning. Nevertheless, we agree that evaluation beyond 150D would provide a more complete assessment of CoRe-BO’s effectiveness in higher-dimensional optimization.

We extend our evaluation to two 273D scientific tasks: (1) MOF CO₂ adsorption optimizes pore-geometry and chemical-environment descriptors to maximize CO₂ uptake at 298 K and 0.15 bar; (2) JARVIS bulk modulus optimizes compositional and structural descriptors to maximize resistance to volumetric compression. Dataset descriptions, the evaluation protocol, and optimization curves are provided in Appendix J. Results on the two 273D tasks are shown below (higher is better).

MethodMOF AUCMOF FinalJARVIS AUCJARVIS Final
CoRe-BO8.0±0.049.1±0.1392.7±7.3421.8±25.1
Vanilla BO7.2±0.48.4±0.8352.6±4.6379.7±4.2
TuRBO5.6±0.26.1±0.7309.7±53.7356.8±58.3
SAASBO6.4±1.97.2±2.7373.5±3.5394.5±0.0
BOPRO5.1±0.15.2±0.1331.3±60.1363.7±57.7
LGBO5.5±0.66.1±1.3359.5±7.3381.6±6.9

CoRe-BO achieves the highest mean AUC and final objective value on both tasks, indicating faster progress and better final solutions within the evaluation budget. These results extend the evidence for its effectiveness to higher-dimensional scientific optimization.

[1] Chuan He, Martin Singull, and T. Jesper Jacobsson. Bayesian Optimisation for the Experimental Sciences: A Practical Guide to Data-Efficient Optimisation of Laboratory Workflows. Advanced Intelligent Systems, 8(5):e202501149, 2026.

Thank you for addressing the questions and weaknesses. Looking at other reviewer's questions, I also found Reviewer bDDL's concerns regarding scoring inconsistency and the risk of directly modifying the lengthscales to be particularly relevant concerns. The robustness experiments and the inclusion of 273D tasks address my key concerns to a great extent. A final question to the authors: in the LLM ablation result, can we attribute consistently good performance across LLM models to specific parts of the pipeline? In my experience, several models may agree on initial parameter selection but there are occasional outliers, wherein a model maybe overly conservative. Hypothetically, if there is a model that proposes a very different list of initial variables for example, can you elaborate how the framework safeguards against it? The reason for asking this question is also to encourage authors to reflect this distinction in their writeup and enable better interpretability of their pipeline, instead of relying solely on empirical evidence to confirm robustness.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

Dear Area Chair and Reviewers,

We sincerely appreciate the time and care you have devoted to reviewing our work. We have provided detailed responses to all concerns and questions and revised the manuscript accordingly. We believe these responses and revisions address the issues raised and clarify the scope and contributions of our work. Your constructive suggestions have helped us improve the paper’s clarity and completeness. All revisions are marked in blue, and our point-by-point responses include references to the relevant sections, figures, and tables to make the changes easy to locate. Thank you again for your thoughtful feedback. We welcome further discussion of any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.