acceptodds
Under review as a conference paper at ICLR 2027

Evidence-Grounded Bayesian Optimization with Collective LLM Reasoning

Abstract

Expensive black-box optimization is challenging under tight evaluation budgets, particularly when reliable prior knowledge is limited. In such settings, context-informed warm starts can be ineffective or misleading. We propose a Bayesian optimization (BO) framework that grounds collective large language model (LLM) reasoning in experimental evidence. The framework uses design of experiments (DoE) to guide evaluations of the target system and derive statistical evidence that complements available prior information. Multiple LLM agents reason over this shared evidence, and their aggregated proposals guide search-region refinement and sequential BO. Our theoretical analysis establishes robustness properties of proposal aggregation, while experiments show that aggregated proposals achieve a lower mean normalized distance to reference optima than individual-agent proposals. On complex yet structured multimodal benchmarks with limited prior knowledge, our method reaches stringent regret targets with 8.8%–70% fewer evaluations than Gaussian process (GP)-based and LLM-assisted baselines. In FCNet hyperparameter-tuning ablations, evidence-based variants often reach top 0.5% configurations within 38 evaluations, earlier than non-evidence-based variants. These results support combining structured experimental evidence with collective LLM reasoning to improve the evaluation efficiency of black-box optimization.

Primary area:
optimization
Keywords:
Bayesian Optimization, Large Language Models, Design of Experiments, Black-Box Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_wr5sOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed an evidence-grounded Bayesian optimization framework under collective LLM reasoning. Instead of directly relying on an LLM, the authors first proposed using design of experiments to collect a small amount of data from the target system. Then they apply some statistical analysis and let multiple LLMs use the results to propose promising search regions. After this, the proposed method performs sequential optimization by combining LLM results, Bayesian optimization, and random exploration. The effect of LLMs decays as more and more observations are collected. The paper provides robustness results for this scheme. Some numerical studies are conducted to validate this idea.

Critical strengths

This paper provides an interesting integration of statistical evidence with LLMs. The idea of use of multiple-agent aggregation to improve robustness seems sound. The paper also provides theoretical analysis for the aggregation framework.

Critical weaknesses

1, The claim regarding the benefit of multi-agent aggregation is not properly justified by the experiment. The single-agent baseline uses only GPT-4o-Mini, while the multi-agent setting uses seven different models, so the improvements may come from stronger or more diverse models rather than from the aggregation itself.

2, There might be potential information leakage in the experimental design. For Rastrigin, the DoE configuration is explicitly chosen “because the global optimum lies at the origin,” which partially assumes that the ground truth is known before conducting the optimization algorithm.

Questions for authors

1, Empty Section A.6, “INITIALIZATION ABLATION SECOND EXPERIMENT.”

2, Incorrect figure numbering. Figure 3(a) is about the experiment on Rastrigin-2D, not FCNet Naval.

3, The notation is inconsistent. In the main theorem, it is A_s^{loc}, whereas in the appendix, it becomes A_r, which can be quite confusing. The same issue applies to the decay exponent: in the main experiment, it is k, whereas in the main text, it is \gamma.

4, Could the authors clarify how the aggregation guarantees translate into optimization robustness? In particular, the medoid theorem assumes a majority of sufficiently accurate agents, and the projected-mean theorem only provides a representation-space distance bound rather than an objective-value guarantee. What additional assumptions are required to justify the claimed robustness of the overall BO procedure?

3 comments
AuthorsOpenReview ↗

We would like to express our sincere appreciation for your insightful feedback and careful assessment. Our point-by-point responses are provided below.

Response to Reviewer wr5s

We sincerely thank the reviewer for the careful reading and constructive comments. We have expanded the experiments, improved the presentation, and clarified our theoretical results and their limitations. We address each concern below.

W1. Our motivation is that the best-performing LLM for a new optimization task is generally unknown beforehand. So it would be wise to trust widom of crowds rather than betting on one individual.

We extend the ablation to three strong models (GPT-5.6, Claude Opus 5.5, and Gemini 3.1 Pro), each evaluated individually and with seven independently sampled and aggregated proposals. We also include a heterogeneous ensemble of seven strong models and a statistical-only baseline. The LLM-based comparisons use matched DoE evidence and the same aggregation procedure.

Table 1 provides three findings. First, aggregation improves all three strong individual models. For example, Strong LLM 1 improves from 0.2288 to 0.1918 in average distance and from 33.90 to 30.09 in regret. Second, our original mixed-strength ensemble (0.1308; 19.91) outperforms all three homogeneous strong-model ensembles. Third, the heterogeneous ensemble of seven strong models achieves the best results (0.1107; 13.17).

These results support the wisdom of crowds: combining different models can outperform repeatedly sampling a single stronger model.

We also clarify the metrics. Avg. Dist. measures the average normalized Euclidean distance to a global optimum, while Regret measures the objective-value gap of initialization proposals, not final BO regret at budget TT. See Section 3, Table 1 and Figure 2.

W2. We deliberately excluded (0,0)(0,0) from the initial DoE for Rastrigin-2D because it is the known global optimum. A standard central composite design (CCD) includes this center point, which would make the benchmark trivially easy by identifying the optimum during initialization, leaving little room to evaluate subsequent optimization. We therefore used 8 CCD points and 5 maximin points, instead of 9 CCD and 4 maximin points.

Q1. We apologize for the confusing appendix presentation. The second initialization experiment is presented in Appendix Figure 8, immediately before Appendix A.6. It compares individual LLM proposals with projected-mean aggregation over 100 Monte Carlo replications.

The results show that aggregation consistently reduces the average normalized distance to the global optima across the six benchmarks, providing further evidence for collective reasoning. See Section 3 and Appendix Figure 8.

Q2. Thank you for identifying the incorrect Figure 3(a) reference. We have corrected it now. See revised Figure 3.

Q3. We agree that consistent notation is important for understanding and reproducing our framework. We have standardized the notation and simplified the pseudocode, moving detailed auxiliary procedures to the appendix.

In particular, we now define Li,sL_{i,s} as the scalar trust-region size around anchor xix_i at iteration ss. We also explicitly specify min-max normalization based on the original variable bounds:

[ \phi_j(x_j)=\frac{x_j-\ell_j}{u_j-\ell_j}. ]

We further clarify the local search domain and the LLM–GP decay parameter γ\gamma. See Sections 2.3–2.5, Appendix A.8, and Appendix B (Notation).

Q4. We appreciate the reviewer's concern about the difference between proposal robustness and end-to-end optimization guarantees. We have clarified the assumptions and conclusions of both aggregation theorems.

Theorem 1 shows that when a majority of LLM proposals are close to a reference region, their medoid also remains within a bounded distance. This deterministic bound does not require independence. Independence is required only for the probability statement, and using different LLMs does not automatically guarantee it.

Theorem 2 shows that averaging proposals reduces the mean squared distance to a reference point in the chosen representation space. Projecting the average back to the feasible domain preserves a related distance bound. Neither result directly guarantees lower objective regret.

Importantly, these are proposal-level guarantees, not end-to-end BO regret guarantees. An objective-value bound would require additional conditions connecting proposal distance to objective values, such as Lipschitz continuity and proximity to a global optimizer.

We also provide a mathematical analysis of the LLM–GP transition. With λ0=0.80\lambda_0=0.80, choosing γ=4\gamma=4 limits direct LLM selection to at most 5% of non-random queries after the midpoint. This explains the decay schedule but does not imply a universal regret guarantee. We explicitly show why our method does not inherit LLINBO's cumulative no-harm guarantee.

These clarifications separate what our theory establishes from what is supported by experiments. See Sections 2.3–2.4 and Appendices A.1, A.2, and A.9.

Dear Reviewer wr5s,

As the discussion phase comes to an end, please acknowledge the author's response and consider leaving them a comment on whether it has resolved some or all of your concerns.

Thanks!

Thank you for the detailed response and additional experiments. While several of my concerns have been addressed, I still have some reservations about the experimental design and the limited theoretical guarantees, and I believe the paper would benefit from further refinement.

Reviewer_5LFUOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a Bayesian optimization framework that combines experimental design, statistical analysis, and multiple LLMs to optimize black-box functions. The method first gathers information about the objective through structured experiments (DoE), then uses an ensemble of LLMs to identify promising search regions. Optimization proceeds by combining LLM suggestions, Gaussian Process-based search, and random exploration. The authors also provide theoretical results on aggregating LLM proposals and evaluate their approach on synthetic functions, hyperparameter tuning, and software configuration tasks, reporting improved sample efficiency over several baselines.

Critical strengths

The paper addresses a practical challenge in LLM-assisted Bayesian optimization: the unreliability of LLM guidance when prior knowledge about the objective function is limited. The empirical evaluation spans several optimization benchmarks and reports promising improvements in sample efficiency over existing baselines.

Critical weaknesses

W1. Insufficient baselines to establish the benefit of heterogeneous collective reasoning.

The multi-agent ablation compares an ensemble of seven heterogeneous LLMs against a single GPT-4o-Mini agent. This comparison does not isolate the contribution of model diversity from differences in model capability and inference budget.

I recommend including at least three strong individual models from the proposed ensemble as additional baselines, each independently sampled seven times, with their proposals aggregated using the same procedure as the heterogeneous ensemble. All methods should receive identical information and use matched inference budgets.

W2. Unclear necessity of LLM-based interpretation of statistical evidence.

Authors claim that LLMs can effectively interpret statistical evidence from DoE to identify promising search regions. However, the framework already computes statistical quantities such as main effects, interactions, feature importance, and surrogate-ranked candidates, which could be used directly to guide search-space refinement without LLMs. The experiments do not provide a clean comparison against a baseline that uses the same DoE evidence for region selection using simpler heuristics followed by GP-based optimization. It remains unclear whether LLM reasoning offers a meaningful advantage over a simpler statistical approach.

W3. Insufficient component isolation and unclear novelty.

The novelty of the paper is unclear. The proposed framework combines several established techniques, including DoE, multi-agent aggregation, search-region refinement, trust-region adaptation, and a time-varying mixture of LLM- and GP-guided optimization. Although the paper presents some ablation studies, these do not cleanly disentangle the contributions of the individual components. In particular, it is unclear how much of the observed improvement comes from better experimental initialization and search-space localization, as opposed to collective LLM reasoning.

Questions for authors

See weaknesses.

3 comments
AuthorsOpenReview ↗

Thank you for the thoughful review. Here is our response. We appcreciate your effort.

Response to Reviewer 5LFU

W1. We thank the reviewer for this important observation. We agree that the original comparison against GPT-4o-Mini does not fully distinguish the effects of model capability, proposal count, and aggregation.

Our motivation is that the most reliable LLM for a given optimization problem cannot always be identified beforehand. To address this concern, we expand our initialization ablation by introducing three strong individual LLMs (GPT-5.6, Claude Opus 5.5, and Gemini 3.1 Pro), each independently sampled seven times and aggregated using the same procedure as our heterogeneous ensemble. We additionally evaluate a heterogeneous ensemble of seven strong LLMs while retaining our original ensemble. All methods receive the same DoE evidence and prompt structure, with matched proposal counts across aggregated conditions.

As shown in the revised Table 1, all three homogeneous ensembles outperform their corresponding individual models, confirming that aggregation remains beneficial even with strong LLMs. More importantly, our original heterogeneous ensemble, despite including weaker models, outperforms all three strong individual models and their homogeneous ensembles, achieving an average distance of 0.1308 and regret of 19.91. It also remains competitive with the seven-strong-model heterogeneous ensemble, which achieves the best results (0.1107 and 13.17). These findings suggest that aggregation and model diversity contribute substantially to initialization quality, while stronger constituent models can provide further improvements.

Additional evidence is discussed in W3 below.

See revised Section 3 and Table 1.

W2. We thank the reviewer for this important question. While DoE provides statistical evidence such as main effects, interactions, and candidate rankings, translating these findings into promising and feasible search subregions remains a challenging task. Our key contribution is to enable LLMs to jointly interpret statistical evidence, variable descriptions, and constraints, transforming observed objective behavior into actionable guidance for subsequent BO.

To directly examine the contribution of LLM interpretation, we introduce a statistical-only region-selection baseline using the same DoE evidence without LLM reasoning. As shown in the revised Table 1, this baseline achieves an average normalized distance of 0.2683 and objective regret of 40.99, compared with 0.1308 and 19.91 for our original heterogeneous ensemble. The stronger heterogeneous ensemble further improves these metrics to 0.1107 and 13.17.

We also extend our sequential optimization ablation in Figure 5. The full method achieves the lowest final regret across all four benchmarks compared with variants without subregion refinement or without in-loop LLM guidance. Additionally, the FCNet ablation in Figure 6(b) supports the contribution of LLM guidance during sequential optimization.

Together, these findings suggest that LLM interpretation provides additional value beyond the tested statistical-only heuristic, although they do not establish superiority over all possible statistical approaches.

See revised Section 3, Table 1, Figures 5 and 6(b).

W3. We thank the reviewer for this important question. While DoE, LLM reasoning, and BO are individually established techniques with distinct methodological foundations, our key innovation lies not in their simple combination, but in their principled integration to enable statistically grounded LLM reasoning for black-box optimization. Rather than relying primarily on pretrained knowledge or generic heuristics, our framework enables LLMs to interpret statistical evidence from observed objective behavior and identify promising search subregions in a data-driven manner. This addresses an important limitation of conventional LLM-guided optimization: pretrained knowledge may be incomplete, biased, or poorly aligned with the unknown objective landscape. Our approach therefore shifts LLMs from knowledge-driven heuristic advisors toward evidence-grounded optimization agents, enabling more informed and adaptive search-space refinement.

To further substantiate this contribution, we have expanded our component ablations in Section 3. Table 1 and Figure 2 examine the effects of DoE evidence, model strength, and collective aggregation on initialization quality, including a statistical-only baseline. Figure 5 evaluates the contributions of subregion refinement and sequential search components, showing that our full method achieves the lowest final regret across all four benchmarks. Figure 6(b) further examines the effects of DoE evidence and LLM guidance on the FCNet benchmark. Together, these results provide empirical support for the value of statistically grounded collective reasoning and its integration with subsequent BO.

See revised Section 3, Table 1, Figures 2, 5, and 6(b).

Dear Reviewer 5LFU,

As the discussion phase comes to an end, please acknowledge the author's response and consider leaving them a comment on whether it has resolved some or all of your concerns.

Thanks!

Thank you for your responses and the effort put into revising the manuscript. The comparisons against stronger individual LLMs and homogeneous ensembles addressed one of my biggest concerns.

However, I remain unconvinced that the proposed framework demonstrates sufficient advantages over more conventional BO approaches to justify its additional complexity. I also remain concerned about the overall conceptual novelty of this work.

While I acknowledge the improvements made in the revised submission, my overall assessment remains unchanged, and I will maintain my original score.

Reviewer_5UqEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper aims to use DOE and aggregated LLM suggestions to guide the sequential sampling in BO. The paper is validated on a range on common BO test functions (Branin, ackley, etc.. ), an HPO task and an application to software configuration.

Critical strengths
  1. The paper tackles heterogeneous decision variables using DOE which is common in black-box optimization tasks.
  2. The paper acknowledges existing work in the area
  3. The ablation studies done are useful in understanding the impact of DOE and collective LLM
Critical weaknesses
  1. Figure 1 is very hard to understand. It shows many steps taken by the method without any explanation. It also has some math that is undefined. It is suggested to simplify the image and only focus on the overarching goal in the introduction.
  2. The overall paper is very hard to follow especially in notation. For example, Sec. 2 starts without defining \mathcal{L} or type_j and desc_j. Similarly, D_summary in 2.3 is not defined (here a note that its definition can be found in the appendix is helpful). Along that line, Sec 2.3 and 2.4 are also very hard to follow as they simply state all design considerations and the modeling approach without the necessary explanation or intuition. The same holds for Theorems 1 where the paper does not provide any discussion about the probability bound obtained. These all make it very hard to appreciate the modeling choices and the reasoning behind them.
  3. Is there any guidance to help select \rho_{DOE}. Can the BO process inform how much we should focus on DOE comparted to Path B ?
  4. L_{i,s} in Sec. 2.4 is not defined, also can you please specify what normalization is being done on the numerical coordinates. Is this a min-max normalization ?
  5. The paper highlights that they aim to hedge against unreliable LLM warm starts but then uses an approach where the LLM influence is decayed with time, meaning they trust the LLM more at the start (for a warm start). Can the authors please shed a bit more light on this ? also how were the parameters chosen ?
  6. For the collective LLM reasoning, it is not clear if the need for ensembles is due to using weaker models. Would the results in Table 1 also hold for a stronger LLM (ex: GPT 5.6 or Opus 5.5). Also, what is Average distance here ? is it average regret ? Also is the regret reported at T ? or the distance between the optimal and the best selected design ?
  7. In Fig 3 the behavior of LLMABO is rather surprising. It is suggest to test a simpler benchmark where all context and data are simply given to strong LLM that then choses the next design to observe. This can help reveal the advantage of DOE and collective reasoning compared to using the LLM directly.
  8. The experiments shown are based on a continuous domain. However the method is designed for heterogeneous decision variables. It is not shown how the method will perform in these cases.
  9. In Fig 3, your approach is not very efficient at the start but catches up later ? What may be causing this trend ?
  10. The warm start GP is not defined.
  11. An interesting benchmark to compare against is one that uses the DOE budget at the beginning and then only uses Direct-Search. This can help better highlight the importance of the Collective LLM Reasoning and the need for DOE.
  12. Most importantly, the authors are strongly encouraged to revise the paper's writing and add more detailed explanations regarding the math and design choices. Here it may help to move some of the algorithms to the appendix to provide more space for explanations in the main paper.
Questions for authors

Questions are included in the comments above.

5 comments
AuthorsOpenReview ↗

We are grateful for your valuable comments and constructive feedback. Below, we provide our responses

Response to Reviewer 5UqE

We thank the reviewer for the valuable feedback. We address each concern below.

W1. We have simplified Figure 1 to help readers better understand the central idea of our framework. See updated Figure 1 and Section 2.1.

W2. We improved clarity by standardizing notation and simplifying pseudocode. The intuition behind Theorem 1 is that when most proposals are close to a reference region, the medoid also stays close, reducing the influence of unreliable proposals. Independence is needed only for the probability bound. We clarify that this does not guarantee BO regret. See Sections 2.3–2.5 and Appendices A.1 and A.8.

W3. We selected ρDoE=0.3\rho_{\mathrm{DoE}}=0.3 based on sensitivity tests under a fixed budget. On several Bo tasks, the performance generally worsens beyond ρDoE=0.4\rho_{\mathrm{DoE}}=0.4, suggesting that more DoE limits sequential optimization. Our allocation is predetermined; adapting it using BO feedback remains a promising extension. See updated Appendix Table 5.

W4. We define Li,sL_{i,s} as the scalar trust-region size around anchor xix_i at iteration ss. We also clarify that numerical coordinates use min-max normalization based on the original variable bounds, mapping them to [0,1][0,1]. See revised Section 2.4 and the Notation Appendix.

W5. Our early LLM guidance uses measured DoE evidence rather than an ungrounded warm start. Collective reasoning reduces reliance on individual models, while GP search gains importance as observations grow. LLINBO (Chang et al., 2025) and AuxBO (Gupta et al., 2026) provide related motivation, but our approach focuses on DoE-grounded collective guidance. The parameters ρ0=0.10\rho_0=0.10 and λ0=0.80\lambda_0=0.80 are user-adjustable empirical choices that control global exploration and the LLM/GP balance, having relatively small effects on performance. Our mathematical analysis in the appendix justifies γ>=4\gamma>=4 to be preferable for general tasks. See Section 2.4 and the theoretical appendix.

W6. We extend the ablation to three strong models (GPT-5.6, Claude Opus 5.5, and Gemini 3.1 Pro), each tested individually and with seven aggregated samples, plus a heterogeneous ensemble of seven strong models and a statistical-only baseline. All use matched DoE evidence. Aggregation improves every strong model, while the heterogeneous strong ensemble performs best (distance 0.1107; regret 13.17). Importantly, our original ensemble of weaker models (0.1308; 19.91) outperforms all three homogeneous strong ensembles, supporting the wisdom of crowds: combining different models can outperform repeated sampling from a stronger model. “Avg. Dist.” measures the mean normalized Euclidean distance to a global optimum; “Regret” measures initialization objective gaps, not final BO regret. See Section 3, Table 1.

W7. On Rastrigin, LLAMBO's repeated proposals near (0,0)(0,0) may explain its early advantage, as also seen in Figure 2(f) of the LLINBO paper. On Bukin, we acknowledge LLAMBO's stronger performance within 50 evaluations. We also evaluate a direct GPT-5.6 optimizer that selects each next point using the full context and observed results, without GP or aggregation. Figure 5 shows that our framework and handicaped versions of it all achieve lower regret than the proposed baseline across benchmarks, supporting the value of combining DoE-grounded LLM guidance with GP search. See Figure 5.

W8. We appreciate the reviewer's concern and would like to clarify that our experiments also include non-continuous settings. FCNet Naval contains categorical, integer-valued, and numerical parameters on finite grids, while x264 involves constrained discrete configurations. These experiments demonstrate the applicability of our framework beyond continuous functions. See the FCNet and x264 experiments in Sections 3 and 4.

W9. Our initial DoE is designed to understand the objective's landscape, not to achieve low regret immediately. These evaluations provide statistical evidence to identify promising subregions for subsequent optimization. This explains why our method improves more slowly at first but catches up and achieves lower regret as BO focuses on promising regions.

W10. The LLM warm-start GP baseline evaluates LLM-proposed initial points first, then runs standard GP-based BO.

W11. We added further ablations in Figure 5, including a variant that uses the same DoE evaluation points but does not use the resulting statistical evidence for region refinement. Our full method achieves lower final regret across all four benchmarks, suggesting that the benefit comes not only from DoE initialization but also from using its evidence to guide subsequent optimization. See Figure 5.

W12. We have simplified the main figure and pseudocode, standardized notation, expanded the methodological explanations, and moved secondary details to the appendix. Thank you again for these helpful suggestions.

Dear Reviewer 5UqE,

As the discussion phase comes to an end, please acknowledge the author's response and consider leaving them a comment on whether it has resolved some or all of your concerns.

Thanks!

Dear Authors,

Thank you for your efforts in addressing the concerns. This is very helpful.

I do believe that the idea proposed has merit, yet the paper will greatly benefit from revisions to the writing and additional experiments especially in non-continuous and heterogeneous settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.