- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a framework for using LLM to select acquisition functions during Bayesian optimization. Instead of following the recommendation from LLM directly, the author proposes to ask the LLM to assign preference scores to each acquisition function and treats these scores as prior over their expected reward. Then those priors are updated using observed reward feedback. The paper models acquisition-function selection as a non-stationary multi-armed bandit problem. The author provides a cumulative regret analysis and evaluates the method on several benchmarks and a high-dimensional black-box optimization problem.
Critical strengths
The idea of using LLM to choose a proper acquisition function and treat the selection process as a multi-armed bandit problem is quite novel and interesting. The author also provides a theoretical analysis of the cumulative regret and conducts several numerical analyses to validate the algorithm.
Critical weaknesses
The paper already formulates AF selection as a multi-armed bandit problem, for which many adaptive selection algorithms already exist. Since the number of candidate AFs is also relatively small, it is not very clear how much additional benefit is provided by introducing the LLM prior, especially since the final AF selection is still made by the Gaussian-UCB bandit algorithm. Although the paper compares against several existing adaptive AF-selection methods and includes a random-prior baseline, the contribution of the LLM prior is still not fully isolated. Furthermore, the authors show that the numerical selector frequently overrides the LLM, and 61% of improvement events are associated with override decisions. That is interesting evidence that the calibration mechanism is useful, but it also raises a question: how much of LPG-UCB’s advantage actually requires an LLM?
Questions for authors
1, Since the number of candidate AFs is relatively small, why is an LLM needed to guide the selection rather than relying only on an existing adaptive bandit algorithm? Could the author further explain why the LLM-based method has superior performance based on the numerical results?
2, As the final AF selection is still determined by the Gaussian-UCB selector, how much of the performance improvement actually comes from the LLM prior?
3, Could the authors provide a direct ablation using the same Gaussian-UCB selector but without LLM guidance, for example, with a uniform prior shared by all AFs? This would help isolate the contribution of the LLM from that of the bandit-based adaptive selection itself.
5 comments
Thank you for your comments!
2. Contribution of LLM prior on performance improvement
We thank the reviewer for this valuable comment. We agree that a more direct evaluation of the contribution of LLM-generated priors would strengthen our study. In the original manuscript, we included comparisons with conventional non-LLM adaptive AF selection methods, including GP-Hedge, No-PASt-BO, and SETUP-BO, as reported in Table 1. Across the 31 synthetic benchmarks, LPG-UCB achieved a mean RAUC of 1.51, compared with 2.48, 2.49, and 2.53 for GP-Hedge, No-PASt-BO, and SETUP-BO, respectively. However, we recognize that these comparisons alone do not fully isolate the contribution of LLM-generated priors from that of the Gaussian-UCB selection mechanism.
To address this limitation and better understand the specific contribution of LLM-generated priors, we conducted additional ablation experiments following the reviewer's suggestion. First, we considered a uniform prior, assigning a fixed prior score of 0.5 to all AFs. This experiment was designed to isolate the contribution of the LLM-generated prior from that of the Gaussian-UCB selector itself, allowing us to evaluate whether LLM guidance provides additional benefits beyond reward-based adaptive AF selection. Second, we considered a linearly scheduled prior to investigate whether the LLM-generated prior provides more effective guidance than a simple heuristic-based strategy. Specifically, the prior scores of exploration-oriented AFs were linearly decreased from 1 to 0, while those of exploitation-oriented AFs were linearly increased from 0 to 1 as the optimization budget progressed. This comparison allows us to assess whether LLM-generated preferences offer benefits beyond a predefined exploration-to-exploitation schedule. We additionally included a random prior, assigning randomly generated prior scores to AFs, to examine whether the observed improvements could be attributed merely to arbitrary variations in prior initialization.
Across the benchmark functions listed in Table 5, LPG-UCB achieved a mean RAUC of 1.38, compared with 2.56, 2.57, and 2.32 for the uniform, random, and linearly scheduled priors, respectively. These results correspond to reductions in mean RAUC of approximately 46.1%, 46.3%, and 40.5%, respectively. These findings demonstrate that the LLM-generated prior provides a meaningful contribution to the performance of LPG-UCB beyond the Gaussian-UCB selection mechanism alone. Furthermore, the superior performance over the linearly scheduled prior suggests that the LLM-generated prior can provide more effective guidance for AF selection than a simple heuristic by leveraging the optimization context.
We have incorporated these additional ablation results and the corresponding discussion into Section 5.1.3 and Appendix G.2 of the revised manuscript, with the changes highlighted in red.
Follow-up on Rebuttal and Revised Manuscript
We would like to respectfully follow up on our rebuttal and revised manuscript. In response to your valuable comments and suggestions, we have further clarified the motivation for incorporating LLM guidance into adaptive AF selection and conducted additional ablation experiments using uniform, random, and linearly scheduled priors while keeping the Gaussian-UCB selector unchanged. These experiments were intended to directly address your central concern regarding the contribution of LLM-generated priors beyond the numerical selection mechanism. All corresponding revisions have been highlighted in red in the revised manuscript for your convenience.
We sincerely hope that these additional experiments and explanations have helped clarify the distinct contribution of LLM guidance and address your concerns. We would be grateful for any further feedback you might have and would greatly appreciate knowing whether our rebuttal and revisions have affected your overall evaluation of the paper.
Thank you very much for your careful review and valuable suggestions, which have helped us strengthen our work.
Thank you for your comments!
Thank you for your detailed and constructive feedback. We have addressed your specific comments below and incorporated the corresponding revisions into the manuscript.
1. Benefits of LLM Guidance over Conventional Adaptive Bandit Algorithms
We thank the reviewer for this important question. We agree that the relatively small number of candidate AFs raises a reasonable question about the need for LLM guidance. Our motivation for incorporating an LLM is to provide additional context-aware guidance for AF selection, particularly when the available optimization feedback is limited. Conventional adaptive bandit methods, such as GP-Hedge, primarily adapt their AF selection based on accumulated reward feedback. While effective, these methods do not explicitly incorporate additional optimization-state descriptors, such as GP length scales, distances between previously evaluated points, and recent improvement history, into their AF selection policy. In contrast, LLMs offer a flexible mechanism for jointly interpreting these contextual features and translating them into informed prior preferences over candidate AFs. This capability is particularly valuable during the early stages of BO, when limited observations are available, as LLM-generated preferences can provide a knowledge-based warm start for AF selection before sufficient numerical feedback has accumulated. Furthermore, scientific optimization problems may provide additional domain-specific knowledge, such as physical constraints or prior experimental insights, which can also be incorporated into LLM-based guidance.
Importantly, LPG-UCB does not rely solely on LLM recommendations. Instead, it uses LLM-generated preferences as prior information within a Gaussian-UCB framework, allowing the initial guidance to be continuously updated based on optimization feedback. This integration combines context-aware LLM guidance with feedback-driven strategy selection, rather than replacing conventional bandit learning. Moreover, the proposed framework is not inherently restricted to AF selection and can be extended to the selection of other BO strategies, such as global and local search regimes.
To further clarify the benefits of using an LLM prior over conventional adaptive bandit algorithms, we have expanded the discussion in Section 7 to emphasize the warm-start capability of LLM-generated preferences, their integration with numerical feedback, and the potential extension to other BO strategy-selection problems. The corresponding revisions are highlighted in red in the manuscript.