- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes HAPBO (Harnessed - Bayesian Optimization), a framework for incorporating expert prior knowledge into Bayesian optimization (BO). Unlike conventional approaches that typically encode prior knowledge as probability distributions or preference information, HAPBO starts from natural-language descriptions of expert knowledge. It uses a large language model (LLM) to convert such knowledge into a structured Prior Specification Language (PSL), which is then deterministically compiled into a numerical belief function. The resulting prior information is incorporated into the acquisition function through a bounded multiplicative factor, in a manner related to BO.
Critical strengths
- The proposed framework is clearly motivated and well presented. The overall methodology is coherent, and the paper provides a structured approach to translating natural-language expert knowledge into a form that can be used by BO. The topic is meaningful, and the proposed framework offers an interesting and potentially useful contribution.
- The experimental evaluation is extensive, covering multiple benchmark tasks and real-world applications, with additional ablation studies and robustness analyses. The paper also provides theoretical analysis to support the proposed approach, making the overall study comprehensive.
Critical weaknesses
I did not identify any major flaws in the overall framework. However, several aspects of the experimental evaluation and methodological design require further clarification. My main concerns are detailed in the questions below.
Questions for authors
-
Motivation for the bounded multiplicative factor. The proposed method selects the next query according to $ x_t \in \arg\max_{x\in\mathcal{X}} A_t(x)M(x), \quad M(x)=(1-\alpha)+\alpha\pi(x). $ In contrast, BO directly uses . What is the motivation for introducing the parameter and the additive term ? What advantages does this bounded formulation offer over the original BO formulation, particularly in terms of robustness to inaccurate or misleading prior information? Could the authors provide further empirical or theoretical evidence to clarify the benefits of this design choice?
-
Unexpected performance of random search and the missing BO baseline. In Figure 2, random search appears to outperform several BO-based baselines, ranking approximately third among the compared methods. This result seems somewhat unexpected, given the potential sample-efficiency advantages of BO. Could the authors explain why random search performs so well in this setting? In addition, why is the original BO method not included as a baseline in Figure 2? Since BO is directly related to the proposed approach, a comparison would help clarify the contribution of HAPBO beyond the use of prior information itself.
2 comments
Author responses are satisfactory and no further concerns remain. The revised manuscript is clearly improved; I recommend acceptance.
Thank you for the authors’ responses to my questions and for the corrections made to the related issues. I have no further concerns at this point. I have reviewed the newly uploaded PDF, and the current version appears much improved over the previous one; the constructive points raised by the other reviewers also seem to have been addressed accordingly. Best wishes.
We very much appreciate reviewer Kg2C's engagement with the methodological and experimental aspects of HapBO, and we thank them for the opportunity to clarify our contributions and further strengthen the paper.
To address the questions by reviewer Kg2C:
Bounded shaping and comparison with πBO.
(a) Motivation for bounded shaping. The additive term (1−α) ensures that the prior cannot eliminate candidates with positive acquisition. Thus, unlike πBO, α-πBO bounds the influence of the prior, and α controls the strength of that influence. Both methods have one scalar parameter (α versus β). (Guay-Hottin et al., Knowledge-Based Systems, 2025)[DOI 10.1016/j.knosys.2025.113039] demonstrated substantially greater robustness of α-πBO to misleading priors across benchmarks. We have highlighted this evidence in Section 3.3 and expanded Fig. 8, where the advantage of bounded shaping is particularly apparent under adversarial priors (HPO adversarial) and medium quality (synthetic ρ=0.5).
(b) πBO as an ablation rather than a baseline. HapBO is deliberately modular and πBO and α-πBO are interchangeable prior-injection components, both compatible with our PSL elicitation and deterministic compilation pipeline. Neither, by itself, provides the language-to-prior machinery that constitutes HapBO's central contribution. Therefore, πBO is not an independent end-to-end LLM-guided BO baseline, but an alternative module within our framework. Fig. 8 provides the more informative controlled comparison. This isolates precisely the contribution of choosing α-πBO over πBO, without confounding it with differences in prior construction.
Unexpected performance of random search. We thank the reviewer for catching this anomaly, which prompted us to identify and correct an implementation bug. We refactored the affected code and reran the experiments. The revised Figs. 2 and 5 now show the expected relative performance of random search and BO-based methods on HPO. We found that the correction did not alter our main findings, as HapBO's performance advantage remains, and the qualitative conclusions of the figures throughout the paper are unchanged. We are particularly grateful for this observation, which helped us improve the reliability of our work.
In response to this and all reviewers, we expanded our experimental evaluation from 15 to 21 figures, added 4 additional SOTA baselines, and substantially broadened our ablations. We sincerely thank reviewer Kg2C and all reviewers for their constructive feedback.