- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The article combines standard localised BO methods (trust regions, or local gradient moves) designed for high dimensional BO, with preferential Bayesian optimisation. An experimental validation is carried out, showing some strong performance. And an ablation study is carried out to determine what is actually important.
Critical strengths
The main subject of the paper is an obvious thing to do, and I don't think it's been done before
The article situates itself well in the literature
The derivation of the correct posterior distributions, derivatives and Hessians is what makes the article more than a plug and play of two concepts into a new article. This is correct so far as I can tell.
Critical weaknesses
The main subject of the paper is an obvious thing to do! It's just not very exciting. It's plugging together local methods with preferential BO, and a small number of necessary calculations to make it happen. While this is solid work, it doesn't feel sufficiently ground-breaking to be an ICLR paper. (I'm not one of those reviewers who insists everything must be revolutionary, but this one feels very much to be just plugging two things together.)
It is very difficult to gain much insight from the experimental results.
The case for cumulative regret is very poorly made.
Questions for authors
For the gradient based methods there seem to be very large requirements of the number of samples. Eg at l250 it is stated that GIBO carries out M experiments to evaluate the gradient at each iteration. Firstly, since we're doing local moves, can't we use previous observations as well to estimate the gradient? And if this is the case, do we need to take quite so many observations on each iteration?
Line 346, how does a line search work here? The referred appendix doesn't explain this.
Why are newly experimented points always (?) compared to the incumbent? I would have thought that more information could be gained at times by comparing two newly-sampled points.
Section 5.4. When we go global to local, wouldn't it make more sense to consider refining from any of the "good" points found by the global method, not just from the best one? Also, talk of finding a promising "region" is a little misleading I feel, unless I've missed something, because I think you refine a single point, not a region?
I could not parse Lines 398-400.
In line 402-403, I do not know what "GP sample-path problems" means.
Minor comments:
- The very first reference, to Garnett (2023), is a very recent reference for a very established method...
- Line 96. Starting with "Instead" doesn't make sense.
- Line 311, I think you've used the preference operator to mean negative semi-definite?
- Line 422, "smoother objectives from Sec 6" doesn't make much sense, since we're in Sec 6 already
4 comments
Answer to Reviewer jcbZ Questions
Questions
Q1: Sample requirements for gradient estimation.
Previous observations are indeed used, as the pairwise GP is fit on all comparisons collected so far. The gradient estimate at the current center therefore also draws on nearby earlier queries, and the GI acquisition in GIPBO accounts for the uncertainty reduction these already provide. The choice follows GIBO, and subsequent analysis identifies observations as optimal for the corresponding local gradient-estimation problem (Wu et al., 2023). We agree that this is a substantial part of the evaluation budget, and our results show that it does not always pay off. Gradient-based methods are most appropriate when accurate local refinement is valuable. A natural mitigation is to stop adaptively once the gradient estimate is sufficiently accurate, as studied for GIBO (He et al., 2025), which could be transferred to preferential feedback. We have added the cost trade-off to the practical recommendations in Section 6 and adaptive stopping as future work in Section 7.
Q2: Clarification of the line search.
The paragraph only referred to the ablations in Appendix C.8, but not to Appendix B.4, where the line search is described. We have reordered the paragraph to first explain the line search with a reference to Appendix B.4, renamed it to "Line Search", and made the description in Appendix B.4 more explicit. Given the search direction , we discretize step lengths along the normalized direction into 100 candidates. Following Thompson sampling, we draw one joint sample from the pairwise GP posterior over all candidates and select the one with the highest sampled utility. This selection uses the surrogate only. The selected point is then queried and compared to the current iterate , and it becomes the next iterate if it is preferred; otherwise, PrefSQP moves to the local sample with the highest posterior mean among those preferred over . While revising this part, we also corrected our PrefSQP implementation to match Algorithm 2 (see the overview of changes).
Q3: Choice of comparison points.
In all our methods, a new point is compared against an already evaluated reference point, which is the local incumbent in GIPBO, the current center in PrefSQP, and the global incumbent in TuRPBO. We agree that comparing two new points can be more informative per comparison. However, since we consider the number of experiments to be the main cost, comparing against an already evaluated point yields one comparison per new experiment, whereas comparing two new points requires two experiments per comparison. In addition, comparing against the reference point directly indicates whether the new point is an improvement, which our methods use to update the incumbent and, in TuRPBO, to adapt the trust region.
Q4: Multiple starting points for local refinement and terminology.
Starting local refinement from several promising points, similar to the multiple trust regions in TuRBO (Eriksson et al., 2019), is a possible extension. Under our budget it involves a trade-off, since the local phase only has about five gradient steps and splitting them across several starting points would leave few steps for each. Note also that the local phase reuses all comparisons from the global phase, so the information about other good points is retained in the surrogate. We have added this as future work in Section 7. Regarding the second point, the global phase indeed provides a single starting point, and "region" refers to its neighborhood, which the local phase then explores. We have clarified this wording in Section 5.4.
Q5: Unclear description of additional experiments.
Due to limited space, this sentence was too condensed, and it also mentioned the Sushi and CarCab benchmarks without introducing them. We have rewritten it as follows. "Among the global-to-local variants, we report G2L-PrefSQP in the main text. Results for G2L-GIPBO and G2L-GIPBOr as well as full experimental details are in Appendix C, with ablations in Appendix C.8. The appendix further includes the two low-dimensional PBO benchmarks Sushi (Kamishima, 2003), built from human preference data, and CarCab (Tanabe & Ishibuchi, 2020), a multi-objective design problem with a nonlinear utility (Appendix C.7)."
Q6: Meaning of GP sample-path problems.
By GP sample-path problems, we mean objective functions drawn as random sample paths from a GP prior with a squared-exponential kernel, with a new sample drawn for each run. This setup allows a within-model comparison (Hennig & Schuler, 2012), in which the surrogate uses the hyperparameters of the generating GP, as well as an out-of-model comparison with learned hyperparameters. We have clarified this in Section 6.
Answer to Reviewer jcbZ Minor Comments
C1: Reference to Garnett (2023).
We cite Garnett (2023) as a comprehensive and widely used textbook reference on Bayesian optimization. Following your suggestion, we have additionally added the seminal work of Jones et al. (1998) and the review by Shahriari et al. (2015).
C2: Incorrect use of "Instead".
We have removed "Instead".
C3: Ambiguous use of the preference operator.
We intended to use the matrix inequality to indicate that the Hessian is negative definite, rather than to express a preference relation. To avoid ambiguity with our preference notation, we have replaced this expression with the explicit statement that is negative definite.
C4: Unclear reference to Section 6.
This sentence was removed when we revised the paragraph.
Thank you for your careful response. I still don't think this is super exciting, due to the fact that you've plugged two things together in a very obvious way. But the improved clarity means I am happy to raise my score to a weak accept.
Answer to Reviewer jcbZ Weaknesses
Thank you for the time and effort invested in reviewing our paper. In the following, we address the stated weaknesses and answer the raised questions.
Weaknesses
W1: Limited novelty and contribution.
We agree that our methods build on existing local BO algorithms, but the calculations needed for the preferential setting are more than a small step. Deriving the joint predictive distribution of the latent utility, its gradient and its Hessian under the Laplace-approximated pairwise GP is what makes derivative-informed PBO possible, and to our knowledge GIPBO and PrefSQP are the first PBO methods to use this information. We believe this is useful because many applications where preference feedback is most natural, such as controller tuning or policy search, are high-dimensional, and our results show that local PBO can substantially improve cumulative performance there, which means fewer poor and potentially unsafe evaluations on the real system. In addition, our benchmarking of complementary local strategies against global PBO baselines across different problem settings gives an honest picture of when local PBO is effective and where it fails, which we believe is informative for both researchers and practitioners. Finally, the derivative framework opens the door to transferring further derivative-based local BO methods to the preferential setting.
W2: Difficult to interpret experimental results.
We agree that the number of methods and settings makes the results hard to digest. We have used part of the additional page to explain more clearly which method performs best in which problem setting and to derive recommendations for practitioners at the end of Section 6.
W3: Insufficient motivation for cumulative regret.
We agree that the motivation was too brief. Cumulative performance captures the quality of the full optimization trajectory, which matters in controller tuning and policy search, where every query is executed on the real system and poor intermediate evaluations can be costly or even crash the experiment (Menn et al., 2026). This is further amplified in typical PBO use cases such as exoskeleton gait tuning for user comfort (Tucker et al., 2020), where the user experiences and judges every queried gait. We have expanded this motivation in the introduction and now consistently use the term cumulative performance throughout the paper.