acceptodds
Under review as a conference paper at ICLR 2027

Adaptive KappaSharp: Condition-Number Shaping for Preferential Bayesian Optimization

11 views

Abstract

Preferential Bayesian optimization (PBO) optimizes objectives accessible only through pairwise user comparisons. The standard approach fits a Gaussian process surrogate for observed pairwise comparisons (PairwiseGP) using the Laplace approximation and selects queries with the Expected Utility of Best Option (EUBO) acquisition function. EUBO queries new candidates at each step, producing pairs that share no candidates with previous queries. Each such pair forms an isolated component in the comparison graph, removing one degree of freedom from the likelihood Hessian and making it rank-deficient. This deficiency is structural and cannot be resolved by changing the surrogate modeling approach. Existing approaches to remedy this issue either waste query budget by forcing comparisons to stay connected, or apply uniform regularization that also perturbs directions already well-constrained by the observed comparisons. We propose KappaSharp that enables a diagonal correction to the Hessian to reduce its condition number, with larger corrections where the prior uncertainty is higher. The correction is only applied in the model fitting step, not query selection. An adaptive variant of KappaSharp is also presented that activates the correction only when the surrogate is confident about recent comparisons, avoiding unnecessary corrections when the problem is well-conditioned. On 11 benchmarks (5–20 dimensions), including a 16-dimensional controller tuning problem in plasma medicine, Adaptive KappaSharp outperforms the standard PBO baseline, with up to ().

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Preferential Bayesian Optimization, Gaussian Processes, Condition Number, Laplace Approximation, Human-in-the-Loop Optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_v87rOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies preferential Bayesian optimization (PBO) with the standard BoTorch setup. That setup fits a PairwiseGP using the Laplace approximation and picks queries with EUBO. The authors point out that EUBO usually picks two brand-new points for each query. As a result, the comparison graph splits into many disconnected pieces. They show that the likelihood Hessian is a weighted graph Laplacian whose rank drops by one for every disconnected piece. They argue that this makes the full Hessian ill-conditioned. Their fix, KappaSharp, adds a diagonal penalty to the MAP problem. The penalty is larger at points where the prior precision is low. Its overall strength is chosen so that the condition number of the Hessian drops to a target value. The penalty changes only the MAP estimate, not the covariance that the acquisition function uses. An adaptive variant turns the correction on only when the model has recently been confident about its comparisons. The paper supports the method with several propositions about the graph structure, the correction and its effect on EUBO. The experiments cover 11 benchmarks with 5 to 20 dimensions, including a plasma controller tuning problem. On the main initialization, the adaptive variant wins on 5 benchmarks, ties on 6 and loses on none against the standard setup.

Critical strengths
  • The paper makes a clear and useful observation. It links EUBO's habit of picking fresh pairs to the Laplacian structure of the likelihood Hessian, and Figure 1 explains this link well. I think people who work with PairwiseGP will find it helpful. The fix is also cheap and easy to add to existing code. It only touches the MAP step and adds little runtime.
  • Few BO papers use 60 paired seeds with common random numbers, five initialization schemes and 16,500 runs in total. The ablations in Table 2 are well designed. They separate the effect of moving the MAP from the effect of changing the Newton step or the covariance. The authors also report the losses of the static variant openly.
Critical weaknesses
  • The paper's explanation of why the method works doesn't match its own evidence. The paper says a rank-deficient likelihood Hessian makes H=K−1+HlikH = K^{-1} + H_{\text{lik}} ill-conditioned, and that KappaSharp fixes this. But K−1K^{-1} is full rank, so the missing likelihood directions don't make HH ill-conditioned on their own. In those directions, the posterior just falls back to the prior. The paper itself says this is the correct answer (Proposition 2 and Theorem 7).
  • The ablations also point the same way. If conditioning were the problem, the MetricOnly variant should help, since it uses the corrected Hessian only to improve the Newton steps. Instead, it gets 1 win and 7 ties. The gains appear only when the MAP itself moves. So KappaSharp is really a change to the model rather than a numerical fix. It fits the mean under a stronger prior that pulls isolated points harder toward zero. Then it pairs that mean with the covariance of the original model. The result is not the posterior of any single model. The paper doesn't explain why this mix should lead to better queries. It also doesn't explain why the method hurts on Ackley-8D and never shows that κ(H)\kappa(H) is actually large in these runs either.
  • The theory also does not address this gap. Propositions 4 and 5 are basic calculus. while the bound in Proposition 6 can never exceed ∣f^0∣|\hat f_0|. Theorem 9 compares MAP-only with full augmentation, not with the baseline. Its bound also grows as λmin⁡(Hη)\lambda_{\min}(H_\eta) shrinks, which is exactly the ill-conditioned case the paper cares about.
  • The experiments don't yet show the method beats reasonable alternatives. Almost all baselines are variants of KappaSharp itself. The connected-querying baseline fixes one candidate to the previous query point and loses badly. That makes me wonder whether it was set up fairly. The natural version fixes the previous winner, and BoTorch's analytic EUBO has a previous_winner option for this. The EP and Gibbs baselines use EI, which the authors admit is a poor fit.
  • Simple alternatives that adjust the same knob are also missing. Examples are tuning the GP output scale or its prior, or running the adaptive rule with a uniform diagonal penalty. The uniform diagonal penalty already gets 3 wins and 2 losses, including +30.7% on DTLZ2-16D. So it isn't clear that the prior weighting is needed.
  • The scoring makes the results hard to interpret. The metric is the percentage change in raw final utility, which depends on where the utility's zero point happens to sit. On Hartmann-6D, the known optimum is 3.32. Measured as regret against that optimum, the adaptive variant's reported −0.9% is about 7% worse than the baseline, and the static variant's −4.8% is about 39% worse. The win, loss and tie counts come from 55 uncorrected t-tests, so a few of the 18 wins could be chance.
  • The paper doesn't say how the adaptive rule's four constants were chosen, and the sensitivity checks reuse the reported benchmarks. Finally, the abstract says "up to +10.9%" while Table 1 shows +24.0% on Levy-10D. The conclusion also calls the gains consistent, even though 6 of 11 benchmarks show no significant change.
  • Please make the abstract and the conclusion match Table 1. The DTLZ2 problems are reduced to a single utility, so calling them multi-objective is confusing. DTLZ2 also comes from Deb et al. (2002), not González et al. (2017). Appendix G says the plasma utility is judged by human inspection, but the experiments use a simulated oracle, so please state clearly which one you used. Since every run uses σ = 0.1, it would help to test other levels of comparison noise as well.

Deb et al., "Scalable multi-objective optimization test problems", CEC, 2002.

Questions for authors
  • Can you plot κ(H)\kappa(H) and the number of connected components over a run for the baseline? Can you also show that the shifted MAP gives a better surrogate, for example by ranking held-out points better, measured by Kendall's tau against the true utility?
  • Is there a model whose posterior matches the MAP-only mix of mean and covariance? Did you run full augmentation, with R in both the MAP and the covariance, in the experiments? If so, how did it do?
  • In connected querying, which point stays fixed? Is it the previous winner or one of the two previous points? How does the method compare when the previous winner is fixed?
  • How does the adaptive rule perform with a uniform diagonal penalty? How does the baseline perform with a tuned or larger output-scale prior?
  • Do your conclusions still hold if you report normalized regret against the known optima and correct for multiple testing?
  • How did you choose τ\tau, α\alpha, the EMA rate and the minimum step? Did you test these values on any benchmarks you didn't also report?
  • Why did you use BoTorch 0.6.0 and PyTorch 1.11? PairwiseGP has changed quite a bit since 2022, so I'd like to know whether the results hold on a current version.
8 comments
AuthorsOpenReview ↗

Response to Reviewer v87r (1/2)

Thank you for a detailed review. We agree with the central point of the comments, and addressing these comments led to the revision described in the general response. Results are for the revised method, Precision-Weighted Refit (PWR), on BoTorch 0.18.1 with 200 queries and 60 paired test seeds, unless stated otherwise.

Explanation vs. evidence, and κ\kappa (Q1, first part). The revision replaces the conditioning explanation with the lengthscale collapse of Sec. 4.2. Fig. 2 plots, for baseline runs of four benchmarks (test seeds 0 to 4), the number of connected components, κ(H)\kappa(H), the largest and smallest eigenvalues of HH, and the smallest distance between candidates. The median κ(H)\kappa(H) reaches 101110^{11} to 101610^{16}, but through λmax⁡(K−1)\lambda_{\max}(K^{-1}); λmin⁡(H)\lambda_{\min}(H) falls by less than two orders of magnitude and stays bounded by the prior, as you argued. The revision uses Proposition 2 to show that, under EUBO and given the within-component differences, the Gaussian process prior decides the relation between components through its lengthscales, the shortest of which collapse (Sec. 4.2, Table 1).

MetricOnly: a change to the model. We agree. The revision describes PWR as a change to how PairwiseGP is fitted, with the effect running through the lengthscale refit (Sec. 5.2), and drops MetricOnly. Without the refit the gain disappears on DTLZ2-16D, DTLZ2-20D and Ackley-8D (Table 3, relative to PWR: +7.8, +7.7 and +11.7 points, all significant after Holm correction).

Not the Laplace posterior of a single model (Q2). Sec. 5.3 states this, and we know of no model whose posterior is the MAP-only mix. Full augmentation, with RR also in the precision, is the Laplace posterior of the probit model with prior precision K−1+RK^{-1} + R, up to the clipping in BoTorch. We ran it on all 13 benchmarks (App. B.8, p. 18): minus PWR, it is significant after Holm correction only on CarCab-7D (-1.8 points, full augmentation better); on ZDT3-10D it is -11.1 (not significant after correction), and on the other eleven benchmarks the two are within 1.8 points. We chose MAP-only on development seeds by a rule fixed in advance (App. C, p. 19).

How the mix changes the queries. Sec. 6 tests this with a factorial experiment on baseline states (eight benchmarks, test seeds 0 to 19; Table 7, p. 22). The refitted hyperparameters make EUBO choose points closer to the observed candidates and to each other on every benchmark. RR in the MAP estimate does the opposite. One reason: EUBO is positively homogeneous of degree one in the posterior means and standard deviations, so shrinking all means by a common factor gives the same maximizer as inflating all standard deviations by its inverse; we expect the non-uniform shrinkage by RR to act similarly. With both of its changes, PWR chooses a tighter pair than the baseline on Plasma-16D and DTLZ2-16D and points farther from the observed candidates on ZDT3-10D, Levy-10D, Ackley-8D and Hartmann-6D. RR in the precision has smaller effects.

A better surrogate? (Q1, second part). Not consistently. On the data of baseline runs (test seeds 0 to 9), Kendall's rank correlation with the true utility changes by -0.15 to +0.07 globally and is lower near the best points on four low-dimensional problems (uncorrected Wilcoxon p<0.05p < 0.05; Table 9, p. 23). The gains come with changes in where EUBO queries, and a complete account of why these changes lower the regret is still open.

Ackley-8D. The mean now improves (20.4% to 10.9%) because the baseline ends above 40% remaining regret on 19 of the 60 seeds (PWR on 6), but PWR is worse on 60% of the seeds. On Ackley-8D and Hartmann-6D, PWR explores more and refines less (Tables 7 and 8). Hartmann-6D is the one significant loss after Holm correction (+1.6 points). Sec. 7.2 states both losses, and the limitations state the one on Hartmann-6D.

Theory. Submitted Propositions 4, 5 and 6 and Theorem 9 are now Propositions 5, 6 and 8 and Theorem 11 in App. B (pp. 16 and 17), as properties of the correction. Proposition 8 bounds the shift of the MAP estimate relative to ∥f^0∥Hˉ\|\hat f_0\|_{\bar H}, and the bound of Theorem 11 is of order η/λmin⁡(Hη)\eta/\lambda_{\min}(H_\eta), as you note, and we use neither as an argument for the method. App. B.8 states that its results do not show that MAP-only is better than the baseline or than full augmentation; the experiments decide this. The main-text theory is Propositions 1 and 2 and what they imply for the kernel.

Baselines beyond variants of the method. The revision adds lengthscale floors of 0.2 and 0.3, freezing the hyperparameters after 50 iterations, a random second point in every fifth query, two outputscale priors, previous-winner querying, expectation propagation (EP) and an exact posterior (Table 3; App. L, M, N). The floors and the freezing worsen ZDT3-10D by about 9 points, and the floors do not reproduce the gain on DTLZ2-16D (-0.1 and -0.5 points).

AuthorsOpenReview ↗

Response to Reviewer v87r (2/2)

Previous winner (Q3). The submitted connected querying followed Xu et al. (2024) and kept the new candidate of the previous query. The revision uses your version (App. M, p. 28): the winner of the last comparison is kept until it loses, and BoTorch's analytic EUBO chooses the other candidate given this winner (30 seeds, 6 benchmarks). At the same number of queries it is not significantly better than the baseline on any of the six, and PWR has a lower mean regret than it on all six. Counted in evaluated candidates it is competitive and significantly better than PWR on DTLZ2-20D and ZDT3-10D (uncorrected; Table 13, p. 29).

EP and Gibbs with expected improvement (EI). We replaced these with three EUBO variants with the same priors and seeds (App. N, p. 29): EP, and the exact posterior with the EP hyperparameters or with the baseline's own. With the baseline's hyperparameters, the exact posterior changes the regret significantly after Holm correction on no benchmark, and PWR has a lower regret than all three on DTLZ2-16D. The EP fits show the same collapse.

Outputscale and uniform correction (Q4). On development seeds, the outputscale prior SmoothedBox(0.25, 1) improves DTLZ2-16D by 3.6 points but worsens ZDT3-10D by 17.3, and SmoothedBox(4, 16) changes no benchmark significantly except Ackley-8D (-11.0, before correction; App. L, p. 28). A uniform R=cIR = cI with the same strength rule and refit is worse than PWR on DTLZ2-16D (+5.0) and DTLZ2-20D (+5.8), significantly after Holm correction, and better on ZDT3-10D (-10.7, significant only before correction; Table 3). With the same activation rule, uniform minus weighted is +3.7 on DTLZ2-16D (significant after Holm correction) and +4.3 on DTLZ2-20D, and no benchmark favors the uniform correction significantly (30 seeds; App. I, p. 26).

Scoring (Q5). We now report the normalized remaining regret 100(f∗−ft)/(f∗−f0)100(f^* - f_t)/(f^* - f_0), with mean curves (App. G, p. 23) and, at iteration 200, paired 95% intervals and Wilcoxon tests with Holm correction over the 13 benchmarks (Table 2); we removed the win/loss/tie counts. After Holm correction, DTLZ2-16D and DTLZ2-8D improve and Hartmann-6D, measured against the optimum as you suggested, is worse (+1.6 points, pHolm=0.013p_{\mathrm{Holm}} = 0.013). The mean regret is lower on all five problems with 14 to 20 dimensions (pooled Wilcoxon p<10−4p < 10^{-4}).

Constants and sensitivity (Q6). We chose α=0.02\alpha = 0.02, the fixed outputscale, MAP-only and the correction at every iteration on development seeds; the first corrected iteration, t=9t = 9, and the exponential moving average (EMA) weight of the gated version, 0.2, were not chosen on them. The 13 benchmarks (6 to 20 dimensions) include a robot-pushing simulator and a plasma-medicine problem. Every choice used only reported benchmarks, and five were not run on development seeds: DTLZ2-8D, RobotPush-14D, CarCab-7D and the new Styblinski-Tang-10D and Rosenbrock-10D (App. C, p. 19). On these five, PWR gains on DTLZ2-8D (significant after Holm correction) and loses on Styblinski-Tang-10D (significant only before correction); the other three are not significant. Vehicle-5D from the submission is shown only in App. G: 97% of the runs of both methods reach zero remaining regret within 20 iterations. App. K (p. 27) varies α\alpha from 0.01 to 0.1, the first corrected iteration and, for the gated version, the threshold τ\tau and the EMA weight, on test seeds 0 to 29, where the Plasma-16D gain is -7.7 (-3.3 over all 60). With another first corrected iteration (5, 15, 25) or EMA weight (0.1, 0.4), only one of 35 uncorrected tests differs significantly from the default. Every setting improves Plasma-16D and DTLZ2-16D significantly before correction, except one gated setting on each, and the losses on Styblinski-Tang-10D and Hartmann-6D grow with α\alpha. We set the gated threshold τ=0.121\tau = 0.121 without regret values, as the value at which the rule was on in about 40% of iterations in preliminary runs (App. I).

Abstract, conclusion, DTLZ2, Plasma and noise. The abstract and conclusion match Table 2, and Sec. 7.2 and the limitations state the losses. DTLZ2 is described as several outcomes that a piecewise-linear utility compares with a target, and is cited to Deb et al. (2002). The Plasma-16D comparisons are simulated from the utility, with no human in the loop (App. C). App. J (p. 26) tests lower and higher noise: with higher noise, PWR has a lower mean regret on ZDT3-10D and CarCab-7D (significant on CarCab-7D before correction), a similar one on DTLZ2-16D and no significant change on Plasma-16D.

Q7. BoTorch version. All revised results use BoTorch 0.18.1 and PyTorch 2.15.0 (App. C). We built the submission on the code of an earlier project of ours that used BoTorch 0.6.0 and PyTorch 1.11. The submitted results did not hold at 50 queries; item 3 of the general response explains why and why we now use 200.

We would be grateful for a reassessment of the revision.

Reviewer_v87rOpenReview ↗

Reply to Authors

Thank you for the revision. The authors replaced the conditioning explanation with lengthscale collapse, added the baselines I requested (previous-winner querying, output-scale priors, uniform penalty, lengthscale floors, EP, and exact posterior), switched to normalized remaining regret with Holm correction, and reported the losses. These changes address my main concerns.

However, two questions remain for me:

  1. The surrogate does not rank points more accurately (Kendall’s tau is mixed and often worse near the best points). The gains therefore come only from changes in the pairs that EUBO selects. The paper's factorial experiment shows that the refitted lengthscales pull the queries closer together while the shifted MAP pushes them farther apart, but the authors do not yet explain why these new pairs lower regret.
  2. MAP-only is not the posterior of any single model, and full augmentation performs similarly. It is therefore unclear whether the benefit comes from a deliberate bias in the mean or from an adjustment whose effect we do not fully understand.

If the authors can address the remaining questions above, I will consider increasing my score.

Reviewer_yJf3OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes KappaSharp, a method for improving the conditioning of the Hessian matrix in preferential Bayesian optimization (PBO) by controlling its condition number during maximum a posteriori (MAP) estimation under the Laplace approximation. The authors introduces a diagonal correction whose magnitude depends on the prior uncertainty, applying larger corrections to directions with weaker prior precision. The paper further introduces Adaptive KappaSharp, which determines whether to activate the correction based on PairwiseGP's confidence in recent comparisons.

The paper provides extensive theoretical analysis to characterize the rank deficiency and potential ill-conditioning of the Hessian in standard PBO, establish the properties of the proposed correction, and justify applying the correction only during MAP estimation rather than directly modifying the posterior covariance (full augmentation).

Critical strengths

The paper is well-motivated and addresses the ill-conditioning issue in PBO.

The paper provides extensive theoretical analysis to justify the proposed algorithm.

The paper additionally proposes Adaptive KappaSharp, and experimental results show that the adaptive version avoids unnecessarily applying the correction compared with standard KappaSharp.

Critical weaknesses

The proposed correction is based on the diagonal entries of the prior precision matrix, while the underlying rank deficiency originates from the disconnected comparison graph and the null space of the likelihood Hessian. It is unclear whether prior-based diagonal corrections effectively target the poorly conditioned directions of the full Hessian.

The experimental validation lacks an analysis of how the Hessian condition number evolves throughout the optimization process.

The adaptive activation criterion lacks strong theoretical justification and is primarily based on empirical observations of the surrogate's confidence.

Questions for authors

See Weakness.

4 comments
AuthorsOpenReview ↗

Response to Reviewer yJf3

Thank you for the positive assessment. Your W1 and W2 raised the same question as the other two reviews, and answering it led to a better-supported explanation of the method (general response). We answer for the revised method, Precision-Weighted Refit (PWR).

What stays and what changed. The graph analysis (Propositions 1 and 2), the precision-weighted correction with its condition-number calibration and the MAP-only posterior stay, and PWR has a lower mean regret than the baseline on all five problems with 14 to 20 dimensions (pooled Wilcoxon p<10−4p < 10^{-4}), with DTLZ2-16D significant on its own after Holm correction. Of the strengths you listed, the first rests on our claim that the correction fixes ill-conditioning, which the revision withdraws; the second and third hold in part (Theory and results; W3). The correction acts through the lengthscale refit, which counters the lengthscale collapse (Sec. 4.2) and now keeps the outputscale fixed.

W1. We agree that a correction based on the diagonal of K−1K^{-1} need not target the poorly conditioned directions of HH. HlikH_{\mathrm{lik}} has one zero eigenvalue per connected component in all 200 fits checked (Proposition 1, Fig. 2a), but the prior bounds λmin⁡(H)\lambda_{\min}(H), and the large κ(H)\kappa(H) comes from nearly coincident candidates (Sec. 4.1). The diagonal of K−1K^{-1} enters the lengthscale refit: since (K−1)ii(K^{-1})_{ii} is the inverse prior variance of f(xi)f(x_i) given all other candidates, one explanation (Sec. 5.2) is that short lengthscales make the candidates look isolated and raise the correction, so the refit favors longer lengthscales; on the same data, the refit lengthens the shortest lengthscale on all eight benchmarks tested (Table 5, p. 21).

W2. New Fig. 2 adds this analysis: over 200 iterations of the baseline on four benchmarks, it shows the number of connected components, κ(H)\kappa(H), the largest and smallest eigenvalues of HH, and the smallest distance between candidates; the median κ(H)\kappa(H) reaches 101110^{11} to 101610^{16}.

W3. We agree that the rule is empirical. PWR drops the rule, a choice we made on development seeds, and applies the correction at every iteration from t=9t = 9; the gated version, which keeps the rule, is in App. I (p. 26), and on the test seeds the two do not differ significantly on any benchmark. The gated version does avoid early corrections: with disjoint initial pairs it delays the correction and removes a loss of PWR on Plasma-16D. A condition-number trigger with the same average activation gives similar results.

Theory and results. App. B (pp. 16 and 17) establishes the properties of the correction, and App. B.8 (p. 18) states that its results do not show that MAP-only is better than the baseline or than full augmentation; the experiments decide this. We chose MAP-only over full augmentation on development seeds by a rule fixed in advance; on the test seeds the two differ significantly after Holm correction only on CarCab-7D, by 1.8 points in favor of full augmentation. The mean regret of PWR reaches the baseline's final mean on 11 of the 13 benchmarks, on eight of them with at least 25% fewer queries (App. G, p. 23); after Holm correction, DTLZ2-16D (10.7% to 5.8%) and DTLZ2-8D improve significantly and Hartmann-6D is worse (Table 2). Without the refit, or with a uniform correction, the variant is significantly worse than PWR on DTLZ2-16D and DTLZ2-20D after Holm correction; the uniform correction is better on ZDT3-10D, significant only before correction (Table 3).

We are glad to answer any further questions.

Dear Reviewer yJf3,

Thank you again for your review and for your positive assessment. We are glad to answer any further question.

Thank you for the detailed response, I will maintain my original positive rating.

Thank you for reading our response and for your positive assessment of the paper.

Reviewer_iF1MOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on preferential Bayesian optimization (BO) settings using pairwise comparisons (PairwiseGP) and Expected Utility of Best Option (EUBO) acquisition criterion, and introduces Adaptive KappaSharp, a framework that lies on the observation that querying new candidate pairs makes disconnected comparison graphs and a rank-deficient likelihood Hessian. To that end, KappaSharp modifies the MAP objective by adding a diagonal quadratic penalty, with larger weights where the GP prior precision is lower. The approximate posterior covariance is then obtained by inverting the original, unaugmented Hessian evaluated at the modified MAP estimate. Thus, the penalty shifts the posterior mean and can indirectly affect the covariance, but is not explicitly included in the posterior precision. An adaptive rule activates this correction based on the model’s confidence on recent comparisons. Numerical tests on 11 benchmarks, 5 initialization settings, and 60 seeds per setting demonstrate the effectiveness of the proposed KappaSharp method.

Critical strengths

S1. The proposed KappaSharp method that modifies surrogate fitting while retaining the existing acquisition criterion, seems interesting and intuitive for preferential BO settings.

S2. The core ideas and methodological concepts are clear and well presented.

S3. The numerical evaluation is broad and shows practical gains of the proposed KappaSharp.

Critical weaknesses

W1. In my view, the central motivation is not sufficiently established as likelihood rank deficiency does not necessarily imply poor posterior conditioning, since the GP prior can stabilize the Hessian. Proposition 2 also concerns conditional, not marginal, uncertainty. Further clarification and/or empirical validation is needed for these claims.

W2. The proposed activation criterion lacks validation as high confidence on recent comparisons does not necessarily indicate ill-conditioning. In my opinion, this connection is needed to strengthen the paper. Also, a comparison against direct condition-number-based activation could also make the paper stronger.

W3. The proposed weighting’s benefit in my mind remains unclear as the diagonal of K^{-1} does not generally identify poorly conditioned directions, and uniform regularization outperforms Static KS on some benchmarks in Table 2. Comparisons using the same adaptive trigger and matched conditioning targets would better corroborate the proposed weighting’s benefit.

Questions for authors

Please refer to the weaknesses above. In addition:

  • Does the proposed diagonal weighting offer an advantage over uniform regularization when both use the same adaptive trigger and condition-number target?
  • How robust is the decisiveness-based activation rule to changes in comparison noise and utility scale?
4 comments
AuthorsOpenReview ↗

Response to Reviewer iF1M

Thank you for the comments. We agree with W1, and W2 and W3 led to new experiments that are now in the revised manuscript; the general response summarizes the revision. Results are for the revised method, Precision-Weighted Refit (PWR), on BoTorch 0.18.1. Regret is the normalized remaining regret 100(f∗−ft)/(f∗−f0)100(f^* - f_t)/(f^* - f_0) in %, where ftf_t is the best true utility among the candidates queried up to iteration tt and f0f_0 the best in the initial design, with f∗f^* the optimum or, if unknown, the best value found (Sec. 7.1); lower is better, and differences are in points.

W1. Fig. 2 shows that the prior stabilizes HH: over 200 iterations, λmin⁡(H)\lambda_{\min}(H) falls by less than two orders of magnitude, and κ(H)\kappa(H) grows through λmax⁡(K−1)\lambda_{\max}(K^{-1}), because the acquisition places some candidates very close to earlier ones (Sec. 4.1). In the revision, the condition number only sets the strength η\eta of the correction. The revision uses Proposition 2(b) only as a statement about the conditional distribution: given the within-component differences, comparisons in one component inform another only through the prior covariance, so the lengthscales decide this link, and Table 1 shows that the shortest ones collapse on the problems with 14 to 20 dimensions. The proof now notes that the marginal uncertainty can be smaller than the prior uncertainty (App. B.2, p. 15).

W2. We agree that decisiveness on recent comparisons need not indicate ill-conditioning. PWR drops the activation rule, a choice we made on development seeds (App. C, p. 19), and applies the correction at every iteration from t=9t = 9; the gated version, which keeps the rule, is in App. I (p. 26). We ran the trigger you suggested, on the moving average of log⁡10κ(H)\log_{10}\kappa(H), with its threshold set on development seeds to the same average activation. Each minus the baseline on 30 test seeds (decisiveness / condition number, points): Plasma-16D -8.6 / -7.7, DTLZ2-16D -5.0 / -4.7, DTLZ2-20D -2.2 / -2.9, ZDT3-10D -12.3 / -12.3, Ackley-8D -5.6 / -2.9, Hartmann-6D +0.5 / +0.8. No paired difference between the triggers is significant, consistent with both acting mainly as a schedule.

W3 and Q1. We ran the matched comparison in two forms (uniform R=cIR = cI minus the weighted correction, points). With the same activation rule, condition-number target and refit (30 seeds, App. I): DTLZ2-16D +3.7 (significant after Holm correction) and DTLZ2-20D +4.3, and no benchmark favors the uniform correction significantly. With the schedule of PWR (60 seeds, Table 3): DTLZ2-16D +5.0 and DTLZ2-20D +5.8 (both significant after Holm correction), and ZDT3-10D -10.7 (uniform better, significant only before correction). We agree that the diagonal of K−1K^{-1} need not identify poorly conditioned directions; Sec. 5.1 presents the weighting as a heuristic choice, and the matched comparisons above support it on DTLZ2-16D and DTLZ2-20D. Since (K−1)ii(K^{-1})_{ii} is the inverse prior variance of f(xi)f(x_i) given all other candidates, the correction is largest at isolated candidates; one explanation (Sec. 5.2) is that, in the refit, it penalizes lengthscales under which the candidates look isolated. On the same data, the refit lengthens the shortest lengthscale on all eight benchmarks tested (Table 5, p. 21).

Q2. With a probit oracle, multiplying the utility by a constant is equivalent to dividing the noise by it, so App. J (p. 26) covers both: 10 times lower and higher noise on DTLZ2-16D, ZDT3-10D and CarCab-7D, 30 seeds. The activation rule responds to the noise: with higher noise it is on in 26%, 4% and 10% of iterations on DTLZ2-16D, ZDT3-10D and CarCab-7D, against 54%, 73% and 86% at the default noise. With higher noise (about 40% wrong comparisons), PWR changes the regret by +1.4, -9.8 and -9.8 points on these problems, and the gated version by +3.6, -0.9 and -3.5 (Table 11, p. 27); of these, only the change of PWR on CarCab-7D is significant before correction. With lower noise, PWR keeps an 11.4-point gain on ZDT3-10D. Under high noise the shortest lengthscale of the baseline collapses less (median 0.37 against 0.16 at iteration 200 on DTLZ2-16D), consistent with the smaller effect. At 10 and 100 times the default noise on Plasma-16D (at most 12% wrong comparisons), PWR changes the regret by -5.7 and +4.6 points and the gated version by -2.1 and +7.0, none significantly.

We would welcome a reassessment in light of these changes.

AuthorsOpenReview ↗

Follow-up on the revision

Dear Reviewer iF1M,

We would be grateful to hear whether the revision and our response address your points. In short:

W1. The revision no longer claims that the correction improves conditioning. Fig. 2 confirms your argument: the prior keeps the smallest eigenvalue of the Hessian bounded.

W2. PWR drops the activation rule. The condition-number trigger that you suggested gives results similar to the decisiveness rule, with no significant paired difference (App. I).

W3. With the same activation rule, condition-number target and refit, the weighted correction is better than the uniform one on DTLZ2-16D (significant after Holm correction) and DTLZ2-20D, and no benchmark favors the uniform one significantly (App. I). With the schedule of PWR, it is better on both after Holm correction, and the uniform one is better on ZDT3-10D before correction only (Table 3).

Q2. App. J tests 10 times lower and higher comparison noise, which also covers the scale of the utility.

We are glad to answer any remaining question.

I would like to thank the authors for their detailed response. Most of my concerns have been addressed. Although the weighting’s benefits and robustness remain setting-dependent, I appreciate the focused responses to my comments which I believe will strengthen the paper; thus I am increasing my rating.

Thank you for your careful reading of our responses and the revision, and for increasing your rating. We agree that the benefits of PWR are setting-dependent, and we will state this clearly in the limitations of the final version.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response: what the reviews led us to find

We thank the three reviewers. All three questioned the central premise of the submission, and Reviewer v87r asked whether the results hold on a current BoTorch release (Q7). Answering both required a major revision of the paper, but the form of the proposed correction is unchanged. Section, table, figure and page numbers refer to the revised manuscript.

What we found.

  1. The correction does not work by improving conditioning. On baseline runs (new Fig. 2), the likelihood Hessian HlikH_{\mathrm{lik}} has one zero eigenvalue per connected component of the comparison graph in all 200 fits checked (Proposition 1), and the median κ(H)\kappa(H) grows to between 101110^{11} and 101610^{16} by iteration 200. This growth comes from the largest eigenvalue: the Expected Utility of Best Option (EUBO) acquisition places some candidates about 10−310^{-3} lengthscales from earlier ones. The smallest eigenvalue stays bounded by the prior, as Reviewers iF1M and v87r argued.
  2. The lengthscales collapse. On the five problems with 14 to 20 dimensions, the median of the fitted per-dimension lengthscales stays near the prior mode (0.52), while the shortest falls to between 0.13 and 0.23 by iteration 200 (Table 1). Under EUBO, comparisons in one component inform another only through the prior covariance (Proposition 2), and a short lengthscale weakens this link. The correction acts through the lengthscale refit: on the same data, the refit lengthens the shortest lengthscale on all eight benchmarks tested, from between 0.15 and 0.22 to between 0.23 and 0.48 (Table 5, p. 21).
  3. Same correction, weaker and later collapse under the new default prior. The default lengthscale prior of PairwiseGP is Gamma(1.2, 0.5) in BoTorch 0.6.0, which the submission used, and the more concentrated Gamma(2.4, 2.7) in BoTorch 0.18.1 (App. C, p. 19). Under the old defaults the shortest lengthscales collapse within the 50 queries of the submission (on DTLZ2-16D, from 0.22 after 10 queries to 0.08 after 50), the regime in which the submitted method gained. Under the new defaults the submitted results did not hold: at 50 queries the baseline is stronger on most benchmarks and the collapse is weaker, and on most problems it still develops up to iteration 200 (Table 1), so the revision uses 200 queries.

The revised manuscript. We propose the title "Precision-Weighted Refitting against Lengthscale Collapse in Preferential Bayesian Optimization"; on the area chair's advice, the submitted title stays for now. We renamed KappaSharp to Precision-Weighted Refit (PWR). PWR keeps the correction ri=η2/(η+(K−1)ii)r_i = \eta^2/(\eta + (K^{-1})_{ii}), its condition-number calibration, and the posterior that uses the corrected maximum a posteriori (MAP) estimate with a precision that does not include the correction (MAP-only). The name reflects the lengthscale refit at the corrected MAP estimate, now with the outputscale fixed (Sec. 5.2, Algorithm 1). All results have been redone (BoTorch 0.18.1 defaults, 200 queries, 13 benchmarks, 60 paired test seeds).

Results. App. G (p. 23) shows the mean regret curves with 95% bands (Fig. 3, p. 24) and the paired differences (Fig. 4, p. 25). The mean regret of PWR reaches the baseline's final mean on 11 of the 13 benchmarks, on eight of them by iteration 149 or earlier, with at least 25% fewer queries. PWR takes up to 2.5 times the wall-clock time of the baseline over 200 iterations (App. E, p. 20), a cost that matters little when each query needs a person and physical experiments. It has a lower mean regret on all five problems with 14 to 20 dimensions (pooled Wilcoxon p<10−4p < 10^{-4}; Sec. 7.2). Per-benchmark Wilcoxon tests, with the multiple-testing correction that Reviewer v87r asked for (Holm; Table 2), find two gains, DTLZ2-16D (10.7% to 5.8%) and DTLZ2-8D (5.3% to 3.5%), and one loss, Hartmann-6D (4.2% to 5.7%). When we change one component at a time (Table 3), the gain on DTLZ2-16D and DTLZ2-20D disappears without the refit and with a uniform correction, and on Plasma-16D with a free outputscale in the refit, all significant after Holm correction; dropping RR from the final MAP estimate helps on Hartmann-6D (significant after Holm correction), and a uniform correction or full augmentation (the correction also in the posterior precision) helps on ZDT3-10D (significant only before correction).

Protocol. We made the design choices of PWR (α\alpha, the fixed outputscale, MAP-only and applying the correction at every iteration) on development seeds (60 to 119 for Plasma-16D, 60 to 89 otherwise) by rules written before their results were available, and did not use five of the 13 benchmarks for them (App. C, p. 19). Some variants look better than PWR on the test seeds, for example the gated version (the correction only when an activation rule is on; App. I, p. 26) and full augmentation on CarCab-7D (App. B.8, p. 18). We kept the configuration chosen on the development seeds and report the others.

AuthorsOpenReview ↗

Updated Revision

We have uploaded an updated revision. It contains two additions that came out of the discussion with Reviewer v87r, and no other change.

  1. App. F now ends with a new part, "Information per query" (pp. 23 to 24). It scores the queries of the baseline and of PWR by one criterion: the mutual information between the outcome of a comparison and the utilities, under the standard fit to the data available before the query (development-seed runs). In the second half of a run, a query of PWR carries about twice the information of a query of the baseline (0.094 against 0.048 bit). The new part also states the limits of this result: the information is higher on Levy-20D as well, where the regret changes by -0.2 points, and on Hartmann-6D, where PWR is significantly worse, so it explains the gains only in part.

  2. The limitations (Sec. 8) now state that the best strength of the correction is open.

Section, table and figure numbers are unchanged. The pages from App. G on have moved by one, so the page numbers that our earlier responses give for App. G to N, Figs. 3 and 4 and Tables 11 and 13 are now one page later.

We thank all reviewers again and would be glad to answer any further questions on the revision or on our responses.

AuthorsOpenReview ↗

Closing note from the authors

Dear Reviewers and Area Chair,

As the discussion closes, we thank you for the time you gave this paper. The reviews changed it for the better. All three reviews questioned the conditioning explanation, and the revision replaces it with the lengthscale collapse of Sec. 4.2. Fig. 2 shows how the condition number of the Hessian evolves during baseline runs, as Reviewer yJf3 asked, and App. I tests a condition-number trigger and a uniform correction, as Reviewer iF1M asked. Our last reply to Reviewer v87r relates the changed queries to the gain of PWR on Ackley-8D and its loss on Hartmann-6D, and decomposes the regret on Levy-20D. It also tests on new seeds an on-demand version that applies the correction only when the EUBO pair of the baseline's model, fitted to the same data, has a low prior correlation: this version showed no significant loss in 52 comparisons with the baseline (13 problems at four iterations) and about the same aggregate gain as PWR (0.17 for both), and its regret at iteration 200 exceeded that of PWR by 2 points or more only on DTLZ2-16D, Ackley-8D and ZDT3-10D (2.0, 5.6 and 8.3 points; App. I). The revision uploaded today adds these analyses to App. F and the on-demand version to App. I; the main text, PWR and Table 2 are unchanged, and so are all section, table and figure numbers (from App. G on, some appendix content moves one page later). App. F counts iterations from 1, as the rest of the paper does; there, "after iteration 50" and the ranges 8 to 50, 51 to 100 and 101 to 199 of our last reply read "after iteration 51" and 9 to 51, 52 to 101 and 102 to 200; the results (3.7%, 10.0%, 125 of 172 runs) are unchanged. App. F now states two limitations: the benefits of PWR are setting-dependent, and we cannot yet predict when the changed queries reach a better basin.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.