acceptodds
Under review as a conference paper at ICLR 2027

Adapting Nonstationary Multi-output Gaussian Processes to Bayesian Optimization

Abstract

Multi-objective Bayesian optimization (MOBO) commonly relies on in dependent Gaussian processes with stationary kernels, limiting its ability to represent nonstationary structure and share information between objectives. However, expressive regression models do not necessarily make reliable BO decisions. We study this mismatch for the multi-output low-rank nonstationary (MO-LRN) Gaussian process: strong training fit can coexist with large off-design errors and optimistic acquisition predictions. We introduce MOLRN-BO, which combines a regularized shared-spectral surrogate with objective-specific residuals, prequential mean correction and tempered covariance scaling, and Pareto-local qLogEHVI optimization with periodic global search. Experiments on 12 deterministic bi-objective benchmarks show that MOLRN-BO substantially improves upon the original MO-LRN and achieves the best average problem ranks for final normalized hypervolume and normalized inverted generational distance among nine evaluated algorithms. It also achieves the strongest adverse-tail performance while remaining competitive with the leading baselines in anytime optimization. Ablation studies further show that the shared spectral construction improves off-design prediction, the local–global decision policy improves optimization performance, and hierarchical calibration reduces systematic candidate bias. These results demonstrate that nonstationary multi-output surrogates can provide strong and robust MOBO performance, but realizing this potential requires their structure, calibration, and acquisition optimization to be explicitly adapted to sequential decision making, rather than judged by surrogate accuracy alone.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Bayesian optimization, multi-objective optimization, Gaussian process, non-stationary kernel

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_xRiJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies why a flexible nonstationary multi-output GP can fit observed data well while making unreliable acquisition decisions. MOLRN-BO combines a regularized shared-spectral covariance with objective-specific RBF residuals, prequential mean correction, tempered covariance inflation, and Pareto-local qLogEHVI search with periodic global rounds. On 12 deterministic bi-objective benchmarks with 20 seeds and 110 total evaluations, it substantially improves over the original MO-LRN and obtains the best average final nHV/nIGD ranks among the evaluated methods. The strongest standard comparator, qLogNParEGO, remains statistically competitive and has better anytime ranks. Ablations distinguish improvements in off-design prediction, the combined search/bias policy, and the prediction-versus-optimization trade-off introduced by hierarchical pooling.

Critical strengths
  1. The diagnosis of the regression-decision mismatch is useful and supported by decision-relevant measurements. Figure 1 and Table 2a examine off-design errors, prequential uncertainty failures, and HVI optimism rather than treating training RMSE as a sufficient surrogate-selection criterion. The strictly historical candidate residuals also avoid using future evaluations in the mean correction. This is consequential practical guidance for expressive BO surrogates.
  2. The evaluation includes paired initial designs, 20 seeds, final and anytime metrics, adverse-tail analysis, and multiplicity-adjusted tests. The improvement over original MO-LRN is convincing, and the paper appropriately acknowledges that superiority to qLogNParEGO is not statistically established. It also reports the negative optimization result for hierarchical pooling: bias decreases while final Pareto quality slightly worsens. This makes the empirical account more informative than a collection of favorable aggregate ranks.
Critical weaknesses
  1. The experiments do not sufficiently isolate the incremental value of the proposed nonstationary surrogate within the complete decision pipeline. Standard stationary baselines do not receive the same local-global search and calibration machinery. Table 2a is a fixed-design predictive diagnostic without RBF residuals, while Table 2b changes localization and mode-specific bias estimation together. Consequently, the largest sequential gain could primarily reflect the acquisition-search policy rather than the shared spectral model. A stationary surrogate under the identical decision policy is an especially consequential missing control, given the close comparison with qLogNParEGO.
  2. The practical advantage remains uncertain because the method adds numerous fixed regularization/calibration settings and repeated covariance diagnostics, yet no controlled overhead or sensitivity comparison is provided (Sections 4.1 and 5). All optimization tests use a small deterministic, bi-objective, single-candidate budget. That scope is legitimate, but it makes the unmeasured cost and tuning dependence important when the strongest simpler baseline is already competitive. Together with the attribution gap, this supports a weak-rejection recommendation in the current version rather than an objection to the correctness of the kernel construction.
Questions for authors
  1. Under the identical local-global search, bias correction, and covariance-scaling pipeline, how does a stationary independent GP (and, if already available, a stationary correlated GP) compare with the shared-spectral surrogate on the existing benchmarks? This would materially change the assessment of the surrogate contribution.
  2. Can the search-localization and mode-specific-bias effects in Table 2b be separated, and can the effect of covariance rescaling be quantified independently? Please distinguish any existing evidence from proposed future experiments.
  3. What is the per-iteration fitting/calibration/acquisition overhead on a common machine, and how stable are the main conclusions to a small, prespecified perturbation of the fixed settings? Also, does the implemented model use per-mixture B_q as in the main text, or the common B specified in Appendix B.1?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 3

We thank the reviewer for your suggestions.

(W1) We agree that our original ablation studies did not sufficiently disentangle the contributions of different components. Therefore, we conducted additional controlled ablation experiments. As presented in our overall notes, we introduced four additional configurations:

ConfigurationKernelCross-objective covariance B_qAcquisition covariance rescaling
A: Original MOLRN-BONonstationaryLearnedAdaptive
BNonstationaryLearnedDisabled (factor = 1)
CStationaryLearnedAdaptive
DStationaryLearnedDisabled (factor = 1)
ENonstationaryFixed to IAdaptive

The comparisons A vs. B and C vs. D isolate the effect of covariance rescaling, while A vs. C and B vs. D isolate the effect of the nonstationary kernel under otherwise identical settings.

As shown in our overall notes, A outperforms C on all four metrics, while B outperforms D on three metrics and ties on the remaining one. These results provide direct evidence that the nonstationary surrogate can improve BO performance under the same decision pipeline. Covariance rescaling also improves several metrics, although its effects are not uniformly positive across all metrics.

(W2) We apologize for limiting our original evaluation to relatively low-dimensional, bi-objective problems. We have added two three-objective problems: WFG9 (20D, 3 objectives) and Penicillin (7D, 3 objectives). Our method ranks first and third, respectively, in final nHV on these two problems. Detailed results are provided in our overall notes.

Regarding computational overhead, we acknowledge that our method is substantially more expensive than simpler GP-based BO methods. A typical iteration requires approximately 10–30 minutes, with around 90% of the time spent on MLL optimization and most of the remaining time on acquisition optimization. The main difficulty comes from the complex structure of the nonstationary kernel, which makes both surrogate fitting and acquisition optimization computationally demanding.

Nevertheless, many practical applications of BO, such as hyperparameter optimization and real-world chemical experiments, involve expensive function evaluations. In these settings, the additional computational cost may be acceptable if it leads to better optimization performance or reduces the number of required experiments. We acknowledge that this trade-off depends on the application.

Regarding your questions:

(1) We have provided detailed additional controlled experiments in our overall notes. Configurations C and D use stationary correlated GPs under the same calibration and acquisition optimization procedures, allowing us to isolate the contribution of nonstationarity.

(2) We separately evaluated covariance rescaling in the new experiments, as presented in our overall notes. Regarding local–global acquisition optimization, we found that conventional gradient-based optimization struggled with the complex nonstationary kernel during GP fitting. For example, the negative MLL often decreased only from approximately 300 to above 200, resulting in poorly fitted surrogate models. This motivated our concern that conventional full-domain acquisition optimization could also struggle with the complex posterior landscape. We therefore introduced the Pareto-local/global strategy, which combines heuristic region selection and initialization with gradient-based acquisition optimization. We acknowledge that the individual contribution of this search strategy has not yet been fully separated from mode-specific bias correction.

(3) The computational cost is approximately 10–30 minutes per iteration on a CPU, with MLL optimization accounting for around 90% and acquisition optimization for most of the remaining time. GPU acceleration may reduce this cost, although we have not conducted a controlled comparison.

Regarding hyperparameter sensitivity, our original experiments showed that Q = 2 provided better off-design prediction than Q = 4. Since increasing Q increases the expressive capacity of the MO-LRN kernel, we interpret this as evidence of potential overfitting with larger spectral capacity, which motivated our choice of Q = 2. However, we acknowledge that this does not constitute a comprehensive sensitivity analysis of all regularization and calibration parameters.

Finally, we sincerely apologize for the error in Appendix B.1 (Eq. 11), which was inadvertently retained from an earlier version of our implementation. Eq. 1 and Sec. 4.3 correctly describe the implemented model: the spectral kernel parameters are shared across outputs, but the coregionalization matrices are learned separately for each spectral mixture. We will correct Eq. 11 and the corresponding description in the revised manuscript.

Thank you for the clarifications and additional experiments. The comparison with stationary surrogates under the same search and calibration settings addresses my main concern about the contribution of nonstationarity. The computational cost remains a limitation, but the new evidence strengthens the case for acceptance. I would raise my rating.

Reviewer_TcDROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how non-stationary Gaussian processes (GPs) can be adapted for multiobjective Bayesian optimization (MOBO). The central motivation is that a surrogate model with excellent training accuracy does not necessarily lead to good optimization decisions, particularly when its predictions are inaccurate or overconfident in unexplored regions of the design space.

Their method on the recently proposed Multi-Output Low-Rank Nonstationary (MO-LRN) Gaussian process. It demonstrates that increasing the spectral complexity of this model can improve training accuracy while simultaneously degrading prediction accuracy at unobserved locations. This mismatch is particularly problematic in Bayesian optimization because the acquisition function may exploit inaccurate predictions.

Critical strengths
  • The main strength is the systematic identification of the surrogate accuracy and optimization performance. The diagnostic results in Figure 1 demonstrate that increasing the spectral capacity of MO-LRN can reduce training error while worsening prediction accuracy away from observed samples. The authors also identify severe underdispersion in the posterior predictions at acquisition-selected locations.
Critical weaknesses
  • My principal concern is the lack of a controlled comparison demonstrating that the proposed nonstationary multi-output surrogate is necessary for the reported optimization improvements. MOLRN-BO combines several modifications: regularized spectral modeling, shared cross-objective structure, RBF residuals, mean-bias correction, covariance rescaling, and Pareto-local acquisition optimization. Although the paper provides ablations, the experiments do not fully disentangle these contributions.
  • Although acknowledged as a limitation, the paper addressed only low-dim, bi-objective deterministic optimization. The state of the art in MOBO is well ahead of these settings.
Questions for authors
  • Can the authors provide an end-to-end comparison between MOLRN-BO and independent stationary GPs using exactly the same posterior-correction and local–global acquisition optimization procedures?
  • In Table 2(b), local–global acquisition optimization and mode-specific bias correction are modified simultaneously. Could the authors report separate ablations for these two components, along with the effect of covariance rescaling?
  • Can the authors provide controlled per-iteration computational costs, including surrogate fitting and acquisition optimization, relative to qLogEHVI and qLogNParEGO? Even a representative timing comparison on identical hardware would help assess whether the improvements justify the additional complexity.
4 comments
AuthorsOpenReview ↗

Response to Reviewer 2

We thank the reviewer for your suggestions.

(W1) We agree that our original ablation studies did not sufficiently disentangle the contributions of different components. Therefore, we conducted additional controlled ablation experiments. As presented in our overall notes, we introduced four additional configurations:

ConfigurationKernelCross-objective covariance B_qAcquisition covariance rescaling
A: Original MOLRN-BONonstationaryLearnedAdaptive
BNonstationaryLearnedDisabled (factor = 1)
CStationaryLearnedAdaptive
DStationaryLearnedDisabled (factor = 1)
ENonstationaryFixed to IAdaptive

The comparisons A vs. B and C vs. D isolate the effect of covariance rescaling while holding all other components fixed. Similarly, A vs. C and B vs. D isolate the effect of the nonstationary kernel under otherwise identical settings.

From the results presented in our overall notes, we observe that A outperforms C on all four metrics, while B outperforms D on three metrics and ties on the remaining one. These results support our claim that the nonstationary kernel can improve BO performance compared with a stationary kernel under matched settings. Regarding covariance rescaling, A outperforms B on two metrics, performs slightly worse on one, and ties on the remaining one. Similarly, C outperforms D on three metrics but performs slightly worse on one. These results suggest that covariance rescaling can also provide benefits to BO performance, although its effects are not uniformly positive across all metrics.

(W2) We apologize for limiting our original evaluation to relatively low-dimensional, bi-objective problems. We have added two three-objective problems: WFG9 (20D, 3 objectives) and Penicillin (7D, 3 objectives). Our method ranks first and third, respectively, in final nHV on these two problems. Detailed results for all four metrics are provided in our overall notes.

Regarding your questions:

(1) The controlled comparisons are presented in our overall notes. Configurations C and D use stationary multi-output GPs with the same posterior correction and local–global acquisition optimization procedures as MOLRN-BO. They therefore isolate the contribution of nonstationarity, although they are not independent stationary GPs.

(2) The new experiments separately evaluate the effect of covariance rescaling, while our original Table 2(c) examines hierarchical versus independent bias pooling. Regarding local–global acquisition optimization, we found that conventional gradient-based optimization struggled with the complex nonstationary kernel during GP fitting. For example, the negative MLL often decreased only from approximately 300 to above 200, resulting in poorly fitted surrogate models. This motivated us to develop a more strong optimization strategy. Similarly, we were concerned that traditional full-domain gradient-based acquisition optimization could struggle with the complex posterior landscape induced by the nonstationary kernel. Therefore, we introduced this Pareto-local/global search strategy, which uses heuristic region selection and initialization followed by gradient-based acquisition optimization.

(3) Our experiments were conducted on a Slurm-managed supercomputing system, using a single CPU core for optimization. Due to variations in the execution environment, we did not consider the recorded runtimes sufficiently controlled for a reliable cross-method comparison. Nevertheless, MOLRN-BO is computationally expensive, generally requiring approximately 10–30 minutes per BO iteration, with more than 90% of the time spent on MLL optimization. We acknowledge this computational overhead as a limitation of the current method.

Dear Reviewer TcDR,

As the discussion phase comes to an end, please acknowledge the author's response and consider leaving them a comment on whether it has resolved some or all of your concerns.

Thanks!

Reviewer_TcDROpenReview ↗

Response to authors

I thank the authors for their response to my comments. It seems like they have provided a response to every comment.

  • on the ablation to isolate individual effects, it is not clear what I should takeaway from the provided table. I was hoping to see some plots and visualizations, but I also acknowledge that it is hard to produce that in a short time frame.
  • on the higher-d results, I was not sure where to find the new results. I checked the paper to see if there was a revision, but couldn't find the new results.

I thank the authors once again and hope to discuss this with the AC and other reviewers.

AuthorsOpenReview ↗

Response to Reviewer 2

We apologize for our incovenient representation of the response - due to space limit, we put all the results in a public note above and quoted the note in each response to the reviewers. Here is the detailed results:

WFG9: d = 20, m = 3, k = 4, a higher-dimensional and three-objective configuration.

Penicillin (from BoTorch): d = 7, m = 3.

The ranks of our MOLRN-BO (configuration A) on these two problems are:

ProblemFinal nHVFinal nIGDHV-regret AUCnIGD AUC
WFG9 (20D, 3 objectives)1111
Penicillin (7D, 3 objectives)3433

We then evaluated all 13 methods (the original nine methods and four additional configurations B–E) on all 14 problems (the original 12 problems and the two additional problems). Their mean problem ranks are shown below. Lower ranks indicate better performance.

MethodFinal nHVFinal nIGDHV-regret AUCnIGD AUC
A: MOLRN-BO full3.793.863.504.36
B: NS no rescaling3.503.864.505.00
C: Stationary adaptive4.575.074.794.79
D: Stationary no rescaling4.645.574.505.50
E: NS B_q = I5.075.145.505.36
qLogEHVI5.366.645.075.71
qLogNParEGO4.504.364.004.14
JES9.147.868.717.86
MESMO11.2111.2110.6410.36
mobopt8.217.298.217.07
NSGA-II9.6110.3610.1110.86
Original MO-LRN9.868.439.938.71
Sobol11.5411.3611.5411.29

The controlled comparisons provide several findings. First, the nonstationary configurations consistently outperform their stationary counterparts under matched calibration and acquisition settings, supporting the contribution of the nonstationary surrogate to sequential optimization. Second, covariance rescaling improves both nonstationary and nonstationary surrogates. Third, learning cross-objective covariance improves all four mean ranks compared with fixing the cross-objective covariance to the identity matrix. Overall, MOLRN-BO remains competitive across all four metrics, while the new ablations provide more direct evidence of the contributions of its individual components.

We also apologize for the time limit that we have still not finished revising the manuscript. We will finish it as soon as possible and update all results & ablation study results with a proper visualization.

Reviewer_FRsHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes MOLRN-BO, a multi-objective Bayesian optimization (MOBO) method whose surrogate is a nonstationary multi-output Gaussian process. Standard MOBO models each objective with an independent stationary GP, which neither shares information between objectives nor captures input-dependent structure. The authors build on MO-LRN (Xu et al., 2026), a recent kernel that provides both through low-rank spectral factors, and identify what they call a regression-decision gap when it is used directly in a BO loop: higher spectral capacity improves training fit but worsens off-design prediction, and predictions at the candidates the acquisition function selects are biased and underdispersed (Sec. 3, App. A). Characterizing this gap is their first contribution.

MOLRN-BO adapts both the surrogate and its use, in three steps:

  1. The surrogate shares spectral structure across objectives, models cross-objective dependence through coregionalization, adds an objective-specific smooth residual, and is fitted with spectral regularization (App. B.1).
  2. The posterior passed to the acquisition function, which they call the decision posterior, is calibrated online, with a mean correction learned from past prediction errors at selected candidates and a tempered inflation of the covariance (App. B.2).
  3. The hypervolume-based acquisition is optimized mostly in local regions around the current Pareto points, with periodic global rounds (App. B.3).

As evidence, experiments on synthetic bi-objective benchmarks compare MOLRN-BO with standard MOBO baselines and the unmodified MO-LRN on final and anytime Pareto-front quality (Sec. 4.2). The authors report the best average ranks for final front quality, competitive anytime performance and a clear improvement over the unmodified MO-LRN, and use ablations of the surrogate structure, the search policy and the bias correction to link these effects to individual components (Sec. 4.3).

Critical strengths

Explains why a better-fitting surrogate can make worse BO decisions. The paper traces the failure of the unmodified kernel in BO to concrete mechanisms: extra capacity is spent on fitting the observed data, while the posterior at the points the acquisition actually selects becomes both wrong and overconfident (Sec. 3, App. A). Each component of the method is then motivated by one of these failure modes, which makes the design easy to follow and to scrutinize. The insight reaches beyond MO-LRN: it shows why regression accuracy is a poor criterion for choosing a BO surrogate, and the diagnostics used here can be reused by anyone placing a flexible kernel inside a BO loop.

Rigorous experimental design. The design makes the comparisons fair and the conclusions statistically grounded (Sec. 4.1, App. C): every method starts from the same data on each problem and seed, performance is measured against fixed, normalized reference fronts, differences are tested with paired, multiplicity-corrected tests rather than read off average curves, and tail behaviour is reported alongside means. The authors also state where their method is not significantly better and where it loses. This matters because the reported results can be trusted at face value, so the assessment can focus on what they show rather than on whether they hold.

Critical weaknesses

No test of whether the nonstationary surrogate helps BO. The paper motivates the method by the limits of independent stationary GPs (Sec. 1) and states that its ablations quantify the effect of the spectral structure on optimization (contribution 3, Sec. 1). The spectral structure is only compared as a regressor on fixed data (Table 2a). In BO, MOLRN-BO is compared with baselines that differ from it in the surrogate, the calibration and the search at the same time (Sec. 4.2).

Missing ablation of covariance rescaling. Underdispersed predictions at acquisition-selected candidates are a main motivation for the method (Sec. 3, App. A), and covariance rescaling is the component that corrects them (App. B.2). All BO ablations keep the covariance rescaling fixed (Sec. 4.3), so the paper gives no evidence that it improves optimization performance.

Questions for authors

Clarifications:

  1. Per-problem results. The paper reports results only as mean ranks across problems (Tables 1 and 4). Could you report the per-problem means and standard errors of final nHV and nIGD for all methods? Without them, the size of the differences between MOLRN-BO, qLogNParEGO and qLogEHVI cannot be judged.
  2. Excluded trials. Tables 1 and 4 use "all valid completed trials". How many trials were excluded for each method and problem, and why?
  3. Coregionalization model. Eq. 1 and Sec. 4.3 describe a separate coregionalization matrix for each mixture, while App. B.1 (Eq. 11) uses one matrix shared by all mixtures. Which model was used in the experiments?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 1

We thank the reviewer for the positive assessment and these helpful suggestions.

Both of your critical weaknesses are very important. Therefore, we conducted additional controlled ablation experiments. As presented in our overall notes, we introduced four additional configurations for comparison:

ConfigurationKernelCross-objective covariance B_qAcquisition covariance rescaling
A: Original MOLRN-BONonstationaryLearnedAdaptive
BNonstationaryLearnedDisabled (factor = 1)
CStationaryLearnedAdaptive
DStationaryLearnedDisabled (factor = 1)
ENonstationaryFixed to IAdaptive

The comparisons A vs. B and C vs. D isolate the effect of covariance rescaling while holding all other components fixed. Similarly, A vs. C and B vs. D isolate the effect of the nonstationary kernel under otherwise identical settings.

From the results presented in our overall notes, we observe that A outperforms C on all four metrics, while B outperforms D on three metrics and ties on the remaining one. These results support our claim that the nonstationary kernel can improve BO performance compared with a stationary kernel under matched settings. Regarding covariance rescaling, A outperforms B on two metrics, performs slightly worse on one, and ties on the remaining one. Similarly, C outperforms D on three metrics but performs slightly worse on one. These results suggest that covariance rescaling can also provide benefits to BO performance, although its effects are not uniformly positive across all metrics.

Regarding your questions: (1) We apologize for not including the detailed per-problem results in the appendix. We will add these results, including those from the new experiments, to the appendix of the revised manuscript. (2) We also apologize for not explaining this clearly in the manuscript. The original MO-LRN method (not our adapted MOLRN-BO) encountered out-of-memory (OOM) errors in two trials when running on the supercomputing system, due to difficulties in optimizing the original model. We have now completed these two trials and updated the results accordingly. We will replace the previous results with the updated ones in the revised manuscript. (3) We sincerely apologize for the error in Appendix B.1 (Eq. 11), which was inadvertently retained from an earlier version of our implementation. Eq. 1 and Sec. 4.3 correctly describe the implemented model: the spectral kernel parameters are shared across outputs, but the coregionalization matrices are learned separately for each spectral mixture. We will correct Eq. 11 and the corresponding description in the revised manuscript.

Dear Reviewer FRsH,

As the discussion phase comes to an end, please acknowledge the author's response and consider leaving them a comment on whether it has resolved some or all of your concerns.

Thanks!

Thanks for the response and for the new experiments.

W1. The A vs. C and B vs. D comparisons answer my main concern: with the same search and calibration, the nonstationary kernel does better than the stationary one.

W2. Thank you for isolating this component. The effect is mixed across metrics, so I would suggest the revised paper to describe it that way rather than as a consistent improvement.

Questions. Thanks for clarifying the excluded trials and the coregionalization model, and for adding the per-problem results. It would also help to report significance tests for the new configurations, as in Sec. 4.2.

I'll keep my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall notes about additional experiment results

We thank all reviewers for their thoughtful and constructive comments. In response to the concerns regarding the contribution of the nonstationary surrogate to BO performance, the effect of covariance rescaling, and the limited scope of experimental problems, we conducted the following additional experiments.

  1. Controlled ablation studies.

We introduced five controlled configurations, including the original MOLRN-BO and four additional variants:

ConfigurationKernelCross-objective covariance BqB_qAcquisition covariance rescaling
A: Original MOLRN-BONonstationaryLearnedAdaptive
BNonstationaryLearnedDisabled (factor = 1)
CStationaryLearnedAdaptive
DStationaryLearnedDisabled (factor = 1)
ENonstationaryFixed to IAdaptive

All five configurations retain the same RBF residuals, mean-bias correction, qLogEHVI acquisition function, Pareto-local/global acquisition search policy, and training and acquisition optimization budgets. The same initial design is used for each problem–trial pair.

These configurations allow us to isolate the contributions of individual components:

A vs. B and C vs. D: Evaluate the effect of acquisition covariance rescaling under nonstationary and stationary kernels, respectively (Reviewer 1, W2; Reviewer 2, W1).

A vs. C and B vs. D: Evaluate the contribution of the nonstationary kernel while holding the remaining decision pipeline fixed (Reviewer 1, W1; Reviewer 2, W1; Reviewer 3, W1).

A vs. E: Evaluate the additional value of explicitly learning cross-objective covariance (Reviewer 2, W1; Reviewer 3, W1).

  1. Additional multi-objective benchmarks.

To broaden the evaluation beyond low-dimensional, bi-objective problems, we added two three-objective optimization problems:

WFG9: d = 20, m = 3, k = 4, a higher-dimensional and three-objective configuration.

Penicillin (from BoTorch): d = 7, m = 3.

The ranks of our MOLRN-BO (configuration A) on these two problems are:

ProblemFinal nHVFinal nIGDHV-regret AUCnIGD AUC
WFG9 (20D, 3 objectives)1111
Penicillin (7D, 3 objectives)3433

We then evaluated all 13 methods (the original nine methods and four additional configurations B–E) on all 14 problems (the original 12 problems and the two additional problems). Their mean problem ranks are shown below. Lower ranks indicate better performance.

MethodFinal nHVFinal nIGDHV-regret AUCnIGD AUC
A: MOLRN-BO full3.793.863.504.36
B: NS no rescaling3.503.864.505.00
C: Stationary adaptive4.575.074.794.79
D: Stationary no rescaling4.645.574.505.50
E: NS B_q = I5.075.145.505.36
qLogEHVI5.366.645.075.71
qLogNParEGO4.504.364.004.14
JES9.147.868.717.86
MESMO11.2111.2110.6410.36
mobopt8.217.298.217.07
NSGA-II9.6110.3610.1110.86
Original MO-LRN9.868.439.938.71
Sobol11.5411.3611.5411.29

The controlled comparisons provide several findings. First, the nonstationary configurations consistently outperform their stationary counterparts under matched calibration and acquisition settings, supporting the contribution of the nonstationary surrogate to sequential optimization. Second, covariance rescaling improves both nonstationary and nonstationary surrogates. Third, learning cross-objective covariance improves all four mean ranks compared with fixing the cross-objective covariance to the identity matrix. Overall, MOLRN-BO remains competitive across all four metrics, while the new ablations provide more direct evidence of the contributions of its individual components.

These results will be incorporated into the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.