acceptodds
Under review as a conference paper at ICLR 2027

Can Multi-Objective Bayesian Optimisation Work Without Posterior Uncertainty?

Abstract

Bayesian optimisation (BO) is a sample-efficient approach for solving expensive black-box optimisation problems. It uses a probabilistic surrogate model to provide a posterior distribution over the unknown objective function and an acquisition function to select solutions for evaluation. Acquisition functions commonly use both the model's posterior mean and posterior uncertainty to balance exploitation and exploration, favouring solutions with better predicted objective values while exploring regions where predictions are less certain. Most multi-objective Bayesian optimisation (MOBO) methods follow such a design. In this paper, we argue that it may not be necessary to consider posterior uncertainty in every MOBO design. In single-objective BO, using only the posterior mean may keep the search near the currently predicted best solution, leaving other regions unexplored. In MOBO, however, considering the posterior means across different objectives (i.e., their nondominance relations) can also facilitate exploration. Rather than focusing solely on solutions with the best posterior mean for each objective, the search can identify solutions representing different trade-offs among the objectives. These trade-off solutions may be located in different areas in the search space, allowing exploration to emerge from these areas without explicitly modelling posterior uncertainty. Based on this, we propose a simple MOBO framework that considers only the posterior means of the surrogate models. We investigate two instantiations of this framework: one directly approximates the Pareto front formed by the posterior means, while the other optimises the scalarised posterior means to learn the corresponding Pareto set. We evaluate it on a wide range of benchmark and real-world problems. The results show that it performs very competitively with state-of-the-art methods, even under noisy settings.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
expensive black-box optimisation, multi-objective Bayesian optimisation, posterior mean

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HLbsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Based on the idea that explicitly utilizing the posterior uncertainty is not always necessary in Multi-Objective Bayesian Optimization (MOBO), this study proposes a novel MOBO framework, termed PMOBO, which relies solely on the posterior mean. Unlike single-objective optimization, this approach adopts the premise that the exploration of non-dominated solutions which are guided by inter-objective trade-offs in MOBO can facilitate diverse area exploration even without explicit posterior uncertainty. Experiments using many benchmarks and real-world problems demonstrate that PMOBO achieves performance equal to or better than existing MOBO methods, regardless of the presence of noise, and further provide empirical evidence that explicitly introducing posterior uncertainty can sometimes degrade performance.

Critical strengths
  1. A Simple yet Significant Idea Questioning the Necessity of Uncertainty in MOBO This work focuses on how diverse candidates are generated in multi-objective settings through inter-objective trade-offs, leading to the proposal of a method that does not explicitly use posterior variance. The research holds significant merit because it challenges the assumed necessity of utilizing uncertainty in existing MOBO methodologies.

  2. Empirical Support Through Extensive Experiments The comparisons across 79 problems, conducted over 30 iterations, and evaluated under both noisy and noise-free conditions against numerous existing techniques are highly commendable. Furthermore, the validation across both Pareto-based and scalarization-based approaches—demonstrating performance degradation when posterior variance was added—suggests that this phenomenon is not limited to a single algorithm.

Critical weaknesses
  1. Lack of Theoretical Analysis Regarding the Claim of Exploration Without Posterior Variance The notion that inter-objective trade-offs can generate diverse solutions does not equate to the ability to adequately explore unknown regions. Furthermore, while trade-offs can be represented using the posterior mean, they cannot fully capture epistemic uncertainty. A theoretical explanation of the conditions under which sufficient exploration is guaranteed is lacking. While not every paper needs to be backed by theoretical analysis, when proposing a new idea, a high-quality paper should ideally include an analysis of why it works, under what conditions and why it succeeds, as well as when and why it fails.

  2. Limited Methodological Novelty Compared to Existing Techniques Prior studies that approximate the Pareto front using the posterior mean and Hypervolume Improvement (HVI), such as DGEMO and MOBO-OSD, already exist. Specifically, maximizing HVI over the Pareto set derived from the posterior mean is equivalent, under certain conditions, to directly maximizing HVI based solely on the posterior mean. Thus, the core novelty appears to lie not in the algorithm itself, but rather in the systematic empirical analysis concerning the necessity of posterior uncertainty.

Questions for authors
  1. What are the conditions under which exploration is successful? Can sufficient exploration actually be achieved using only the posterior mean, even when the trade-offs between objectives are weak or when the Pareto set is extremely narrow? A theoretical answer to this question is required.

  2. How can the mechanism of exploration be empirically verified? Beyond achieving a high Hypervolume (HV), could the methodology verify the exploration mechanism driven by trade-offs by examining the spatial distribution of selected points or by analyzing the posterior variance?

  3. What is the fundamental difference from existing posterior-mean-based methods? What specific advantage does the proposed candidate generation mechanism offer compared to existing methods like DGEMO or MOBO-OSD, or compared to direct maximization of HVI using only the posterior mean?

  4. What happens when non-GP surrogates are utilized? If only the posterior mean is necessary, would similar performance be achievable using predictions from methods like Radial Basis Function (RBF) interpolation or neural networks? Furthermore, how does the total computation time—including model fitting and candidate selection—compare to existing methods?

  5. What is the source of superior performance in noisy settings?

Reviewer_vQo2OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes a multi-objective Bayesian optimization (MOBO) framework called PMOBO. Unlike conventional MOBO methods that incorporate GP posterior uncertainty into their acquisition functions, PMOBO uses only posterior means to guide the search. The authors propose two variants: Pareto-based and scalarization-based. The Pareto-based variant optimizes the posterior means of all objectives using NSGA-II to approximate the Pareto front, then selects the next observation by maximizing hypervolume improvement (HVI). The scalarization-based variant modifies Pareto Set Learning (PSL; Lin et al., 2022) by replacing its surrogate objective μ(x)−βσ(x)\mu(x)-\beta\sigma(x) with μ(x)\mu(x). Both variants are simple and straightforward. Extensive experiments on 79 benchmark and real-world problem instances demonstrate competitive performance against existing MOBO methods.

Critical strengths
  • Simple and straightforward approach: Both proposed variants are easy to understand and implement.
  • Extensive benchmark evaluation: The experiments cover 79 problems from BBOB, DTLZ, and real-world RE benchmarks, with comparisons against several established MOBO methods. Noisy setting is also considered to some extent.
  • Strong empirical performance: Despite its simplicity, the proposed Pareto-based method seems to achieve competitive hypervolume performance on many tested problems.
Critical weaknesses
  1. Limited novelty of the main contribution. The central research question is whether posterior uncertainty is necessary for MOBO. However, several prior works have already implicitly demonstrated that posterior means alone can lead to competitive performance, particularly when combined with HVI. For example, DGEMO (Konakovic Lukovic et al., 2020) directly uses GP posterior means as acquisition functions to approximate the Pareto front, followed by HVI maximization (based on the posterior means) to select the next observations. Similarly, MOBO-OSD (Ngo et al., 2025) also employs posterior-mean-based HVI for candidate selection. The strong performance of these methods already suggests that posterior-mean-based HVI can be effective without explicitly incorporating posterior uncertainty into candidate selection. PMOBO follows essentially the same idea, with the primary methodological difference being its use of NSGA-II for Pareto front approximation (instead of the exploration mechanisms in DGEMO and MOBO-OSD). Therefore, while the proposed method is simple and effective, I think the methodological novelty limited.

  2. Incomplete experimental comparisons. Although the experiments cover a large number of benchmark problems, several important comparisons are missing. In the noisy experiments, the authors exclude DGEMO, PSL, and MOBO-OSD because their original studies focus on noiseless settings. I find this justification weak, as PMOBO itself does not introduce any explicit mechanism to handle observation noise either. All these methods rely on surrogate models that can be fitted to noisy observations. Therefore, excluding these three closely related methods makes the noisy comparisons less comprehensive. Furthermore, the paper does not evaluate PMOBO in batch (parallel) optimization settings, which are commonly considered in recent MOBO literature (DGEMO, MOBO-OSD, PSL, EHVI, etc.). Batch setting is important as it enable MOBO to scale to large number of observations, which may be required to account for the larger number of surrogate models.

  3. Limited evaluation of the scalarization-based variant. Although the paper proposes two variants of PMOBO, the experimental evaluation focuses primarily on the Pareto-based variant. The scalarization-based variant is evaluated only against the original PSL, without the same baselines, noisy experiments, or uncertainty ablations. This makes the scalarization-based variant appears to be a secondary contribution, hence limiting the evidence that the proposed posterior-mean-only principle can generalize across different MOBO strategies.

  4. Limited insight from the posterior uncertainty experiments. Section 6.2 investigates the effect of incorporating posterior uncertainty by replacing the posterior means with LCB acquisition functions. However, this is only one particular way of providing uncertainty into MOBO. Increasing β\beta makes the acquisition values increasingly optimistic, potentially distorting the estimated Pareto front. Therefore, the observed performance degradation is not surprising in my opinion. Other methods, such as EHVI, incorporate uncertainty directly through expected hypervolume improvement rather than treating confidence bounds as objective predictions. Nevertheless, the effectiveness of EHVI suggests that incorporating posterior uncertainty is not inherently harmful to MOBO. Overall, this experiment only demonstrates that posterior uncertainty does not improve the proposed Pareto-based PMOBO. It provides limited insight into the broader question of whether posterior uncertainty is beneficial for the general MOBO.

Questions for authors
  1. Can the authors include DGEMO, PSL, and MOBO-OSD in the noisy experiments to provide a more comprehensive comparison?
  2. How would PMOBO perform in batch optimization settings? Can the authors provide experimental results to demonstrate its effectiveness?
  3. Can the authors better summarize the contribution of the proposed NSGA-II warm-start strategy? Although an ablation study is provided, the large number of results makes it difficult to assess its overall effectiveness. I think the authors can report the average percentage improvement or a direct comparison of (w/o warm start) against the baselines would help clarify how much the warm-start strategy contributes to PMOBO's performance.
  4. How do the authors reconcile their findings with the effectiveness of uncertainty-aware methods such as EHVI? Since PMOBO and EHVI both aim to maximize hypervolume improvement but use different optimization strategies (NSGA-II on posterior mean vs direct gradient-based HV optimization), it is unclear where PMOBO's performance gains come from.
  5. Can the authors provide more details about the GP surrogate settings such as the hyperparameter priors, kernel parameter bounds, noise modelling, etc.?
Reviewer_SEC6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper challenges a conventional assumption in Bayesian optimization: that posterior uncertainty must be explicitly incorporated into the acquisition function to achieve an effective balance between exploration and exploitation. In conventional single-objective Bayesian optimization, relying exclusively on the posterior mean often leads to excessive exploitation, since the optimizer repeatedly favors regions believed to contain the best solutions. The authors argue that multi-objective optimization is fundamentally different because the existence of competing objectives naturally produces multiple trade-off solutions. Exploring these trade-offs may provide sufficient diversity without explicitly using posterior uncertainty.

Critical strengths
  • the empirical evidence to back up their claim
Critical weaknesses
  • I fail to understand why the authors claim uncertainty is not necessary for effective MOBO? While the authors show empirically this might be try, the why is missing.
  • Also, to be clear, the authors demonstrate their method with GP posterior mean which, by itself, is a byproduct of the uncertainty in the GP. That is, the posterior mean is the "expected" value the GP produces at a candidate input. Without posterior uncertainty, there is no meaning behind expecation.
Questions for authors
  • Appendix B.1 indicates that PMOBO uses NSGA-II with 100 individuals and 500 generations at every Bayesian optimization iteration. Can the authors report the corresponding acquisition optimization time and total wall-clock time relative to the competing methods, particularly qPOTS and qLogEHVI? How sensitive is PMOBO's performance to reducing the number of NSGA-II generations, for example from 500 to 100 or 50?
  • Can the authors provide direct experimental evidence that the nondominance relationships among posterior-mean predictions lead to meaningful exploration of previously unobserved regions? For example, could they characterize decision-space coverage, discovery of previously missed Pareto-front components, or performance on problems with disconnected or highly localized Pareto-optimal regions?
  • How would the method adapt the presence of constraints?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.