acceptodds
Under review as a conference paper at ICLR 2027

Uncertainty-Aware Offline Data-Driven Multi-Objective Optimization

11 views

Abstract

In offline data-driven Multi-Objective Optimization (MOO), optimization is performed using surrogate models trained only on an offline dataset. When only limited data are available, surrogate models may exhibit substantial epistemic uncertainty and prediction errors. These errors can lead to incorrect dominance judgments and mislead the search process. Existing uncertainty-aware offline data-driven Multi-Objective Evolutionary Algorithms (MOEAs) often rely on Gaussian Process Regression (GPR), limiting their direct applicability to other surrogate models. Meanwhile, generative-based methods may struggle to explore beyond the offline data distribution when it is far from the true Pareto front. We propose EBU-DR, which combines an Empirical-Bayes Uncertainty (EBU) adjustment with a Dual-Ranking (DR) strategy for evolutionary survival selection. EBU uses five-fold out-of-fold predictions to estimate the shrinkage parameters. When predictive signal is sufficient, it shrinks predictions toward the offline objective mean; otherwise, uncertainty estimates are used directly. DR performs non-dominated sorting on both surrogate-predicted and uncertainty-aware objective values, while crowding distances are calculated using only the surrogate-predicted objectives. We evaluate EBU-DR with four surrogate configurations across 29 benchmark problems and different offline dataset sizes, comparing it with representative offline MOO methods.

Primary area:
optimization
Keywords:
Data-driven optimization, Multi-objective optimization, Multi-objective evolutionary algorithms

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_357qOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies offline data-driven multi-objective optimization, where surrogate models are trained on a fixed dataset and no further real objective evaluations are allowed. The authors argue that surrogate predictions can be overly optimistic in low-density regions and can mislead dominance judgments. They propose EBU-DR, a framework with two components. First, an Empirical-Bayes Uncertainty (EBU) adjustment uses five-fold out-of-fold predictions to estimate objective-wise shrinkage parameters, producing an uncertainty-aware objective fUA-sur that shrinks surrogate predictions toward the offline data mean when predictive signal is weak, and otherwise falls back to the uncertainty estimate. Second, a Dual-Ranking (DR) strategy concatenates the surrogate-predicted objectives and the uncertainty-aware objectives into a 2K-dimensional hybrid vector, performs non-dominated sorting in this hybrid space, and computes crowding distances using only the original K surrogate objectives. The framework is combined with GPR (RBF and Matérn), quantile regression, and BNN surrogates, and evaluated with NSGA-II on 29 benchmark problems, five offline dataset sizes, and several offline MOO baselines. The paper claims more wins than losses against surrogate-only NSGA-II, competitive HV ranks, and the best overall mean rank for GPR (Matérn)+EBU-DR, while acknowledging that improvements are concentrated in GPR configurations and that better optimization performance does not always correspond to lower surrogate prediction error.

Critical strengths
  1. Offline data-driven MOO with small datasets is an important setting, and the paper correctly identifies that surrogate-only dominance can be misled by optimistic predictions in low-density regions. The illustrative example in Figure 1 makes the failure mode clear.

  2. EBU-DR does not require a new surrogate architecture or a new evolutionary algorithm; it can be plugged into NSGA-II and combined with different surrogate models. This modularity is a genuine practical advantage, and the paper does evaluate four surrogate configurations rather than only GPR.

  3. The paper considers 29 benchmark problems, five offline dataset sizes, multiple random permutations and optimization seeds, and a reasonably wide set of offline MOO baselines including probabilistic, deep-learning, and generative methods.

Critical weaknesses
  1. EBU is essentially a shrinkage estimator derived under a Gaussian/additive-error model, and DR is a concatenation-based ranking trick. The paper does not provide a guarantee that sorting in the 2K-dimensional hybrid space corrects dominance errors, nor does it characterize when the EBU weight is consistent with the actual surrogate error. For QR and BNN surrogates, the Gaussian likelihood and additive-error assumptions behind Eq. (3)–(4) are not obviously satisfied. The fallback to s_k(x) when the covariance signal is weak is a heuristic, and the paper does not analyze its effect on the ordering of solutions. This matters because the central claim is an uncertainty-aware ranking framework, not merely an empirical heuristic.

  2. The empirical evidence is narrower than the framing suggests, and the key comparison is not decisive. The strongest results are obtained with GPR surrogates, especially Matérn; QR and BNN show many ties and limited gains. The ablation compares EBU-DR only against Normal and a fixed-quantile DR variant. It does not compare against simple and standard uncertainty-aware alternatives, such as adding a variance penalty to the objective, LCB/UCB-style optimistic/pessimistic scalarization, or ensemble-variance regularization.

  3. PCD and ParetoFlow are generative methods that are not subject to the same 10,000 surrogate-evaluation budget; MOBO is excluded because its results are incomplete. The paper says method-specific training and search mechanisms follow original implementations, but the comparison mixes methods with different computational budgets and different notions of “offline” evaluation.

  4. Scalability and computational cost are not adequately analyzed.

Questions for authors
  1. Can you add comparisons against simple uncertainty-aware selection baselines, such as adding a variance penalty to the surrogate objective, LCB/UCB-style scalarization, or ensemble-variance regularization, under the same surrogate and evaluation budget?

  2. Can you provide a more formal justification for DR? Specifically, under what conditions does non-dominated sorting in the 2K-dimensional hybrid space correct a surrogate-induced dominance error, and when can it introduce a new error?

  3. The EBU derivation assumes a Gaussian likelihood and additive error, but QR and BNN surrogates do not naturally satisfy these assumptions. How stable is the estimated c_k and τ_k^2 across folds and dataset sizes, and how often does the covariance-signal condition in Eq. (4) trigger the fallback branch? Can you report these statistics?

  4. why is the comparison with Multiple Models on IGD+ not discussed more prominently, given that it outperforms GPR (Matérn)+EBU-DR on that metric?

Reviewer_71n3OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes an adaptation mechanism for evolutionary algorithms to deal with multi-objective optimization problems where there is uncertainty regarding the evaluation assessment of certain (scarce) data points used to do the optimization.

Critical strengths

None

Critical weaknesses
  • Lack of motivation. In this paper, the authors address multi-objective optimization problems that include uncertainty in certain datapoints. The problem might be interesting, however, being a researcher with broad experience in the optimization field, I have doubts about the relevance of the problem. I think the authors could contribute with real illustrative examples, and use such real data (instead of artificial benchmarks) in the experimentation.

  • Poor Problem definition. It is not clear whether "surrogate" refers to objective function, or a construction on top of objective functions, or either a model that tries to approximate objective functions. The paper requieres a proper definition of the different terms. Moreover, the authors should provide a broad definition of MOPs, and afterwards, introduce mathematically the uncertainty feature on top of that.

  • Obscure literature revision. The provided references reveal that there are many different research paths to follow in the literature (evolutionary algorithms, learning techniques, hybrids...), however, the authors choose one of this directions without properly supporting such decision.

  • Wrong bibliography usage. The style followed by the authors to cite the works is erroneous. In most of the cases the authors should use the latext command \citet, instead of what they are using currently.

Questions for authors

The paper is far for being accepted. In my opinion, the paper is not at a mature stage. I encourage the authors to assess the weaknesses identified above.

Reviewer_FmrcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose EDU-DR, an uncertainty aware predictor that can be used in conjunction with existing data-driven, Multi-Objective Optimization (MOO) algorithms. More specifically, the authors note that many existing surrogate based optimization methods rely too heavily on the inaccurate surrogates, which may lead to incorrect dominance judgments. To alleviate this issue, the authors propose to Empirical Bayes Uncertainty (EDU) to create an uncertainty aware surrogate, estimated using five-fold out-of-fold predictions, which are then paired with NSGA-II algorithm. To utilize these uncertainties, the authors propose to create an augmented objective, consisting of the original surrogate values and the uncertainty aware values, which is then used as input for non-dominated sorting. However, the crowding distances are calculated only from the traditional surrogate predictions.

The authors consider 4 types of surrogates, and evaluate the performance in 29 tasks from the offline-moo benchmark. The performance of the solutions sets is evaluated with hypervolume and IGD+, while the learned surrogates are evaluated using MSE with respect to the test set included in the benchmarks. The results showcase that the proposed method can achieve better average rank than the baselines in hypervolume, while losing out in IGD+. Moreover, the authors note that the prediction accuracy on the test-set doesn’t necessarily correlate with the final solution set performance.

Critical strengths
  • Authors study a well motivated question of how to make surrogate based MOO more robust against errors in the predictions, which is a common issue when pairing existing optimization algorithms with surrogates.
Critical weaknesses
  • Limited benchmark tasks. The evaluations are performed only on synthetic and RE tasks. I would like to see evaluations also on discrete domain tasks, such as scientific design tasks.
  • Limited novelty / applicability: the authors propose quite straightforward modification to existing surrogate based methods. However, their application of the uncertainty aware predictions is specific to making it difficult to see how this approach could be applied more broadly to other existing MOO methods.
Questions for authors
  • Currently, the non-dominated sorting is performed using a combination of standard surrogate predictions and uncertainty aware predictions. How would the proposed method perform against just using the uncertainty-aware predictions for non-dominated sorting?
  • You mention MOBO methods as one of your baselines, yet don’t include it in any of the results. Either remove the mention, or include the results.
  • The ablations on EBU-DR components (Table 1) and dataset size (Table 3) show only the number of wins and losses. It would be nice to also see the numerical values to better understand how big of an impact the different choices / dataset size make.
  • Similarly, having full numerical evaluation results used for producing Table 2 would be appreciated.
Reviewer_oYJCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the Offline Data-Driven Multi-Objective Optimization (Offline MOO) problem, as surrogate models are trained on a fixed offline dataset. Then, these model are used to guide the optimization process without further real-world evaluations. A major issue in this setup is that surrogate models can produce inaccurate predictions, particularly when the optimization algorithm explores regions out of the training data. Such errors can lead to incorrect dominance decisions.

Overview contributions:

  • Empirical-Bayes Uncertainty Adjustment (EBU): The authors employ five-fold cross-validation to estimate parameters from out-of-fold (OOF) predictions and uncertainty estimates. They then adjust the surrogate predictions by pulling those with high uncertainty toward the mean of the offline objectives.
  • Dual-ranking strategy: This approach combines surrogate predictions and uncertainty-aware values into a hybrid objective space to perform non-dominated sorting within the NSGA-II framework, while the crowding distance continues to be calculated based on the original surrogate objectives.

I lean toward weak rejection for two main reasons: (1) EBU relies on statistical assumptions that may not hold for biased or misspecified surrogate models. This limitation relates to the generalization of the framework. (2) The paper does not demonstrate that its improvement comes from uncertainty awareness rather than from the weakened dominance induced by the 2K dimensional sorting.

Critical strengths
  1. The problem is relevant and motivated ,addressing a challenge problem in offline data-driven MOO.
  2. The method is designed to be compatible with different surrogate models and requires no additional real evaluations.
  3. The authors provide comprehensive results, as the experimental evaluation covers 29 benchmark problems, multiple offline dataset sizes, four surrogate configurations, and 10 data seeds and 10 optimization seeds.
Critical weaknesses
  1. Inconsistent effectiveness across surrogate models: The additional contribution of EBU over DR appears to be concentrated mainly in GPR models. For QR and BNN, EBU often becomes inactive. This result raise a question about the generality of EBU.
  2. The mechanism behind DR is not established: The paper does not clearly explain why the Dual-Ranking (DR) mechanism improves the Pareto search process. Expanding the objective space from K to 2K by combining surrogate objectives with uncertainty-aware objectives, is merely a heuristic design; the paper fails to sufficiently analyze how this combination influences non-dominated sorting, selection pressure, and the quality of the final solutions.
  3. Experimental improvements are not consistent across evaluation metrics and benchmark families. GPR (Matern)+EBU-DR achieves the best overall and HV ranks but not the best IGD+ rank (Multiple Models) or MSE rank (Prob-MOEA/D).
Questions for authors
  1. Could the authors provide further analysis of why EBU becomes inactive for QR and BNN, and discuss how this affects the general applicability of the proposed framework?
  2. Can the authors include an EBU-only variant that sorts on f_UA-sur alone? This would show whether the improvement comes from the uncertainty-aware objectives themselves or from combining them with f-sur.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Responses

Dear AC, SAC, PC

Thank you for organizing the comments and responses. We have corrected minor errors, added experiments and analyses, and uploaded the revised manuscript. We summarize the responses here and [Response to Reviewer xxxx], starting with the Introduction section.

1. Introduction

We added a real-world example of a smart building MOP to show that offline data-driven MOO research is valuable for tackling real-world MOPs. [To 71n3]

We clarified why we chose evolutionary methods: MOEAs can explore beyond the offline data distribution (compared with generative methods), and surrogate prediction errors can affect dominance judgments during evolutionary search. [To 71n3]

2. Preliminaries and 3.Method

We have defined the general MOP and surrogate modeling, and shown the difference between offline data-driven and surrogate-assisted MOO in Section 2. We have defined surrogate predictions and uncertainty estimates in Section 3. [To 71n3]

We added detailed explanations of EBU adjustment. [To 357q]

We added an example of when EBU-DR is effective/ineffective, and the limitation that DR may remove a correct dominance relation. [To 357q]

4. Experiment

We clarified that all methods use the same offline datasets and output 100 candidate solutions, but the computational budgets may differ. [To 357q]

We corrected an error to unify the IGD+ reference settings, and included MOBO results. GPR (Mat'ern)+EBU-DR retains the best overall rank. [To Fmrc, 357q]

We added 6 discrete benchmarks, including scientific design tasks. Multiple Models with COMs ranks first, followed by QR+EBU-DR. Across all 35 benchmarks, GPR (Mat'ern)+EBU-DR retains the best overall rank. [To Fmrc]

We added tables of per-benchmark numerical results in the Appendix, following ParetoFlow and PCD. [To Fmrc]

We added an EBU-only variant experiment to examine the contribution of DR. EBU-DR achieves more wins in IGD+ and HV, especially with GPR, but no consistent advantage in MSE. [To oYJC, Fmrc]

We analyzed why EBU is often inactive for QR and BNN. In most cases, no shrinkage occurs, and EBU-DR becomes Normal. [To oYJC, 357q]

We added comparisons with uncertainty-aware baselines: fixed-quantile DR and standard-deviation penalty (fsur,k(x)+βsk(x)f_{\mathrm{sur},k}(\boldsymbol{x})+\beta s_k(\boldsymbol{x}). EBU-DR achieves more wins in IGD+ and HV but performs worse in MSE. [To 357q].

We added a Dimension Expansion Analysis, examining whether the benefits of DR come from increasing the number of objectives KK -> 2K2K. Compared with other 3 shuffled variants, EBU-DR removes more incorrect (18.71%) than correct (4.10%) dominance relations, but it cannot guarantee that all removed relations are incorrect. Detailed investigation of the EBU-DR mechanism is provided in the Appendix. [To oYJC, 357q]

We added a Computational Cost Analysis, the additional cost of EBU-DR comes from EBU calibration (out-of-fold training and predictions) on the surrogate side and dual-ranking survival selection on the optimization side. One-time EBU calibration substantially increases training time (144%-465%), while DR adds little optimization time (3%). [To 357q]

We added experiments combining EBU-DR with other MOEAs, NSGA-III and SMS-EMOA. EBU-DR shows consistent improvements with NSGA-III relative to NSGA-II, while the results with SMS-EMOA are mixed. [To Fmrc, 357q]

5. Conclusion

We summarized the new experimental results and updated the main findings. We added future work on studying large-scale MOPs, many-objective problems, and additional real-world problems. [To oYJC, Fmrc, 71n3, 357q]

Appendix

We added EBU calibration diagnostics, including shrinkage / no-shrinkage / fallback, parameter stability across folds, and dominance changes caused by fallback. The results show that EBU calbration is surrogate-dependent and fallback may remove correct dominance relations. [To oYJC, 357q]

Appendix added the discrete benchmark settings and results, per-benchmark numerical results, dataset-size ablation results, and surrogate training costs. [Fmrc, 357q]

Thank you for your time!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.