- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper present a new methods for MOBO problems. with finite function evaluations. The proposed searching method is flexible so that it can be used with different AC functions. Many problems show the effectiveness of the proposed method.
Critical strengths
Multi-objective optmization is an important problem.
Critical weaknesses
-
The font size in the tables are too small, making the content difficult to read.
-
The experimental evaluation is limited in scope, relying solely on long-established benchmarks suites such as ZDT and DTLZ. Given the advances in multi-objective optimization over the past two decades, the authors should include more recent and challenging benchmark problems to provide stronger evidence of the proposed method’s effectiveness.
-
No real world problems.
-
Too many words in Page 9, experiments are not so strong.
Questions for authors
-
How is the reference points are selected in hv testing?
-
How can you involve more real-word tesing problems?
-
how is the performance on m=4,5,6,7? m=2/3 are just very easy to understand. too many existing works.
-
What is the usage of Lem 1. It looks like that you have not built more theorems, so it is uncecssay to include lems.
5 comments
Response [2/3]
Q3: Evaluation of CTD on problems with more objectives ().
CTD is expected to be competitive on many-objective problems. This is because CTD only considers convergence in the first stage, hence approaching the Pareto front substantially faster in a higher-dimensional space, whereas competing methods consider a number of directions simultaneously towards the Pareto front, resulting in slow convergence within a limited budget.
To confirm this, we have included the experiments on DTLZ problems with . Table A below presents the results. As shown, CTD achieves statistically better or equivalent HV than all competing methods on at least 26 out of 28 problems. We have now incorporated this comparison into the paper (Tables 3-6, page 26).
Table A: HV Results on DTLZ with .
| Method | DTLZ1(4) | DTLZ2(4) | DTLZ3(4) | DTLZ4(4) | DTLZ5(4) | DTLZ6(4) | DTLZ7(4) |
|---|---|---|---|---|---|---|---|
| Sobol | 2.88e+05~ | 1.47e+00+ | 0.00e+00+ | 5.43e-01+ | 3.90e+00+ | 1.91e+01+ | 0.00e+00+ |
| ParEGO | 1.06e+06~ | 1.26e+00+ | 9.06e+08~ | 5.06e-01+ | 4.56e+00+ | 1.29e+02~ | 3.23e-01~ |
| TS-TCH | 1.74e+05~ | 1.23e+00+ | 0.00e+00+ | 4.24e-01+ | 4.19e+00+ | 2.98e+01+ | 0.00e+00+ |
| PSL | 2.34e+06~ | 1.35e+00+ | 1.58e+09~ | 5.30e-01+ | 4.70e+00+ | 1.33e+02~ | 2.48e-01~ |
| JES | 4.05e+05~ | 1.13e+00+ | 0.00e+00+ | 3.40e-01+ | 3.79e+00+ | 2.56e+01+ | 0.00e+00+ |
| EHVI | 2.30e+06~ | 1.18e+00+ | 1.68e+09- | 3.19e-01+ | 4.29e+00+ | 1.33e+02~ | 1.48e-01~ |
| MORBO | 5.24e+05~ | 1.35e+00+ | 0.00e+00+ | 3.85e-01+ | 4.26e+00+ | 3.10e+01+ | 0.00e+00+ |
| MOBO-OSD | 2.08e+06~ | 9.63e-01+ | 1.41e+09~ | 5.85e-01+ | 4.43e+00+ | 1.34e+02~ | 1.96e-01~ |
| qPOTS | 3.09e+05~ | 1.30e+00+ | 0.00e+00+ | 4.11e-01+ | 4.07e+00+ | 2.89e+01+ | 0.00e+00+ |
| CTD | 1.50e+06 | 2.51e+00 | 1.04e+09 | 1.65e+00 | 4.79e+00 | 1.34e+02 | 2.50e-01 |
| Method | DTLZ1(5) | DTLZ2(5) | DTLZ3(5) | DTLZ4(5) | DTLZ5(5) | DTLZ6(5) | DTLZ7(5) |
| Sobol | 3.23e+07~ | 3.98e-01+ | 1.91e+10+ | 5.30e-01+ | 1.75e+01+ | 3.82e+02+ | 0.00e+00+ |
| ParEGO | 1.33e+07~ | 2.46e-01+ | 2.18e+11+ | 4.07e-01+ | 2.02e+01~ | 1.19e+03~ | 3.80e-01+ |
| TS-TCH | 3.00e+06+ | 2.45e-01+ | 0.00e+00+ | 4.11e-01+ | 1.86e+01+ | 4.52e+02+ | 0.00e+00+ |
| PSL | 9.36e+07~ | 3.68e-01+ | 5.99e+11~ | 5.20e-01+ | 1.96e+01~ | 1.19e+03~ | 4.85e-01+ |
| JES | 7.84e+06~ | 2.46e-01+ | 0.00e+00+ | 3.04e-01+ | 1.78e+01+ | 3.97e+02+ | 0.00e+00+ |
| EHVI | 8.01e+07~ | 3.26e-01+ | 4.84e+11~ | 3.47e-01+ | 1.88e+01+ | 1.20e+03~ | 3.71e-01+ |
| MORBO | 5.76e+06~ | 3.99e-01+ | 1.56e+10+ | 4.74e-01+ | 1.92e+01+ | 4.35e+02+ | 0.00e+00+ |
| MOBO-OSD | 9.17e+07~ | 4.65e-01+ | 6.01e+11~ | 4.20e-01+ | 2.04e+01~ | 1.18e+03~ | 4.26e-01~ |
| qPOTS | 3.09e+07~ | 4.02e-01+ | 0.00e+00+ | 5.17e-01+ | 1.89e+01+ | 4.52e+02+ | 0.00e+00+ |
| CTD | 2.66e+07 | 1.30e+00 | 4.46e+11 | 1.36e+00 | 2.05e+01 | 1.19e+03 | 5.56e-01 |
| Method | DTLZ1(6) | DTLZ2(6) | DTLZ3(6) | DTLZ4(6) | DTLZ5(6) | DTLZ6(6) | DTLZ7(6) |
| Sobol | 6.74e+09~ | 6.16e+01+ | 1.08e+13~ | 4.32e+01+ | 7.36e+00+ | 4.16e+03+ | 0.00e+00+ |
| ParEGO | 5.67e+09~ | 6.38e+01+ | 8.99e+14~ | 4.32e+01+ | 7.88e+00+ | 1.07e+04+ | 5.68e-01~ |
| TS-TCH | 6.25e+08~ | 6.46e+01+ | 3.88e+12~ | 4.43e+01+ | 7.79e+00+ | 4.99e+03+ | 0.00e+00+ |
| PSL | 1.32e+10~ | 6.03e+01+ | 1.87e+15- | 4.12e+01+ | 7.18e+00+ | 1.07e+04+ | 0.00e+00+ |
| JES | 6.73e+09~ | 6.13e+01+ | 2.14e+13~ | 4.29e+01+ | 7.20e+00+ | 4.12e+03+ | 0.00e+00+ |
| EHVI | 2.00e+10~ | 6.14e+01+ | 1.66e+15~ | 4.18e+01+ | 7.97e+00+ | 1.08e+04+ | 2.96e-01~ |
| MORBO | 3.48e+09~ | 6.47e+01+ | 0.00e+00+ | 4.44e+01+ | 7.74e+00+ | 5.29e+03+ | 0.00e+00+ |
| MOBO-OSD | 1.61e+10~ | 6.02e+01+ | 1.80e+15~ | 4.39e+01+ | 8.19e+00+ | 1.09e+04+ | 3.28e-01~ |
| qPOTS | 1.45e+08~ | 6.49e+01+ | 3.95e+12~ | 4.45e+01+ | 7.88e+00+ | 4.87e+03+ | 0.00e+00+ |
| CTD | 8.65e+09 | 6.94e+01 | 8.53e+14 | 4.80e+01 | 8.64e+00 | 1.09e+04 | 3.28e-01 |
| Method | DTLZ1(7) | DTLZ2(7) | DTLZ3(7) | DTLZ4(7) | DTLZ5(7) | DTLZ6(7) | DTLZ7(7) |
| Sobol | 2.27e+13~ | 3.14e+02~ | 2.48e+20+ | 2.11e+02+ | 1.24e+01+ | 3.86e+04+ | 0.00e+00+ |
| ParEGO | 9.99e+12~ | 3.32e+02~ | 3.75e+20~ | 2.18e+02~ | 1.35e+01+ | 8.69e+04~ | 2.91e-01+ |
| TS-TCH | 1.43e+13~ | 3.26e+02~ | 2.73e+20+ | 2.20e+02~ | 1.35e+01+ | 4.62e+04+ | 0.00e+00+ |
| PSL | 5.40e+13- | 3.00e+02~ | 3.71e+20~ | 2.03e+02+ | 1.21e+01+ | 8.48e+04+ | 0.00e+00+ |
| JES | 3.69e+12~ | 3.14e+02~ | 2.52e+20+ | 2.10e+02+ | 1.25e+01+ | 4.14e+04+ | 0.00e+00+ |
| EHVI | 3.21e+13- | 2.98e+02~ | 4.11e+20~ | 2.03e+02+ | 1.30e+01+ | 8.61e+04~ | 2.85e-01+ |
| MORBO | 6.87e+12~ | 3.28e+02~ | 2.83e+20+ | 2.20e+02~ | 1.34e+01+ | 4.70e+04+ | 0.00e+00+ |
| MOBO-OSD | 3.21e+13~ | 3.00e+02~ | 3.96e+20~ | 2.03e+02+ | 1.29e+01+ | 8.60e+04~ | 3.38e-01~ |
| qPOTS | 1.43e+13~ | 3.25e+02~ | 2.74e+20+ | 2.20e+02~ | 1.34e+01+ | 4.76e+04+ | 0.00e+00+ |
| CTD | 5.93e+12 | 3.33e+02 | 3.56e+20 | 2.25e+02 | 1.41e+01 | 8.70e+04 | 4.55e-01 |
Response [3/3]
Q4: What is the usage of Lemma 1?
A lemma is a mathematical result that can be used to support a theorem or provide theoretical motivation for a method [9]. In our paper, Lemma 1 is adopted from [10] to provide theoretical motivation for the EI-based switching criterion.
W1: The font size in the tables is too small.
To improve readability, we have reduced the standard deviations from two decimal places to one and tightened the spacing between the means and standard deviations.
W2: More recent and challenging benchmarks should be included.
In this work, we considered DTLZ benchmark problems as they are widely used in the MOBO literature [2-7]. That said, taking the reviewer's suggestion, we have now included the bi-objective BBOB benchmark suite [11]. BBOB is designed to capture common challenges in real-world optimisation, including separability, conditioning, and multi-modality [12], and its functions have been shown to exhibit diverse landscape characteristics representative of real-world problems [13-15].
The results are reported in Table 2 of the manuscript (page 25). As can be seen, CTD achieves statistically better, equivalent, and worse HV than the competing methods in 313, 118, and 9 out of 440 pairwise comparisons, respectively. These results further demonstrate the effectiveness of CTD across a broader range of optimisation problems.
W4: Too much text on page 9, and experiments are not strong.
Page 9 of the original paper discusses the results of the sensitivity analysis and the alternative acquisition function, followed by the conclusion. The detailed results are provided in the appendix (table 2, page 25).
Our experimental evaluation includes comparisons with well-established methods under noiseless, noisy, and high-dimensional settings, sensitivity analyses, ablation studies, experiments with alternative acquisition functions, and analyses of switching times and distances to the Pareto front. Regarding the experimental performance, across all 1,158 pairwise comparisons with well-established methods (Section 6.1), CTD achieves statistically better, equivalent, and worse HV in 811 (70.0%), 306 (26.4%), and 41 (3.5%) cases, respectively.
References
[1] Quality evaluation of solution sets in multiobjective optimisation: A survey, ACM Computing Surveys, 2019.
[2] Efficient multiobjective optimization employing Gaussian processes, spectral sampling and a genetic algorithm, JOGO, 2018.
[3] Differentiable expected hypervolume improvement for parallel multi-objective Bayesian optimization, in NeurIPS, 2020
[4] Pareto set learning for expensive multi-objective optimization, in NeurIPS, 2022
[5] Unexpected improvements to expected improvement for Bayesian optimization, in NeurIPS, 2023
[6] qPOTS: Efficient batch multiobjective Bayesian optimization via Pareto optimal Thompson sampling, in AISTATS, 2025
[7] Multi-objective Bayesian optimization via adaptive -constraints decomposition, in ICML, 2026.
[8] Multi-objective and multi-fidelity Bayesian optimization of laser-plasma acceleration, Physical Review Research, 2023.
[9] Handbook of Writing for the Mathematical Sciences, SIAM, 1998.
[10] Regret for expected improvement over the best-observed value and stopping condition, In ACML, 2017.
[11] Using well-understood single-objective functions in multiobjective black-box optimization test suites, Evolutionary Computation, 2022.
[12] Real-parameter black-box optimization benchmarking 2009: Noiseless functions definitions, INRIA Report, 2009.
[13] BBOB instance analysis: Landscape properties and algorithm performance across problem instances, in EvoStar, 2022.
[14] Transfer of multi-objectively tuned CMA-ES parameters to a vehicle dynamics problem, in EMO, 2023.
[15] Pareto landscape: Visualising the landscape of multi-objective optimisation problems, in PPSN, 2024.
Response [1/3]
Q1: How are the reference points selected in the HV testing?
For the HV testing, we follow a commonly used setting in multi-objective optimisation [1]. Specifically, we collect all solutions obtained by all methods across all independent runs and set the reference point by extending the nadir point by 10% of the range between the ideal and nadir points in each objective.
We have clarified this in the "Performance Indicator" subsection of Section 5 in the revised manuscript.
Q2&W3: No real-world problems.
We would like to respectfully clarify that our experiments included six real-world optimisation problems: four-bar truss design, pressure vessel design, hatch cover design, vehicle safety design, car side-impact design, and laser plasma acceleration. These problems are widely used in the MOBO literature and cover different application domains [2-7]. For example, laser plasma acceleration [8] involves optimising three objectives, such as electron beam energy and charge, and represents a practical application of multi-objective optimisation in accelerator physics. We appreciate the reviewer's suggestion and would also be happy to consider including additional real-world problems to further strengthen the empirical evaluation.