- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors decouple evaluations for MOBO and identify a subset of objectives and constraints to be evaluated every round. Their approach outperforms existing baselines across synthetic and real-world problems.
Critical strengths
- The problem is well-motivated, and the solution seems sensible and effective. Treating the constraints as part of the optimization procedure seems important.
- The empirical results also seem promising, and the method outperforms many existing baselines across a diverse set of problems, including synthetic tasks, real tasks, with and without constraints, etc.
Critical weaknesses
- The benefits of the proposed mechanism are not demonstrated clearly. The differences between qPOTS-DOE and the existing benchmarks, especially HVKG which the authors describe as the main competitor, are not explained in the related work and instead require detailed reading of the corresponding papers. In addition, many of the existing competitive methods are single-task GPs, and it would be useful to demonstrate the benefits of the MTGP approach explicitly by comparing it to existing MTGP methods.
- The benefits of the two-stage decoupling mechanism are also unclear, and it would be interesting to see the contributions of different components of this setup, such as the impact of including the constraints. It could also be useful to evaluate the benefits of the correlation metric more directly, perhaps on different synthetic datasets. For the benefits of the optimal subset, it may be interesting to compare the optimal subset algorithm with alternatives.
I'm open to updating my score if the reasons for the improved results are demonstrated more clearly.
Questions for authors
- Have you tried ablating the two stages of the algorithm? How robust is it to different hypers, like the number of objectives selected?
- Do qLogNEHVI baselines also use the MTGP?
Dear Reviewer nkzQ,
As the discussion phase comes to an end, please acknowledge the author's response and consider leaving them a comment on whether it has resolved some or all of your concerns.
Thanks!