OPBO: Order-Preserving Bayesian Optimization

Bayesian optimization (BO) is widely used for optimizing expensive black-box problems under limited evaluation budgets. However, its performance often degrades as the problem dimension increases. Most high-dimensional BO methods address dimensionality while retaining value-based surrogate objectives. We argue that the surrogate learning objective itself may also contribute to this difficulty. We therefore study replacing value-based surrogate objectives with order-based objectives in high dimensions and characterize when order learning becomes preferable to value regression. Our analysis shows that the advantage of order learning increases when the numerical representation is harder to fit and when the dimension is large relative to the number of observations. These results motivate Order-Preserving Bayesian Optimization (OPBO), which replaces value regression with neural ranking surrogates. We compare value-based and order-based surrogates across 20 synthetic instances and four real-world tasks under Standard BO, TuRBO, and HeSBO. Under the same evaluation budget, ranking surrogates generally perform better, with larger gains at higher dimensions. The source code is available at https://anonymous.4open.science/r/OPBO-89B4/.