State-Adaptive Multi-Scale Search for Scalable Bayesian Optimization

Bayesian optimization (BO) is attractive for expensive black-box problems, but exact Gaussian-process (GP) inference scales as $O(n^3)$ in the number of observations and high-dimensional search introduces a separate statistical and optimization challenge. These pressures are acute in robotics and other costly tuning settings: reusable hyperparameter histories may initially be small because each trial is expensive, yet a sequential campaign can rapidly accumulate hundreds or thousands of observations. We introduce Adaptive Multi-Scale Dirichlet-Guided Variational Gaussian Process (\method), a scalable sparse-GP BO method that combines variational inference with state-adaptive spatial allocation. Beyond the usual incumbent-centered exploitation and uncertainty-driven exploration signals, a Dirichlet-process mixture summarizes promising occupied modes and provides an additional data-driven search center. Across six synthetic benchmarks spanning 2–500 dimensions and 500–5000 total evaluations, achieves the best mean endpoint result among the three directly compared methods on every task, reducing final regret relative to adaptive-depth FocalBO by $48.2\%$ on average. It also obtains the best final objective on 11-D HalfCheetah and the 32-D FrankaBell manipulation task. Algorithm analysis shows that the controller uses all four search actions and that its high-dimensional gains persist throughout the optimization trajectory rather than appearing only at the endpoint. Recorded wall-clock cost across the eight retained tasks is about $60.4\%$ lower than adaptive-depth FocalBO, while an acquisition study shows robust performance under TS, UCB, and EI.