Wall-Clock Complexity for Zeroth-Order Optimization with Tunable Oracle Fidelity

Zeroth-order (black-box) optimization is commonly applied when gradients are unavailable and objective evaluations rely on expensive simulations. In many such applications, the oracle fidelity is tunable: higher-accuracy queries reduce noise but incur higher computational costs. To capture this trade-off, we study an accuracy-aware wall-clock model where each query with fidelity $\delta$ has a cost $c(\delta )$, and we minimize the total time $T_{\mathrm{total}}\propto {\sum }_{k=1}^{N}c({\delta }_k)$, subject to a target accuracy constraint. We show how the choice of oracle type, noise model, and optimization scheme induces explicit wall-clock-optimal choices for the algorithmic parameters. For instance, we demonstrate that accelerated methods can be wall-clock inferior to non-accelerated schemes. Furthermore, we characterize the conditions under which a constant fidelity strategy is optimal in the Big-O sense. Our framework provides a unified methodology to translate convergence guarantees into practical fidelity and batching recommendations. We apply this framework to several optimization schemes under various noise settings, including Tsybakov and adversarial noise.