Optimizing the Bias–Uncertainty Trade-off with Partially Ordered Bias Bounds

We study the problem of aggregating multiple estimators to optimize the bias–uncertainty trade-off, especially when latent bias bounds are known only through their ordering and estimator uncertainties are represented by intervals. This problem naturally arises in hyperparameter tuning when candidate estimators cannot be directly validated, as in off-policy evaluation and pointwise nonparametric function estimation. However, existing methods cannot straightforwardly handle multidimensional hyperparameters for general estimator families because the associated bias bounds are only partially ordered. To address this issue, we formalize the aggregation of partially ordered intervals by minimizing the worst-case oracle-approximation ratio, leading to a novel aggregation rule called Optimal Aggregation of Confidence Intervals (OACIS). Our main contribution is a comprehensive theoretical analysis of this minimax problem, including the boundedness of the oracle ratio, uniqueness of the solution when this ratio is finite, stability, statistical error guarantees, and a linear-time algorithm. We also evaluate OACIS in synthetic and real-data experiments.