Bandits with Multiple Optimal Arms: Minimax Regret and Non-Adaptivity

We study multi-armed bandits (MAB) with multiple optimal arms, motivated by the fact that many practical decision making problems admit multiple correct answers. For $K$-armed bandits with $A$ optimal arms, we first provide a sharper analysis of previous sub-sampling algorithms (De Heide et al., 2021; Zhu & Nowak, 2020), establishing a $\tilde{O}\left(\frac{K-A}{\sqrt{KA}}\sqrt{T}\right)$ minimax regret, where $T$ is the total number of interactions and $\tilde{O}(\cdot )$ drops all constant and logarithmic factors, improving the previous $\tilde{O}(\sqrt{KT/A})$ regret. We then provide a matching lower bound up to logarithmic factors, indicating that our established rate is nearly minimax optimal. We further show that the knowledge of $A$ up to $\tilde{O}(1)$ factors is necessary to achieve near optimal regret, as near optimal algorithms for one number of optimal arms must incur substantially larger regret than optimal regret for a smaller number. Overall, our results provide a comprehensive minimax characterization of $K$-armed bandits with $A$ over the entire range of $1\le A\le K-1$.