- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper analyses when mechanistic priors in multi armed bandits can improve decision making. The authors expose the fact that there is no offline criterion which can certify incorporating such prior models will help decision making, and therefore propose a criterion which they call the mechanistic information (the mutual information between the optimal and recommended policy). The paper proceeds by proposing a (supposedly) offline-computable pre-trial model certificate. Finally, the paper analyzes low sample regimes and derives bounds, as well as the asymptotic regime. Their approach is validated on synthetic data.
Critical strengths
- The paper aims to fill a gap that is highly relevant. I truly believe that the proposed research gap is highly valuable.
- The paper is self-contained, introduces the necessary notation, assumptions, and provides all the proofs (I did not check them in detail, but they appear to be sound).
Critical weaknesses
-
The title and motivation are somewhat misleading. The paper is called "the value of mechanistic priors in sequential decision making", and the motivation and introduction mirror this motivation. However, the work the proceeds by only covering the static bandit setting. There is nothing wrong with proposing a new theory for the easier bandit setting, but the paper should be titled and framed accordingly.
-
The proposed certificate assumes oracle information, and is therefore generally not usable in practice. This is my main concern: Deriving an as the mutual information between optimal and proposed policy is fine, but building on this as the fundation for everything that follows (e.g., ) is problematic. The main motivation is to propose a certificate that is computable offline and can guide the deployment of mechanistic priors. However, this leads to a large mismatch between theory and practice. There is no way to obtain these quantities from a mechanistic prior in general, which makes the proposed approach, generally, inapplicable in practice.
-
This connects directly to the second weakness: the proposed method is only validated on purely synthetic data (likely because the optimal policy, and therefore the quantities and can be obtained in closed form. This is, unfortunately, insufficient for validation, and strengthens my second concern above.
-
The proposed theory is a basic adaptation of existing work on quantifying regret and information tradeoff (Shufaro et al., 2025) and Bayesian regret upper bounds for Thompson sampling (Russo & Van Roy, 2016). To be fair with the authors, the second one is acknowledged by the authors. Still, this limits the novelty of the proposed theory.
Shufaro et al., 2025. On Bits and Bandits: Quantifying the Regret-Information Trade-off. In ICLR, 2025.
Russo & Van Roy. An Information-Theoretic Analysis of Thompson Sampling In JMLR, 2016.
Questions for authors
-
Can the authors explain how the relevant quantities can be computed when there is no oracle information available?
-
Can the authors explain how their approach translates to sequential decision making?