- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This study investigates whether in-context reinforcement learning behaves as Bayesian posterior sampling. Using decision-pretrained transformers, it tests two necessary conditions, invariance to sufficient statistics and the martingale property, for Bayesian decision-making. In experiments across Gaussian, Bernoulli, and linear bandit problems, the former condition is violated in all models, while the latter is largely violated in the variant models as well.
Critical strengths
While many studies have suggested that ICL or ICLR is Bayesian in some sense, I believe that investigating what inference algorithms ICRL actually implements is significant.
Critical weaknesses
-
The overall framework of using a martingale test to demonstrate that ICL is non-Bayesian overlaps with Falck et al. So, I consider this to be an incremental contribution.
-
A more importantly, I think the GPT-2 used in this work is too small. For instance, studies on Bayesian ICL, such as Ma et al., Wakayama et al., and Xie et al., use significantly larger GPT-2 architectures and much larger pretraining datasets. Consequently, when this study claims that DPT is non-Bayesian, I suspect the underlying cause stems from incomplete pre-training (e.g., insufficient optimization, inadequate sample size, or limited model capacity) rather than an intrinsic limitation of the ICRL mechanism.
Falck et al. (2024). Is In-Context Learning in Large Language Models Bayesian? ICML2024.
Ma et al. (2026). Optimal In-context Adaptivity and Distributional Robustness of Transformers. arXiv:2510.23254.
Wakayama et al. (2025). In-Context Learning Is Provably Bayesian Inference. ICML 2026.
Xie et al. (2022). An Explanation of In-context Learning as Implicit Bayesian Inference. ICLR 2022.
Questions for authors
The first contribution bullet states that "We...derive two necessary conditions from it, namely sufficient-statistic invariance and the martingale property". However, aren't these conditions already well-established in the prior work? Specifically, does the former appear in Appendix C of Wakayama et al. as well as in classical Bayesian statistics literature? And is the latter a restatement of the arguments in Falck et al.?
If these do indeed represent distinct and novel contributions, please clarify your specific claims so that I will reconsider the informativeness score. Otherwise, explicitly clarifying that these are existing properties while respectfully referencing prior work would improve the soundness of the paper.