- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Distributional Bayesian Optimal Experimental Design (DistBOED), a distributional RL method for sequential BOED that models the full conditional distribution of the information gain (IG) rather than only its expectation (EIG). The authors evaluate DistBOED in three scenarios using three risk-sensitive objectives, showing that DistBOED with risk-sensitive objectives produces policies that reduce dispersion or improve lower-tail IG relative to mean-seeking policies.
Critical strengths
- The method is well-motivated: in sequential BOED, a policy that optimizes total EIG may favor designs with high expected information gain but also high risk. It is sensible to bias the policy towards more protection via risk-sensitive objectives over the IG distribution.
- Principled formulation from the risk-sensitive objectives to the score function.
Critical weaknesses
- The formulation assumes a finite design menu, limiting its scalability to continuous design spaces.
- The baseline is limited to only DAD: comparisons with baselines that could take risks into account would strengthen the claims.
Questions for authors
- [1] could possibly handle risk in sequential BED via explicit constraints and online planning. Could you discuss how DistBOED and the method differ and share in settings?
- At inference time, can the protection level of the learned policy be adjusted without retraining?
[1] Guo, Yujia, et al. "Constrained Bayesian Experimental Design via Online Planning." arXiv preprint arXiv:2605.26990 (2026).
We are grateful to the reviewer for the positive comments on the motivation and formulation and for the constructive feedback, which we address below.
Responses to questions
Q1. Comparison to other methods
Response. We agree that COPEx could possibly be extended to address risk in information outcomes. As presented, it optimizes expected information gain under design constraints, so a risk-sensitive extension could optimize or constrain a function of campaign information gain.
The difference is when the design optimization is performed. COPEx combines pretrained networks with online multi-step planning, while DistBOED learns a policy in advance and uses it to select designs from the observed state without solving a new design-optimization problem during the campaign.
Moving this optimization into training is particularly useful when many campaigns must be run or designs must be selected with little delay. This computational benefit is demonstrated by DAD [2] and iDAD [3], which report millisecond design selection at deployment.
COPEx would therefore offer more flexibility to accommodate changing constraints during a campaign, while DistBOED provides a reusable policy for a specified risk preference. DistBOED is particularly suited to repeated experiments in the same setting where rapid design selection matters.
Q2. Inference-time adjustments
Response. Not for a single trained policy in the current implementation. As discussed in Q1 above, DistBOED trains policies for specified risk preferences. Changing the protection level requires further policy optimization or selecting another pretrained policy; this would be a disadvantage compared to an online planning method like COPEx. The learned critic can still evaluate alternative risk preferences under the existing policy without retraining.
Additional weaknesses not covered above
W1. Finite design menu
Response. We agree that the current implementation is limited to finite design menus. The risk-sensitive objectives can also be defined for continuous designs, but extending DistBOED would require a continuous-action policy, a critic accepting continuous designs, and an alternative to explicit summation over candidate designs in the actor update. We acknowledge the finite design menu in the manuscript’s limitations and leave the continuous extension to future work.
W2. Baselines
Response. A central contribution of this work is a unified framework for optimizing risk-sensitive objectives over campaign information gain. Methods such as DAD and COPEx do not directly optimize these objectives, so constructing comparisons would require extending existing methods.
We have nevertheless added a scalar actor-critic baseline that optimizes the same risk-sensitive objectives (Appendix F). This baseline achieves broadly comparable performance and provides a direct comparison with the distributional approach. We have also evaluated DAD with an explicit destruction penalty. The destruction penalty addresses a different concern, avoiding a physical event, but allows us to examine how targeting a physical source of information loss compares with targeting information risk directly. We discuss these results in more detail in the response to reviewer
rPYW.[2] Foster, A., Ivanova, D. R., Malik, I., and Rainforth, T. (2021). Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design. Proceedings of the 38th International Conference on Machine Learning, 139:3384–3395.
[3] Ivanova, D. R., Foster, A., Kleinegesse, S., Gutmann, M. U., and Rainforth, T. (2021). Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoods. Advances in Neural Information Processing Systems, 34:25785–25798.