acceptodds
Under review as a conference paper at ICLR 2027

TestBED: An Open-Source Toolkit for Bayesian Experimental Design

Abstract

Bayesian experimental design (BED) is a principled framework for intelligent data acquisition with applications across a wide range of scientific and industrial domains. However, the development and adoption of BED remains hindered by a lack of standardised software and benchmarks. Existing methods are typically implemented in bespoke codebases and evaluated on a small subset of tasks, making methods difficult to apply, compare, or extend. To address this, we present TestBED, a general-purpose, open-source toolkit for Bayesian experimental design. TestBED decomposes the BED process into modular, reusable components spanning tasks, objectives, training mechanisms, policy architectures, and evaluation procedures, together with scalable implementations of core primitives such as expected information gain estimators. We provide a wide range of these components, which can be freely composed, allowing existing methods to be reproduced within a common framework, while also enabling previously unexplored configurations and new methodologies to be easily implemented and evaluated. Using TestBED, we further conduct a large-scale benchmark of contemporary BED methods, including controlled studies of under-appreciated but surprisingly critical methodological choices, such as the importance of time embeddings in design policy architectures. By providing a modular, extensible software foundation and a reproducible benchmarking suite, we hope that TestBED will accelerate methodological research in BED and lower the barrier to its real-world application.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
bayesian experimental design, experimental design, data acquisition

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_KLJrOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The article presents TestBED, a modular and extensible software framework for developing and assessing Bayesian experimental design (BED) methods. It brings together the majority of current methods, and a suite a test tasks. Most importantly, it defines an interface between different components of BED methods that allows for modular extensibility. This will allow researchers in BED to have a set of test tasks, and standardised implementations of core components, with which to implement and experiment with new developments. In turn, this will enable much greater comparability between BED methods, in contrast with the current state of affairs (everybody develops everything from scratch, including the test tasks, so results are rarely like for like comparable).

The article demonstrates an immediate benefit of the TestBED in discovering clearly that using time as a feature in the design state improves performance in methods that were originally released without time dependence. Similar ablations/borrowing of ideas are likely to follow.

Critical strengths

The article is, in my opinion, a significant enabler of future research in BED

The article is very clear, and the software is already available to download

I would like to use this software suite in my own research!

Critical weaknesses

There isn't a large core "research" contribution here. However, I think that the contribution to future development of BED is likely to be significant

Questions for authors

None

1 comment

Thank you for your positive assessment of our work, and are encouraged to hear that you are likely to find TestBED useful in your own research. We are looking forward to seeing what TestBED will enable the BED community to achieve.

(W1) There isn't a large core "research" contribution here. However, I think that the contribution to future development of BED is likely to be significant

We appreciate the reviewer’s comment that TestBED is likely to contribute significantly to the development of future BED methodology. While the infrastructure of TestBED is the primary contribution of our work, we also highlight several scientific contributions that go beyond the framework itself:

  • TestBED provides a new, unifying abstraction of the BED pipeline into modular components (tasks, designers, objectives, and algorithms) which provides a common foundation for existing and future methods. These abstractions are what makes the components of TestBED interchangeable across diverse methods (Section 3).
  • We also derive several new methodologies not seen in the literature, namely, new REINFORCE-style estimators for ActionBED and JointVarPost that go beyond the original methods and enable their application on non-reparametrisable tasks (Appendix B).
  • Our benchmark (Section 5) highlights several new empirical findings, including the importance of time embeddings and architecture in policy-based methods, and the relative importance of high-accuracy EIG estimation in DAD-style approaches.

As noted by the reviewer, we see these findings as demonstrative of the kinds of future research TestBED will additionally enable.

Reviewer_J64vOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper presents TestBED, a PyTorch toolkit that decomposes Bayesian experimental design (BED) into tasks, designers, objectives, and algorithms, with rollout hooks for methods that modify the design loop. The authors implement several existing policy-based methods (DAD, RL-BOED, ALINE, ActionBED, JointVarPost, and static and random baselines) within this framework. They benchmark them on eight tasks under a shared Attention-based policy architecture and a 12-hour wall-clock budget, reporting sPCE and sNMC bounds on the EIG (L = 10⁹, N = 2,048) and downstream MSE and CMSE from a refitted action policy. Three controlled studies follow: an architecture ablation on location finding (Attention vs. DeepSets, with and without time embeddings; Table 1), an analysis of the number of contrastive samples L for DAD (Table 2), and a variational-family ablation for JointVarPost (Table 11). The paper also derives REINFORCE-style estimators for JointVarPost and ActionBED on non-reparameterisable likelihoods (Appendix B).

Critical strengths
  • The modular decomposition (Figure 1) is a clear and useful abstraction. Separating the design space, rollout hooks, objective, and fitting algorithm is well motivated, and Figures 3 and 4 show concisely how a new task or method plugs in.
  • Reproducibility is taken seriously. The paper provides an anonymised repository, fixed seeds, standardised recipes, and detailed hyperparameters in the appendix (Tables 3, 4, 8).
  • The benchmark is broad. Eight tasks cover static and dynamical systems, discrete and continuous outcomes, and a range of identifiability structures, which gives a more informative picture than most single-paper comparisons.
  • The architecture ablation (Table 1) is a useful contribution. It shows that time embeddings give large gains for both DAD and ActionBED, and that DAD and ActionBED prefer different architectures. This matters for practitioners.
  • The ALINE reproduction (Table 7) is a good verification step, and the explicit list of deviations from the original ALINE implementation (Appendix D.6) is transparent.
  • The estimator study (Table 2) gives a concrete recommendation: prioritise accurate EIG estimates over more gradient steps.
Critical weaknesses
  1. Hyperparameters are tuned to different degrees for different methods. Appendix D states that for most methods the authors “follow the original authors’ recommendations where available,” and that RL-BOED and ALINE had their design spaces modified. Some methods were therefore run at their published settings, while others were adjusted by the authors to work in this framework. The paper does not describe a common tuning budget, and it does not say whether any method received a hyperparameter search. A benchmark is only fair if each method receives comparable effort.

2.The downstream loss is refitted from scratch for every method, using a fixed MLP. Table 10 evaluates designs by fitting a small MLP to predict θ from the rollout, with a fixed architecture and a fixed training procedure. Since different designs produce histories with very different structure, the quality of this fitted decoder may dominate the measured downstream loss. The paper notes that ActionBED was refitted “for the sake of a fair comparison,” but the same concern applies to every design. A method whose histories are easier for an MLP to decode can score better even if its designs are less informative.

Questions for authors

same as weakness

2 comments

We thank the reviewer for their valuable feedback and positive evaluation of our work. We are encouraged to hear that they found the modular design of TestBED and our benchmark to be valuable contributions. We address the reviewer’s two concerns below, and we are happy to discuss any additional points that may arise during the discussion period.

(W1) Hyperparameters are tuned to different degrees for different methods. Appendix D states that for most methods the authors “follow the original authors’ recommendations where available,” and that RL-BOED and ALINE had their design spaces modified. Some methods were therefore run at their published settings, while others were adjusted by the authors to work in this framework. The paper does not describe a common tuning budget, and it does not say whether any method received a hyperparameter search. A benchmark is only fair if each method receives comparable effort.

Thank you for raising this point; we agree that fair benchmarking requires comparable effort across methods. Each method in TestBED was treated with the same protocol. No method received an exhaustive hyperparameter search, but no method was run with naive settings.

Hyperparameter protocol: We started from the hyperparameters recommended in the original papers, and made modest manual adjustments to ensure they were well suited to our settings. For example, we adjusted optimiser settings because our runs typically train for far longer than the original papers. Importantly, hyperparameters are generally fixed per method, rather than tuned per task, with only small task-specific changes when necessary. One example is that we found gradient clipping was helpful in stabilizing our new REINFORCE estimator for ActionBED on discrete-outcome tasks (Appendix B). Our results evidence that this protocol did not disadvantage any method. Our results generally match or exceed the published numbers where available (see Table 13 and Table 9).

We did not perform full hyperparameter sweeps because the compute required is prohibitive. Each (method, task, seed) run requires approximately 12 GPU hours, so even a small grid of 25 hyperparameter configurations across 5 seeds for all 8 tasks and 5 methods would require more than 60,000 GPU hours.

Design Spaces: The bounded design spaces for RL-BOED and ALINE were not a tuning choice. Both methods require bounded design spaces, which their original implementations already use. We therefore added bounds on tasks whose design spaces are otherwise unbounded (Location Finding and Temporal Discounting for both). The bounds only restrict which designs the policies can propose. The prior and likelihood are unchanged, so EIG is the same quantity for every method.

More broadly, compared with the current practice of evaluating methods in separate codebases with different tasks, architectures, and evaluations, TestBED holds all of these fixed and makes each hyperparameter explicit through a fully reproducible and transparent recipe, so that comparisons are like-for-like.

To make our protocol more explicit, we have added an additional discussion in Appendix D.

(W2) The downstream loss is refitted from scratch for every method, using a fixed MLP. Table 10 evaluates designs by fitting a small MLP to predict θ from the rollout, with a fixed architecture and a fixed training procedure. Since different designs produce histories with very different structure, the quality of this fitted decoder may dominate the measured downstream loss. The paper notes that ActionBED was refitted “for the sake of a fair comparison,” but the same concern applies to every design. A method whose histories are easier for an MLP to decode can score better even if its designs are less informative.

We would first like to note that our EIG evaluation (Figure 5, Table 13) does not use this MLP predictor. Hence, this prediction-based metric is an additional, complementary metric to the usual EIG evaluation, whose results should be taken in context with these EIG values.

Second, in many real-world BED problems, we are not interested merely in producing informative data, but instead in solving some concrete, downstream problem (Rossa et al., 2026). A natural downstream problem that is relevant to many tasks is to predict θ\theta from the gathered data, which is precisely what our setup aims to do. In fact, data from which it is hard to predict θ\theta would indeed constitute bad data.

While the exact performance of the predictions does depend on our specific setup (a small MLP predictor trained as described in Appendix C), we believe this to be a broadly sensible choice. In particular, using a shared architecture and training protocol across all tasks and methods provides a fair basis for comparison. We further note that an exhaustive selection of downstream predictors for each method and task would be computationally prohibitive.

That being said, other choices of predictors could also certainly be tried (e.g., a random forest). We would be happy to provide some preliminary results with an alternative prediction model if the reviewer would find this to be a useful addition.

We agree though that this point could be made more clear in the paper, and have added a new discussion at the end of Appendix C.

Reviewer_BTRWOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Benchmarks for BED have been scattered across the literature without a single repository of benchmarks. This paper introduces a library for Bayesian Experimental Design (BED). The BED workflow is decomposed to a set of primitives/modules: task, designer, objective, algorithm, and recipes.

The library includes implementations of

  • Benchmark tasks including location finding, stochastic pendulum, and others.
  • Methods such as RL-BOED, ActionBED, Deep Adaptive Design, and others.
  • Neural Network backends such as Deep Sets and Attention Networks.
  • Common estimators such as EIG.

The paper also presents an array of experimental results using the recipes on the benchmark datasets, including experiments with time embeddings.

Critical strengths
  • A library of test benchmarks with implementations of common methods and tasks while being flexible -- is a progress towards reproducibility and faster research. Implementations of common evaluation functions such as EIG, sNMC, etc is an added advantage towards better code reuse.
  • The implemented methods are compared on the benchmark tasks, showing that no method is a clear improvement over the other methods. It is also shown that using time embeddings leads to improved performance for some of the methods.
Critical weaknesses
  • While the paper seems to be complete and exhaustive with a lot of the details provided in the appendix, some of the important details are missing from the main paper. The paper mentions that there are 4 major components: task, designer, objective and algorithm. However, it is hard to relate/imagine how the implementations of the existing methods map to this framework.
  • The paper also mentions pytorch. For which of the components is it necessary to use pytorch and its auto-grad capabilities?
  • Specific examples of instantiations of the modules are missing from the paper making it hard to evaluate how interesting the contribution is.

Some missing literature on other paradigms in BED: Mehta et. al., “An Experimental Design Perspective on Model-Based Reinforcement Learning,” ICLR 2022

Questions for authors
  • Can you provide a slightly more detailed idea of what the individual module map to would look like for any example method?
  • How are the time embeddings added the models?
  • Can you provide some context or introduction about the implemented methods? Introducing the methods out of the blue without background can confuse readers.
3 comments

We thank the reviewer for their valuable, constructive feedback, and positive assessment of our work. We are glad to hear they found TestBED an important step towards more effective BED research. We respond to the reviewer’s specific concerns below, and are happy to provide additional details if needed.

(W1/Q1) it is hard to relate/imagine how the implementations of the existing methods map to this framework… Can you provide a slightly more detailed idea of what the individual module map to would look like for any example method?

(W3) Specific examples of instantiations of the modules are missing from the paper making it hard to evaluate how interesting the contribution is.

Thank you for these suggestions; we agree that additional examples would be a valuable addition to the paper. We have updated Figure 1 so that it is a complete, runnable implementation of DAD (Foster et al., 2021) in TestBED, color-coded by modules. This example shows how users can implement methods by directly using the modules implemented within TestBED, without having to modify the modules themselves. The Task is highlighted in blue, the Designer is green, the DesignObjective is orange, and the remaining algorithmic aspects are in gray. The full DAD implementation in TestBED has additional utilities, like model checkpointing, but this example shows the core method only requires a few lines of code.

To further illustrate how TestBED works in practice, we have added Appendix H. This contains a full example of implementing and using a new task within TestBED, as well as a runnable implementation of ActionBED (Rossa et al., 2026). In particular, the ActionBED example shows this method is implemented in a way that is highly similar to DAD, directly using modules already provided by TestBED.

We would be happy to provide any additional examples the reviewer would find helpful.

We also note that TestBED has a complete set of technical documentation, which includes further examples and details for the modules in TestBED. Please see the README in the anonymized code provided, which shows how you can launch this documentation locally.

(W2) The paper also mentions pytorch. For which of the components is it necessary to use pytorch and its auto-grad capabilities?

PyTorch is used as a general-purpose numerical engine within TestBED. In particular, any tensors used within TestBED are expected to be PyTorch tensors, which allows us to take advantage of the autograd engine within PyTorch when needed.

Autograd is used whenever designs or policies are optimised using gradient-based methods; this typically requires anything involving neural networks, sampling, densities, or calculations based on these to be done using PyTorch. For instance, a SequentialDesigner uses gradients to directly optimise its designs (which are PyTorch tensors), and a PolicyDesigner uses gradients to optimise its neural network policy (which is a PyTorch nn.Module).

Given that modern BED methods are increasingly policy-based, we expect that having this autograd engine within TestBED will be key in driving future methodological research. Crucially, PyTorch tensors also remain usable for non-policy-based and gradient-free approaches.

(W4) Some missing literature on other paradigms in BED: Mehta et. al., “An Experimental Design Perspective on Model-Based Reinforcement Learning,” ICLR 2022

Thank you for bringing this work to our attention. Mehta et al. (2022) use BED to decide which transitions to query in model-based reinforcement learning, and it is our aim that TestBED will be helpful in developing future work of this kind. We have added a reference to this in our introduction.

(Q2) How are the time embeddings added the models?

For both the Attention and DeepSets networks, time embeddings are handled via a learnable torch.nn.Embedding layer, i.e., each t∈{1,2,…,T}t \in \{1, 2, \dots, T \} gets a learned, fixed-dimensional representation e(t)e(t).

  • Attention: for a history of length tt, each input datapoint (xs,ys)(x_s, y_s) is first embedded through a small MLP hh and has the corresponding time embedding added, yielding zs=h(xs,ys)+esz_s = h(x_s, y_s) + e_s. The class token c0c_0 also has the corresponding timestep embedding added, i.e., ct=c0+e(t)c_t = c_0 + e(t). The resulting sequence (z1,z2,…,zt,ct)(z_1, z_2, \dots, z_t, c_t) is then passed through a standard transformer-style architecture. See Figure 8, Appendix E.
  • DeepSets: each input datapoint (xs,ys)(x_s, y_s) has its corresponding time embedding concatenated to yield zs=(xs,ys,es)z_s = (x_s, y_s, e_s) before being passed to a standard DeepSets-style architecture. See Figure 9, Appendix E.

Full architectural details are in Appendix E. We agree our description of how time embeddings are handled could have been clearer though, and have updated Appendix E accordingly.

(Q3) Can you provide some context or introduction about the implemented methods? Introducing the methods out of the blue without background can confuse readers.

We agree this would be helpful. While Appendix D already contains a brief description of each implemented method, we have also added a short description of each in Section 3 so that readers have more context before the experiments.

Thank you for the clarifications, I have increased my score

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.