- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents an approach towards improving sample efficiency of Bayesian optimization (BO) by restricting the search space to inputs that are plausibly optimal given an approximate simulator. Concretely, the search space is implicitly defined through optimal solutions of an inverse problem associated with a simulation under a particular set of task-specific parameters. To avoid repeated solution of the inverse problem, a neural net is trained to predict the solutions in its forward pass, optimized in expectation w.r.t. a prior distribution over simulation parameters. Theoretically, regret bounds are provided that depend only on the dimensionality of the resulting amortized search space, rather than the ambient dimension, along with error terms from amortization and misspecification. Results are provided on synthetic test functions, and a case study in personalized pharmacokinetics/pharmacodynamics, under various feedback modalities.
Critical strengths
- Speeding up BO in high-dimensional domains is a relevant problem, and exploiting prior information encoded in simulators is an interesting approach. I find the considered setting quite relevant, and the PK/PD case study interesting as well.
Critical weaknesses
- The analysis is rather simplified; in particular it assumes away the statistical problem of amortization (i.e., that the amortized network is trained only in expectation over a known prior), and rather hides it by assuming the error of amortization is uniformly bounded by \epsilon_am. This assumption seems unnecessarily strong -- in practice I would expect the approach really only requires an accurate estimate of the inverse map near the true / best approximating parameters.
- While Appendix D shows a substantial reduction of wall-clock computation time from ~52 to <1s per patient, overall this still seems like a small gain compared to likely comparatively slow feedback ("function evaluation") times in this application... More generally, the question of when the overhead of amortization pays off seems not sufficiently explored.
Questions for authors
- What is the computational cost of amortization in the PK/PD application? How does it relate to the per-patient optimization (cf Appendix D)? Did you explore tradeoffs regarding amortization (i.e., when in terms of number of offline samples used to train the inverse map vs sample acquisitions/evaluations of the learnt inverse map does amortization pay off within acceptable error)?
- How is the prior over the simulation parameters chosen? What is the effect of its misspecification (i.e., placing little mass on the “true” parameters)?
- \Delta_{model} is only specified in the appendix?
2 comments
Thank you for the clarification. I will keep my score.
We agree that the assumption of uniform boundedness of the amortization error assumption is stronger than the proof requires. We revised Proposition 2 and Appendix B.3 so that the bound depends on the approximation error at a parameter whose exact simulator solution maximizes the true utility over the simulator-induced search space.
In practical applications, the constraint is often on the online amortized optimization time. Table S2 shows that relying on direct optimization implies acquisition times that can exceed 6h, against 3s when using amortization. However, we agree that in our PK/PD application, the slow feedback makes the acquisition time less relevant. Therefore, our focus is rather on the better performance in terms of regret.
We computed the offline training time in our PK/PD application, keeping the same training configuration, and found it to be about 2.2 hours (5.3 min per epoch). We now report this value in the paper (Appendix D, section "Training cost").
Appendix F compares amortized optimization with direct L-BFGS-B optimization. The mean time to produce a regimen was 0.116 ms for a single network forward pass and 11.715 s for direct optimization (please note that we updated these numbers to exclude the one-off JIT compilation of the ODE solver).
\qquad T_{\mathrm{direct}}(n) = n\\,t_{\mathrm{direct}}.$$ where $T_{\mathrm{offline}}$ is the offline training cost. So amortization saves compute when: $$n > \frac{T_{\mathrm{offline}}} {t_{\mathrm{direct}}-t_{\mathrm{am}}}, \qquad t_{\mathrm{direct}}>t_{\mathrm{am}}.$$ In our PK/PD experiment, $T_{\mathrm{offline}} \approx 2.2 \text{ hours}$, so $T_{\mathrm{offline}} /({t_{\mathrm{direct}}-t_{\mathrm{am}}}) < 1$. This means that the extremely high cost of optimizing the acquisition function when using direct optimization of the dosing regimen implies that amortization is always beneficial. > How is the prior over the simulation parameters chosen? We construct $p(\phi)$ from the published population PK model of Senek et al. and PD model of Westin et al. Their fixed effects and random effects specify typical parameter values and inter-individual variability. We sample independently from the reported marginals because the joint distribution is unavailable, then apply the plausibility filter in Appendix E (Eq. 37) to obtain the training sample. The search space used during Bayesian optimization, $\Phi$, spans the $0.025$ to $0.975$ quantiles of the prior marginals. > What is the effect of its misspecification (i.e., placing little mass on the “true” parameters)? A prior that assigns little mass to a relevant parameter region can affect BOASS in two ways. Sparse training coverage can increase the approximation error of $\mathcal E_w$ in that region. A shifted or overly concentrated prior can also move the quantile bounds so that the online search excludes parameter values that generate good regimens. The effect on attainable utility depends on the resulting image $\mathcal M=\mathcal E_w(\Phi)$: $$\Delta_{\mathcal M} =\max_{x\in\mathcal D} U(x) - \max_{\phi\in\Phi} U\bigl(\mathcal E_w(\phi)\bigr).$$ Previously, our misspecification experiments varied the target regimen to study the effect of an optimum lying outside the learned search space. We added a new set of experiments in Appendix G (Out-of-distribution patients), which answers the reviewer's question by applying the BOASS method to a population of patients whith a PK/PD parameters distribution that differs from the prior used when training the amortized optimizer (the PK/PD model inverse map). These experiments show the robustness of our method to prior misspecification, and Appendix G provides interpretation for this robustness. > $\Delta_{model}$ is only specified in the appendix? We corrected this issue in the new version. We put the definition of $\Delta_{\mathrm{model}}$ from Appendix B.3, Eq. (19) in Proposition 2, Eq. (4).The time it takes to perform one acquisition varies across iterations. Table S2 reports values that are about 3 seconds when using amortized optimization (tam), against 6 hours (tdirect) for direct optimization.
Let's denote n the number of BO iterations. The compute costs for whole BO runs with n iterations are: