acceptodds
Withdrawn from ICLR 2027

What a Cheap Simulator Buys in Amortized Inference

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Amortized simulation-based inference needs many simulations, so practitioners train on a cheaper approximate simulator and spend a limited budget of exact simulations where it seems most useful. Reported savings, not measured error, justify both choices. We measure the error, in stochastic gene-expression models whose exact posteriors are available. Three findings follow. First, mixing surrogate and exact simulations in one training objective shifts the estimator's target, with a posterior-mean bias that grows as in the surrogate share (). At every budget we measure, down to exact simulations, the mixture carries more bias and wider posteriors than training on the exact simulations alone and is never better calibrated, and a two-stage scheme that keeps the fidelities apart only ties the exact-only estimator. Second, placing the exact budget by burstiness is a weak lever. Even for a count surrogate whose error is concentrated, it improves on random placement by to in distance to nominal coverage, while reversing the same ordering costs . Third, a classifier between the two simulators, with no neural posterior estimator and no reference posterior, reproduces the ordering of the trained estimators over six surrogate-system pairs and separates missing support from wrong shape. Matched controls trace the corruption to a continuum standing in for a discrete low-copy state, and the account carries over to a three-state promoter, a stochastic epidemic and public single-cell data. In our systems the approximation often costs more than the exact solver it replaces.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
simulation-based inference, neural posterior estimation, amortized inference, model misspecification, multifidelity inference, surrogate simulators, calibration, stochastic gene expression

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_mQQ3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether approximate simulations help train a neural posterior estimator when only a limited number of exact simulations are available. It compares exact-only training, mixed training, and surrogate pretraining followed by exact-data fine-tuning. The authors find that mixed training introduces bias, while two-stage training provides little improvement over exact-only training in their experiments. The paper also examines Fano-based allocation of exact simulations and proposes a classifier-based diagnostic for assessing surrogate quality before training an NPE. Experiments primarily focus on stochastic gene-expression models, with additional evaluations on SIR and single-cell data.

Critical strengths
  1. The matched simulator controls help identify the sources of inference error. The comparison between LNA and NB isolates the effect of distributional shape, while the comparison between CLE and the hybrid highlights the importance of preserving discrete promoter dynamics. These experiments provide useful insights into why certain approximations lead to poor posterior estimates.

  2. The paper compares exact-only, mixed, and two-stage training under the same exact-simulation budget. This provides a controlled evaluation of whether surrogate simulations offer benefits beyond using the available exact simulations alone. The use of multiple evaluation metrics, including coverage, bias, and SBC, further strengthens the empirical analysis.

Critical weaknesses
  1. The main experiments do not sufficiently test the regime where surrogate simulations would be most valuable. In Section 4.2, exact-only training already achieves near-nominal coverage 0.91 for konk_{on} with only 160 simulations. Therefore, the finding that surrogate pretraining provides no improvement may reflect the relative simplicity of the benchmark rather than a general limitation of multifidelity NPE. This concern is further compounded by Table 4, where several exact simulators are faster than the CLE surrogate.

  2. Incorrect interpretation of classifier AUC as total variation distance. The diagnostic in Section 4.4 relies on the approximation $\mathrm{TV}\approx 2\mathrm{AUC}-1$, but this identity generally holds for optimal classification accuracy, not ROC AUC. While classifier AUC may still provide a useful ranking of surrogates, the reported discrepancy values cannot be interpreted as estimates of total variation distance without further justification. This undermines the quantitative interpretation of one of the paper's main contributions.

Questions for authors
  1. Can the authors evaluate whether two-stage training provides benefits in settings where the exact-only NPE is substantially undertrained or poorly calibrated due to a limited simulation budget? Such experiments would help clarify whether the negative results in Section 4.2 generalize beyond the relatively simple benchmark considered.

  2. Can the authors justify the approximation TV≈2AUC−1\mathrm{TV} \approx 2\mathrm{AUC}-1 in Section 4.4? Since this relationship generally holds for optimal classification accuracy rather than ROC AUC, could the authors provide a theoretically justified TV estimator and verify whether the surrogate ranking remains consistent?

Reviewer_iKekOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates the trade-off of using a cheap, approximate or low-resolution simulator in the context of simulation-based inference. A specific gene-expression model is used as a case study, where ground-truth posterior is accessible. It reports that mixing the approximate and exact simulator biases the inferred posterior given the same budget, informed placement of exact simulator runs is ineffective, and that a pre-SBI stage classifier stage can detect the mismatch between the approximate and exact simulators and is predictive of the eventual inference quality.

Critical strengths
  • The paper studies an important problem in the SBI setting: it is attractive to use a cheaper surrogate model to produce rough simulations for pretraining the density estimator, before targeted selection of high-quality of simulations, such that inference quality should improve under similar compute budgets.
  • It proposes some interesting ideas, especially the pre-SBI check of comparing the approximate and high-fidelity simulators with a classifier, though the result is intuitive in that the worse the prior simulations are, the worse the inference result trained on those of course will be.
Critical weaknesses
  • the paper reads almost entirely and blatantly LLM-generated. The AI Use Statement discloses use for drafting and editing parts of the paper, and the structure and title of the paper, which is generous at best.
  • it's evaluated on a single simulator which, while there is access to ground truth posteriors, is not at all representative of the breadth of problems.
  • independent of the LLM usage, it is incomprehensible, especially the abstract (e.g., what is f?), but also everywhere else, to the extent that it is difficult to evaluate the idea and results of the paper. I have now spent more time on this review than the authors did on writing this paper, potentially on the entire project.
Questions for authors
  • are the authors actually able to describe, in their own words, the findings here?
  • do the conclusions hold for other simulator models?
Reviewer_DqGzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies how training neural posterior estimation (NPE) on an approximate ("surrogate") simulator affects the accuracy of the resulting posteriors. The main study uses a stochastic gene expression model with a tractable likelihood, so reference posteriors are available. Accuracy is assessed with a wide variety of metrics, such as interval coverage, posterior-mean bias, SBC, posterior contraction, C2ST and posterior leakage.

Critical strengths
  • The question is relevant in practice: surrogate simulators are often used to make SBI affordable.
  • The study employs a wide variety of calibration diagnostics.
Critical weaknesses
  • I found the paper hard to follow. For example: "The causal account that localises the corruption to a continuum standing in for a discrete low-copy state (Section 4.1) and the transfer to a second promoter, an epidemic and public single-cell data (Section 4.5) are instrument and scope rather than contribution". Several more examples like this exist.
  • The contributions are not clear.
  • The analysis rests on a single model only. It is unclear why this model was chosen and whether the conclusions generalize.
Questions for authors
  • Why this model in particular? A tractable likelihood does not distinguish it, since SBI benchmarks are generally chosen so that reference posteriors are available.
  • Could the authors explain the sentence quoted above, and state their main contribution in one or two sentences?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.