acceptodds
Under review as a conference paper at ICLR 2027

Sample-Size Extrapolation in Simulation-Based Inference via Statistical Scaling

Abstract

In simulation-based inference (SBI), neural posterior estimators trained on a bounded range of sample sizes can become inaccurate on larger datasets as their parameterisations do not specify how posterior location, uncertainty, and shape must change as observations accumulate. We propose center-estimated NPE (CE-NPE), which combines a statistical center with an explicit contraction rate and learns a local center correction and residual densities in posterior-scale coordinates. The center is an estimate, such as a closed-form statistic or a simulator-based moment match, which does not require an evaluable likelihood. Our analysis explains how contraction amplifies center error, decomposes Gaussian posterior error into shape, mean, and covariance contributions, and gives a conditional transfer bound that retains training-boundary error. Across regular benchmarks, CE-NPE improves reference-posterior agreement over the tested free-conditioning baselines at five times the largest training size, without retraining. Matched controls and nonregular examples characterize the scope of these gains and identify the conditions on which they depend: an appropriate contraction rate, an accurate center, and a stable residual representation.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Simulation-based inference, Neural posterior estimation, Amortized inference, Sample-size extrapolation, Posterior contraction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1HMNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper tries to make a neural posterior estimator trained on small datasets work on larger datasets without retraining. Its method, center-estimated NPE (CE-NPE), combines a statistical center with an explicit contraction factor, leaving the network to learn a local center correction and residual density. For a larger dataset, the center is recomputed and the same network is reused with the prescribed scaling.

Critical strengths
  1. The paper studies a useful problem. A network that accepts datasets of different sizes does not necessarily produce an accurate posterior when the sample size exceeds the training range. The paper makes this distinction clear.

  2. The method has a clear statistical motivation. Using a statistical center and an explicit contraction factor reduces what the network needs to learn. The analysis also explains why getting the contraction rate right is not enough when the center is inaccurate.

  3. The paper includes some useful experiments. The comparisons between free conditioning, structural scaling, and CE-NPE, together with the fixed-center controls, provide some evidence about the roles of centering and scaling within the tested settings.

Critical weaknesses
  1. The additional value of the network is still unclear. The authors have already provided a way to learn a good center, and dedicated the contraction rate. This raises the question of whether a classical method (like Gaussian approximation) using the same center and model information could achieve comparable accuracy. The existing fixed-center controls do not answer this question, since they still use a neural network to learn covariance. A direct comparison would help establish what the learned correction and residual density actually add.

  2. The method relies on model-specific choices of the statistical center and contraction rate. Although the paper discusses these choices and their limitations, it remains unclear how to select and validate them. Even with a suitable statistical center and contraction rate, there is not theoretical guarqntee that the correction and residual density learned at small n remain accurate at larger n. Keeping center error bounded in posterior-standard-deviation units is not enough to guarantee an accurate posterior.

  3. The benefit of the additional computation is unclear. As the authors acknowledge, CE-NPE is partially amortized when center estimation requires numerical optimization. That is, numerical optimization must be redone for each new dataset to estimate the statistical center. In fact, existing literature already has fully amortized posterior sampling for high-dimensional observations (Kim et al. 2025). Note that their work also used a simultaneous summary statistics learning. Another example is Wildberger et al. (2023) which is an amortized flow matching-based method. Schmitt et al. (2024) proposed consistency model posterior estimation (CMPE), which enables fully amortized posterior sampling with a small number of network evaluations and no dataset-specific optimization. A comparison of posterior accuracy and total computational cost would help clarify whether CE-NPE’s additional optimization is worthwhile.

  • Kim et al. (2025), Deep Generative Quantile Bayes.

  • Wildberger et al. (2023), Flow Matching for Scalable Simulation-Based Inference.

  • Schmitt et al. (2024), Consistency Models for Scalable and Fast Simulation-Based Inference.

Questions for authors
  1. On a suitable benchmark, could you compare CE-NPE with a Gaussian approximation using the same statistical center and a covariance estimated from local curvature?

  2. For SIR or Lotka–Volterra, how do posterior accuracy and inference time change with the optimization budget for the statistical center, keeping the trained network fixed?

3 comments

W1 / Q1. What does the network add relative to a same-center classical Gaussian? We ran this comparison on all six benchmarks, with the covariance from the posterior Hessian at the center, from the Gauss–Newton (GN) curvature of the moment objective, or from a parametric bootstrap of the center. C2ST at n=20/100n=20/100, the largest training size and five times it (classical controls: three evaluation seeds):

TaskHessian (needs likelihood)GNBootstrapCE-GCE-NSF
Gaussian linear0.625/0.5410.663/0.5490.642/0.5810.513/0.5720.510/0.558
GL uniform0.616/0.5480.674/0.5600.650/0.5940.608/0.5740.593/0.562
Mixture0.502/0.498—0.554/0.5090.549/0.5430.515/0.511
Bernoulli GLM0.599/0.5260.807/0.6090.637/0.5760.666/0.6850.642/0.652
SIR0.546/0.5130.562/0.5170.553/0.5250.579/0.7720.559/0.740
Lotka–Volterra0.588/0.5330.596/0.5360.590/0.5420.713/0.7370.665/0.757

The classical Gaussians have four limitations that CE-NPE does not share:

  1. They need more than the simulator. The Hessian requires an evaluable likelihood, so it cannot be used in the likelihood-free setting that CE-NPE addresses; we report it only as a reference. GN requires the center to be a Gauss–Newton fit, so it does not exist for the mixture.
  2. They are unreliable at small nn. At n=10n=10 their joint 95% coverage falls to 0.12 (GN) and 0.81 (bootstrap), against 0.92–0.97 for CE-NPE on all six tasks, and within the training range each of them is less accurate than CE-NPE on both Gaussian-linear tasks.
  3. Without a likelihood they are slow. The bootstrap refits the center on 200 simulated datasets per query, about 30 and 400 times slower than CE-G on SIR and LV.
  4. They are Gaussian. In the nonregular example T1 (Appendix E), a Gaussian at the same center places half of its mass outside the posterior support, so a classifier using the support alone already reaches C2ST 0.75, against 0.59 for CE-NSF at n=100n=100.

Where no likelihood is available, the classical option is the method-of-moments Gaussian at the same center, with covariance (J⊤nΣ−1J)−1(J^\top n\Sigma^{-1}J)^{-1} from the center's moment curves and the sample covariance Σ\Sigma of the observations. We ran it on the held-out simulated population-genetics data of Appendix G (400 datasets per size, generating parameters known, no bootstrap). It is less accurate than CE-NPE at every size (training sizes 5–20):

nnNLL at the generating parameter, MoM GaussianNLL, CE-GJoint 95% coverage, MoM GaussianJoint 95% coverage, CE-G
103.90-0.090.660.95
201.45-0.730.790.96
500.11-1.410.860.94
100-0.35-1.820.830.87

Their strength is the large-nn regime of regular models. At five times the training maximum they are close to the ideal, as the Bernstein–von Mises theorem predicts: CE-NSF is within 0.02 of the best of them on GL, GL uniform and the mixture, and behind on the GLM, SIR and LV (0.65–0.76 against 0.51–0.61). We will add them to the main comparison as this large-nn reference and state when CE-NPE is the better choice

W2. How are the center and rate selected and validated, and what guarantees remain at larger nn?

Selection. See the general comment G1, G2. Our response to Reviewer TF4Z, W5.

Guarantees. Appendix B states what holds once the network is fixed, and its results order the Gaussian estimators by KL divergence to the true posterior as nn grows. Proposition 3 splits KL(Pn∥Qn)\mathrm{KL}(P_n\Vert Q_n) exactly into a Gaussian-shape term and mean and covariance terms, each bounded above and below by the errors of the fitted mean and covariance in the local coordinates. By Proposition 1, a learned correction that stays bounded keeps these errors bounded when it acts on the posterior scale, as in CE-G, so the KL of CE-G beyond the shape term stays bounded. A learned center whose error in parameter units does not shrink at rate n−αn^{-\alpha}, which nothing in Free-G or Scale-G enforces, makes the mean term grow as n2αn^{2\alpha}, and a fitted width that does not contract at the true rate makes the covariance term grow; their KL diverges. This is a partial ordering: CE-G below Free-G and Scale-G, with Free-G and Scale-G not ordered against each other. Proposition 2 gives how center and rate errors grow with the extrapolation factor. We will state the ordering as a corollary in Appendix B.

Validation. Accuracy at a new size reduces to how small the network's local errors remain there, which can be checked at the deployment size with no reference posterior and no retraining: simulate a few hundred datasets of that size and compute the coverage of the generating parameters (or SBC) together with their held-out log density. We will add this procedure to the paper.

W3. How does partial amortization compare with fully amortized alternatives in accuracy and cost? CE-NPE is partially amortized only through the center: for a new dataset, the only computation besides the network is cstatc_{\mathrm{stat}}, which needs no simulator calls and no retraining and takes milliseconds on four tasks and about one second on SIR and LV (general comment, G3). DQB, FMPE and CMPE are fully amortized over datasets, but none of them addresses sample sizes beyond those seen in training. To apply them here, a set encoder and a size input must be added, which makes them free size-conditioned estimators. We will discuss them in related work. Within the response period we implemented FMPE in this way (same encoder, log⁡n\log n input and dataset budget as our baselines, three seeds); we did not run DQB or CMPE.

Accuracy. FMPE fits the training range only on the two Gaussian-linear tasks (0.564 and 0.687 at n=20n=20), and there it degrades to 0.940 and 0.942 at n=100n=100, against 0.558 and 0.562 for CE-NSF. On the other four tasks it is above 0.92 already at n=20n=20, so we draw no conclusion from them.

Cost. Sampling is cheaper than with FMPE, which integrates an ODE for every sample: for a batch of 400 queries CE-G is faster than FMPE on every benchmark.

Is the additional optimization worthwhile? Among learned estimators, yes: every variant with free size conditioning that we tried (other size encodings, best-effort tuning, the center as an input, 15 times the training, FMPE) is at 0.83–1.00 at n=100n=100, against 0.51–0.77 for CE-NPE. The comparison with a same-center Gaussian is under W1.

Q2. How do SIR and LV accuracy and runtime change with center-optimization budget, keeping the network fixed? With the trained networks fixed (three seeds), we recomputed the center at n=100n=100 with increasing budgets. Trajectories are deterministic ODE solves and the time is per query; the last column is the median posterior-mean error of CE-G in reference SDs. This sweep used a separate training run, so its values differ slightly from those elsewhere.

TaskGrid, GN steps, starts (LV only)TrajectoriesTimeC2ST, CE-G / CE-NSFMean error (SDs)
SIR11211^2, 01220.18 s0.999 / 1.00060.0
11211^2, 21380.53 s0.887 / 0.8743.77
21221^2, 24580.53 s0.811 / 0.8010.84
41241^2, 21,6980.53 s0.785 / 0.7570.47
41241^2, 4 (default)1,7140.89 s0.783 / 0.7540.46
81281^2, 66,6101.24 s0.783 / 0.7540.45
LV747^4, 0, 12,4020.31 s1.000 / 1.00082.8
747^4, 2, 12,4260.93 s0.944 / 0.9551.95
949^4, 2, 46,6610.93 s0.788 / 0.8170.28
13413^4, 2, 828,7610.92 s0.727 / 0.7560.18
13413^4, 4, 8 (default)28,9531.53 s0.718 / 0.7500.17
17417^4, 6, 1684,6892.15 s0.718 / 0.7490.17

Accuracy saturates at the default budgets: larger budgets change C2ST by at most 0.001. Smaller budgets trade accuracy for time; on SIR, two Gauss–Newton steps instead of four save 40% of the time at a cost of 0.002–0.003. Runtime is set by the number of Gauss–Newton steps, since the grid is evaluated in one batched solve. The appendix states three refinement steps for SIR; the code and all reported results use four, and we will correct the text.

I appreciate the authors for the response and additional experiments. The center optimization budget experiment largely addresses my question on this point.

That being said, the benefit over classical approximations part and the scope of the theoretical claims remain insufficient to me. I therefore maintain my rating for now.

Reviewer_TF4ZOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper suggests adapting a center estimation of simulation-based inference with learning or manually designed statistics in the domain-specific case. This method may extends the utilization of neural posterior estimation by creating a variable z_n to incorporate the sample size scaling and mean shift. However, the method itself is limited to the standard Gaussian normalization approach, which could be a common practice in the field.

Critical strengths

I do not see much strength in this paper. Maybe, a lesson that we need to rescale the confidence after receiving more data...

Critical weaknesses

The choice of c_stat seems to be very important in CE-G. The c_stat is the source of information in the mean shift when the extrapolation occurs. However, the choice of c_stat is a simple line of "In our benchmarks, c_stat ranges from .... moment matching (Appendix C)" The impact of c_stat choice needs to be investigated by varying the choice as well as varying the scenario of N.

The real comparison is not done yet. The true competitor of this method is using the additional x observations to compute c_stat to recalculate \theta with conventional neural poterior estimation.

Also, the computation burden is not discussed. The training simulator budget is matched across estimators, but CE-NPE additionally requires simulator calls at test time to compute c_stat. If we intend to use a new method in neural posterior estimation, the call on the simulation needs to be also compared across the models. Under this case, CE-NPE, Structure Scaling, and Free-Conditioning can be compared.

I cannot see the valur of learning component in this paper. The neural part of this neural posterior estimation is not contributing much. The main component would be manually chosen scaling and statistics. Already, covariance modeling is done by previous neural posterior estimation models.

The ablations also show that the gains come mainly from the imposed rescaling rather than from center estimation, which the paper presents as its main idea. With the center held fixed, imposing n^{-1/2} alone reduces C2ST and KL in Table 16 and 17. The rescaling itself is textbook Bernstein–von Mises rather than a contribution of this paper. The paper should restate its contribution accordingly: the empirical evidence supports “an explicit contraction rate around any consistent statistic” rather than “center-estimated NPE”, and the separate effects of mean matching and rescaling should be reported consistently across all six benchmarks, not only on three.

I need to see the separation of improved estimation contributions: mean matching or rescaling. The rescaling can be effectively reduce the likelihood estimation error while the main focus of this paper is the mean estimation. The rescaling is not much introduced by this paper, rather it is coming from a textbook.

Figure 1 uses the notation of s, e, r, etc, which are defined at the introduction phase. This requires a short introduction on key variables and functions in the caption.

Questions for authors

see my weakness section

2 comments

We stand by the novelty and importance of our work. To our knowledge, the contraction rate of the posterior, one of its most basic structural properties, has not previously been built into amortized neural posterior estimators, although other structure, such as symmetry under transformations and physical constraints, routinely is and is known to improve the fit.

W1. The choice of statistical center. We varied the center as you suggest, at sizes 10 to 100; the table is in the general comment (G2). What decides accuracy is the error of the center on the posterior scale, and this decides the choice of estimator: the centers that fail are those whose error does not shrink with nn (a fixed bias, or an optimizer stopped too early), as Proposition 1 predicts, whereas the likelihood-free center is within about 0.02 of the exact-likelihood MAP on SIR and LV beyond the training range.

W2. Conventional NPE using cstatc_{\mathrm{stat}}. CE-NPE uses no additional observations: cstatc_{\mathrm{stat}} is computed from the same nn observations x1:nx_{1:n} that every estimator conditions on, so a conventional NPE has exactly the same information. We ran the comparison you ask for: the free estimators (conventional NPE) are given cstatc_{\mathrm{stat}} as an extra input appended to the encoder output (both heads, six tasks, three seeds). C2ST:

TaskGaussian head, Free → +cstat+c_{\mathrm{stat}}, n=10n=10Gaussian head, Free → +cstat+c_{\mathrm{stat}}, n=100n=100Gaussian head, CE, n=100n=100Spline head, Free → +cstat+c_{\mathrm{stat}}, n=10n=10Spline head, Free → +cstat+c_{\mathrm{stat}}, n=100n=100Spline head, CE, n=100n=100
GL0.571 → 0.5370.886 → 0.8530.5720.551 → 0.5280.870 → 0.8360.558
GL uniform0.837 → 0.6420.984 → 0.9380.5740.577 → 0.5360.919 → 0.8930.562
Mixture0.873 → 0.8660.951 → 0.9510.5430.868 → 0.6730.953 → 0.9880.511
GLM0.996 → 0.7310.999 → 0.9560.6850.976 → 0.7130.999 → 0.9470.652
SIR0.882 → 0.7110.990 → 0.9690.7720.744 → 0.6520.962 → 0.9560.740
LV0.993 → 0.9620.999 → 1.0000.7370.967 → 0.8710.998 → 0.9930.757

The statistic improves the fit in range, but at n=100n=100 these estimators remain at 0.84–1.00, against 0.51–0.77 for CE-NPE. As an input, the relation of cstatc_{\mathrm{stat}} to the posterior location and width must still be learned from sizes 5–20; CE-NPE imposes it. The gain therefore comes from imposing the centering and scaling, not from the information in cstatc_{\mathrm{stat}}.

W3. No additional simulator calls are required. CE-NPE makes no simulator calls at test time: the center is computed from the observed data alone, in milliseconds on four tasks and about one second per dataset on SIR and LV (general comment, G3).

In training the center is computed for every simulated dataset, which on SIR and LV makes a training step slower than for the free estimators. To check that this extra computation does not explain the results, we retrained Free-G, Free-NSF and Scale-G on SIR and LV for 60,000 steps, 15 times the standard budget and about the training time of CE-NPE on LV (half of it on SIR). C2ST (three seeds):

TaskEstimatorStepsn=10n=10n=20n=20n=50n=50n=100n=100
SIRFree-G60,0000.6250.6160.7640.910
Free-NSF60,0000.6180.6070.7480.871
Scale-G60,0000.6250.6130.7470.869
CE-G4,0000.6400.5790.6630.772
CE-NSF4,0000.6020.5590.6140.740
LVFree-G60,0000.8220.8510.9080.977
Free-NSF60,0000.8110.8370.8960.957
Scale-G60,0000.8050.8210.8590.939
CE-G4,0000.7380.7130.6710.737
CE-NSF4,0000.7070.6650.6550.757

With the longer training the three estimators come close to CE-NPE within the training range on SIR. Beyond it the gap narrows on SIR without closing and stays large on LV, so the longer training does not remove it.

W4. What does the neural component contribute? The center and the rate supply only a point and a scale. Everything else (the finite-sample correction of the location, the covariance and the shape of the posterior) is learned by the network from simulations. Three comparisons show that this matters.

Location. Without the learned correction (posterior mean fixed at the center), C2ST within the training range is worse on GL (0.676 against 0.507 at n=10n=10) and the GLM (0.766 against 0.704); on the other four tasks the center alone is already accurate.

Shape. The spline residual is not restricted to a Gaussian. In the nonregular example T1 (Appendix E), whose posterior has a hard boundary, CE-NSF reaches C2ST 0.59 at n=100n=100 with the exponent estimated from the training range, against 0.72 with a Gaussian residual.

Covariance. Earlier NPE models also learn a covariance, but not on the contraction scale. With the same statistical center, a freely learned scale gives 0.61–0.94 at n=100n=100, and the imposed scale 0.52–0.75.

The scaling itself is classical. What is new is that, among the variants we tested, only these centered, rescaled coordinates let a learned estimator transfer beyond its training range: neither the statistic as an input, nor 15 times the training, nor the imposed scale alone achieves this.

W5. Do the gains come from centering or rescaling, and does the separation hold on all six benchmarks? Both are needed, on all six. The table gives C2ST at n=100n=100. In the “learned center” rows the network outputs the posterior mean freely; in the “statistical center” rows the mean is fixed at cstatc_{\mathrm{stat}}, computed outside the network (the sample mean on GL and GL-unif), and only the covariance is learned. With the imposed scale, this is CE-G without the learned correction Δϕ\Delta_\phi.

EstimatorGLGL-unifMixtureGLMSIRLV
Learned center, free scale (Free-G)0.8860.9840.9510.9990.9900.999
Learned center, imposed scale (Scale-G)0.7150.7370.9161.0000.9850.998
Statistical center, free scale0.7280.7320.6140.8170.8970.940
Statistical center, imposed scale0.5420.5490.5170.7220.7500.704
CE-G0.5720.5740.5430.6850.7720.737
CE-NSF0.5580.5620.5110.6520.7400.757

Your reading of Tables 16 and 17 holds on all six tasks: at the statistical center, imposing the rate improves C2ST on every task (rows 3 and 4). But the rate alone is not enough: imposed on a learned center (rows 1 and 2), it helps only on the two Gaussian-linear tasks and changes the other four by at most 0.04. Replacing the learned center by the statistical one improves all six tasks at either scale (rows 1 and 3, rows 2 and 4). Both are needed.

We agree that the contribution statement should name the explicit contraction rate, and the table shows that it must name the center as well. We will restate it as: “CE-NPE imposes an explicit contraction rate around a statistical center that tracks the posterior at that rate, inside an amortized estimator, and learns the remaining location, covariance and shape in those coordinates; the scaling is the classical Bernstein–von Mises one.” The table above will be in the main text. We do not write “any consistent statistic”: the sample median is consistent, yet CE-G centered on it stays at C2ST 0.81–0.84 on GL.

W6. Can the Figure 1 caption define its notation? Yes. We will add to the caption: “Here sn=eϕ(x1:n)s_n=e_\phi(x_{1:n}) is the output of the permutation-invariant encoder eϕe_\phi, cnc_n is the center of the local coordinates zn=nα(θ−cn)z_n=n^{\alpha}(\theta-c_n), rϕr_\phi is the residual density of znz_n, cstatc_{\mathrm{stat}} is the statistical center, and Δϕ\Delta_\phi is the learned local correction.”

Reviewer_zUEmOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This work addresses the problem of learning an amortized neural posterior estimator in the setting where inference must be performed on a varying number of iid observations, including sample sizes beyond those seen during training. While standard approaches use permutation-invariant embeddings (e.g., deep sets), along with fixed features representing the number of observations, this work proposes combining a statistical center with an explicit posterior contraction rate. The estimator then learns a local center correction and a residual posterior density in centered, rescaled coordinates. The authors provide theoretical analysis supporting the proposed parameterization and evaluate the method across six adapted regular SBI benchmarks, showing how it performs at sample sizes within and beyond the training range. They also provide several ablations to further analyze the benchmark performance and examine the method’s scope and failure modes. and demonstrate the approach with a real-world genomics application.

Critical strengths
  • This work addresses a clear and practical problem for neural posterior estimation.
  • The proposed parameterization is well-founded, and the statistical motivation is clearly presented in the paper.
  • The empirical results show impressive improvements across benchmark tasks (Fig. 3), although the baselines themselves seem to underperform more than I would have expected within the training regime.
  • The authors provide a thorough set of ablations, including “best-effort” baselines and fixed-center controls. I appreciate the authors’ direct discussion of alternative explanations, scope, and failure modes.
Critical weaknesses
  • The free baselines perform poorly within the training regime on several tasks (Fig. 3). While the authors add “best-effort” ablations to improve their performance, the fact that these methods still underperform within the training regime is puzzling to me. This makes it harder to determine how much of the out-of-range advantage is specifically attributable to improved extrapolation. Further explanation of these baseline failures would strengthen the empirical conclusions.
  • The approach requires an accurate problem-specific statistical center and a suitable contraction rate. More practical guidance on obtaining these, including the computational cost of constructing the center, would improve the paper.
Questions for authors
  • Can you position this work with other recent approaches to this problem, e.g., https://arxiv.org/abs/2605.07972? An empirical comparison would also be useful, although I appreciate that this may be beyond the scope of the current work.
  • Can you report posterior NLL on held-out simulated datasets for the population-genetics application (Sect. 5.3), where the generating parameters are known?
  • In addition to the compositional comparison (Table 20, Appendix F.6), it would be useful to see how the method performs compared with NLE/NRE baselines.
  • Why choose a log n parameterization for the free baselines? Section 3.1 motivates this as a convenient numerical scale, but how sensitive are the baseline results to alternative representations? Appendix F.5 examines this question for the structured residual rather than the free baselines.
  • Can this work accommodate hierarchical nuisance structure, where observations are conditionally iid given both the target parameters and shared nuisance variables?
2 comments

W1. Why do the free baselines perform poorly within the training range? Because at n=5n=5–2020 the posterior is already far narrower than the prior, and a free estimator must locate it to within a fraction of that width from the raw observations; with the same training budget, the free network fails to learn this. At n=10n=10 the Free-NSF posterior width is 2.2 (Bernoulli GLM), 3.1 (mixture), 1.5 (SIR) and 2.9 (Lotka–Volterra, LV) times wider than the reference. New experiments confirm this reading: we gave the free estimators the statistical center as an extra input (both heads, six tasks; the C2ST table is in our response to Reviewer TF4Z\color{red}{\textsf{TF4Z}}, Q2). This reduces these ratios to 1.1, 2.1, 1.4 and 1.5 and improves the in-range fit, but beyond the training range they remain behind our method, CE-NPE: their C2ST at n=100n=100 is 0.84–1.00, against 0.51–0.77 for CE-NPE. The free estimators must learn from sizes 5–20 how the posterior width shrinks with nn; CE-NPE imposes it.

W2. Center and contraction rate selection, and cost? The answers are in the general comment: the rate in G1, the center and how candidates are compared in G2, and the cost in G3.

Q1. Relate to PAIRS (arXiv:2605.07972). PAIRS pretrains an encoder on sets of at most two observations and then finetunes the inference head at the target sample sizes; CE-NPE is never trained beyond its training range. We will add this distinction to related work.

We also ran a PAIRS-style estimator of our own implementation, since the authors have not released code (three seeds). With its head trained at the evaluation sizes, as in PAIRS, its C2ST at n=100n=100 is 0.612, 0.667 and 0.804 on GL, GL uniform and SIR, against 0.558, 0.562 and 0.740 for CE-NSF, which is trained only on sizes 5–20. On the mixture, the GLM and LV our implementation does not fit even its training range, so we draw no conclusion there.

Q2. Posterior NLL on held-out simulated population-genetics data. The table gives the negative log density at the generating parameter on the 400 held-out simulated datasets for each sample size in Appendix G (mean over three seeds):

nnFree-GFree-NSFScale-GCE-GCE-NSF
10-0.071-0.257-0.089-0.092-0.144
20-0.670-0.880-0.694-0.728-0.718
50-1.150-1.467-1.210-1.412-1.113
100-1.367-1.662-1.234-1.822-1.268

The estimators are trained on sizes 5–20. Free-NSF has the lowest NLL up to n=50n=50, and CE-G at n=100n=100.

Q3. NLE and NRE comparison. We trained likelihood and likelihood-ratio estimators on individual observation–parameter pairs, composed them over the nn observations, and sampled the posterior by SMC. C2ST at n=100n=100:

EstimatorGLGL-unifMixtureGLMSIRLV
NLE0.6610.7150.6140.9220.7620.983
NRE0.7940.9590.9641.0000.9791.000
CE-NSF0.5580.5620.5110.6520.7400.757

CE-NSF is lowest on all six tasks, by 0.10 or more on five; on SIR NLE is close. Within the training range NLE is competitive on GL, GL uniform and the mixture (0.561/0.594/0.557 at n=20n=20) and less accurate on the GLM, SIR and LV (0.778/0.641/0.939).

Q4. Why log⁡n\log n, and how sensitive? The posterior scale is a power of nn, so its logarithm is linear in log⁡n\log n. We also tested no size input, 1/n1/n, and a linear size input, for both heads. At n=100n=100 on all six tasks, log⁡n\log n is the best of the four or within 0.004 of the best.

Q5. Shared hierarchical nuisance variables. A shared nuisance parameter can be included in a jointly inferred vector when the joint model is regular, identified, and contracts at a known rate; the marginal for the target parameters then follows from the joint posterior. Weak identification or group-level variation would need different rates or parameterizations. We have not evaluated this setting and will state it as an extension.

I thank the authors for their detailed rebuttal. The additional evaluations have certainly strengthened the paper, and I appreciate the amount of work put into this rebuttal. I am keeping my score (weak accept).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank the three reviewers for their reviews. All three asked how the statistical center is chosen and what it costs, and Reviewers zUEm\color{red}{\textsf{zUEm}} and 1HMN\color{red}{\textsf{1HMN}} also asked how the contraction rate is chosen. We answer these points once here (G1–G3); the individual answers refer to them.

G1. Rate. Where standard asymptotics supply the rate, we use it: n−1/2n^{-1/2} for regular parametric models by the Bernstein–von Mises theorem (van der Vaart, 1998, Ch. 10), and the known rate in nonregular cases (e.g. n−1n^{-1} at a support boundary; Bochkina and Green, 2014). Where they do not, the rate is estimated from simulations at the training sizes, without a likelihood or reference posterior, from the log–log slope of the spread of the center (Appendix F.4).

G2. Center. The default is the method-of-moments estimate, which needs only the simulator: θ\theta is chosen so that the noise-free mean output of the simulator matches the mean of the data. On the two Gaussian-linear tasks this is the sample mean; on the GLM, SIR and LV it is solved by Gauss–Newton steps. Candidates are compared by simulation before training: nαn^{\alpha} times the error with respect to the generating parameters should not grow across the training sizes, and the candidate with the smallest error is used.

To show how the choice matters, we retrained CE-G with alternative and perturbed centers (three seeds). The table reports C2ST and the fitted posterior-mean error in reference SDs (median over 400 cases); the complete table will be in the appendix.

TaskCenterC2ST at n=10n=10C2ST at n=20n=20C2ST at n=50n=50C2ST at n=100n=100Posterior-mean error (SDs), n=20n=20Posterior-mean error (SDs), n=100n=100
GLsample mean (paper)0.5070.5130.5530.5720.050.12
+ non-contracting bias0.5080.5170.6860.8650.050.72
+ contracting noise0.9280.9130.9200.9210.790.79
median instead of mean0.8070.8190.8260.8390.480.53
Mixtureten EM steps (paper)0.6440.5490.5270.5430.050.05
one EM step0.6700.5830.5310.5340.110.11
EM initialization only0.7700.7590.7350.7460.800.83
GLMconverged Gauss–Newton (paper)0.7040.6660.6640.6850.130.12
two Gauss–Newton steps0.9120.9310.9660.9901.064.52
SIRlikelihood-free moments (paper)0.6400.5790.6630.7720.170.41
exact-likelihood MAP0.5800.5570.6420.7720.060.31
grid search only, no refinement0.9440.9620.9780.9876.716.9
LVlikelihood-free moments (paper)0.7380.7130.6710.7370.190.13
exact-likelihood MAP0.6660.6750.6600.7230.040.10

What decides accuracy is the error of the center on the posterior scale, and this decides the choice of estimator. The centers that fail are those whose error does not shrink with nn (a fixed bias, or an optimizer stopped too early), as Proposition 1 predicts. The exact-likelihood MAP is shown only as a reference, since SBI has no likelihood; beyond the training range the likelihood-free center is within about 0.02 of it on SIR and LV.

G3. Cost of the center. The center requires no simulator calls, in training or at test time, and no retraining for new data. On GL, GL uniform, the mixture and the GLM it is the sample mean or a few analytic iterations and costs milliseconds. On SIR and LV it is a Gauss–Newton fit of the deterministic mean ODE of the model, about one second per dataset, most of it on a grid shared across datasets; this per-dataset fit is what makes CE-NPE partially amortized there.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.