acceptodds
Under review as a conference paper at ICLR 2027

The Degeneracy Distillery

Abstract

Models are usually parameterized by scientific convention, not according to the combinations of parameters that the data can actually distinguish. When observations depend on only a few combinations of many parameters, simulation-based inference is unnecessarily high-dimensional: neural estimators spend expensive simulator calls learning redundant directions. Even at modest dimensionality, nonlinear degeneracies can make inference highly simulation-intensive and obscure the quantities actually being measured. We present the degeneracy distillery, which automatically discovers these combinations and expresses them in closed form, simultaneously reducing the effective parameter dimension and the simulation budget required for downstream inference. From parameter–simulation pairs alone, without likelihood evaluations, simulator gradients, or an observed data realization, the method learns an information-aware coordinate map across the prior and distills it into compact symbolic expressions. A joint objective determines the number of identifiable coordinates while driving the transformed information metric toward the identity. The resulting coordinates are both low-dimensional and interpretable: they reveal how the original parameters combine to control the data and restrict downstream inference to the directions the data actually constrain. Across scientific applications, the method recovers known structure from simulations alone: from noisy epidemic trajectories and chirp mass from gravitational-wave inspirals. For full inspiral–merger–ringdown waveforms, it instead discovers total mass and an asymmetric mass combination that yields better-conditioned and better-calibrated inference than conventional chirp-mass coordinates. In controlled scaling experiments, the method recovers rank-1 and rank-2 structure embedded in parameter spaces with up to 32 dimensions. Neural posterior estimation in the discovered coordinates tracks an oracle given the true parameterization, while inference in the original parameters degrades with ambient dimension. By removing redundant parameter directions before density estimation, the distilled coordinates substantially reduce the simulation budget required for simulation-based inference: – fewer simulations at matched validation log-probability and approximately fewer at matched CRPS.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
simulation-based inference, representation learning, information geometry, symbolic regression, parameter identifiability, scientific machine learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_MNz7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

In order to make simulation-based inference more efficient, the paper proposes a pipeline for finding parameter combinations that more clearly reflect what actually changes the data. To do so, they first use simulated parameter–output pairs to estimate which parameter changes make a noticeable difference to the outputs, measured by Fisher matrices. They then use this estimate to train a neural network which maps the original parameters to new coordinates that bring the Fisher matrix closer to the identity. Finally, they use symbolic regression to search for simple equations mapping the original parameters to the new coordinates, balancing how well each equation matches the network with how complex it is. Importantly, this does not require new simulations calls. They validate this approach using both synthetic and real-world scientific applications. In the synthetic examples, they find the pipeline generally recovers the expected parameter structure. For the scientific applications, they find it recovers recognizable parameter combinations as well as hidden useful alternatives.

Critical strengths

One strength of the paper is the interpretability of the symbolic regression step. In contrast to other neural representations, the fact that the learned coordinates are converted into interpretable formulas gives this method the ability of both understanding the model and using the coordinates in later inference.

Another strong contribution from the paper is the finding that in the scientific simulator examples the method can both recover recognizable relationships as well as find new hidden relationships. This is consequential because it gives evidence that the method can produce actual scientifically meaningful results outside of the simpler synthetic tests.

Critical weaknesses

One weakness in the paper is that the demonstrated scale of the method is limited. They show dimension reduction up to 32 parameters however that was a controlled example with only a small number of relevant parameters and may not demonstrate that the full symbolic pipeline actually scales to large parameter spaces. The authors acknowledge this however it still is significant that the method may not be able to scale.

Additionally, some of the efficiency gains are ambiguous with regards to how simulations are re-used. The NPE comparisons demonstrate improved sample efficiency however the paper does not consistently report the total simulation cost. This makes the end-to-end efficiency benefit harder to assess and is significant because efficiency gains are a main contribution to the proposed method.

Questions for authors

Could the authors clarify how exactly simulations are compared? Specifically, for SIR it is not clear to me whether simulations were re-used or whether the simulations used pre-inference are factored in to the comparison. A per-experiment count of unique simulator calls would make the efficiency comparisons easier to interpret.

4 comments

We thank Reviewer MNz7 for a thoughtful review and for recognising the value of interpretable symbolic coordinates and scientifically meaningful parameter combinations. We also appreciate the requests for clearer simulation accounting and a sharper distinction between our demonstrated scaling results and broader scalability.

1. Simulator-call accounting.

We agree that the relevant cost is the number of unique simulations used by the complete learning procedure. Discovery-training and discovery-validation simulations must both be counted, and simulations reused for downstream inference must be counted only once.

Our accounting distinguishes the following discovery datasets:

ExperimentDiscovery simulationsEqual simulator cost from
Hidden banana, d ≤ 322000, the NPE set itselfEvery point shown
Heater, d ≤ 121000, the NPE set itselfEvery point shown
Rosenbrock500 (250 training + 250 held-out)N = 500
SIR, original experiment1000 (500 training + 500 validation)N = 1000
SIR, reduced-budget experiment500 (450 training + 50 validation; see below)N = 500
GW (TaylorF2, IMRPhenomD)1000 (500 + 500)N = 1000
Weak lensing1000N = 1000

We will make these counts and the reuse of simulations explicit in the experiment descriptions and figure captions. Independent evaluation simulations will be listed separately; any simulations used for training or model selection will be included in the learning cost.

The total simulation cost includes discovery and NPE training, with shared simulations counted only once. Points below the discovery budget therefore compare NPE training budgets rather than total simulator calls, and we will label them accordingly.

For Rosenbrock, restricting the existing curves to NPE training budgets of at least 500 gives approximately 4-fold and 9-fold differences in the training budget required to reach matched validation log-probability, at eta-training budgets of 500 and 1,000 respectively. We will distinguish these curve-based comparisons from the fully accounted simulator-call totals. The 14-fold endpoint occurs at an NPE training budget of 100, below the discovery budget, and will not be described as an end-to-end saving.

2. Additional SIR check at a smaller discovery budget.

The original SIR discovery experiment used 1,000 simulations including validation. We separately tested discovery using 500 simulations in total, split into 450 training and 50 validation, with observation noise 0.05, across three random seeds.

A symbolic R0R_0-type coordinate was recovered in all three runs with different random seeds, with gradient cosine approximately 0.99 against the R0R_0 gradient. This demonstrates recovery at the smaller total budget.

3. Scaling the method.

Prompted by your comment, we tested input screening with the two-parameter Rosenbrock structure embedded among d=100 input parameters. Using 2,000 simulations per trial (1,700 for training and 300 held out for scoring), we ranked the original inputs using their sensitivity under the one-step map of Eq. (6): for each input, the score is the square root of the corresponding diagonal entry of the held-out average of JᵀJ; equivalently, the root-mean-square norm of the corresponding Jacobian column over the 300 held-out parameter points. Scores are compared against ambient dimensions held at the prior.

In each of ten trials, each with a different randomly chosen pair of active inputs, an independent set of simulations and a different network initialisation, the two active inputs ranked first and second among all 100. Successive reductions from 100 to 25, then 7, then 5 inputs retained both active inputs in every trial, bringing the number of retained inputs within the range tested in Fig. 2A.

This extends the evidence for identifying relevant inputs in a controlled sparse problem. It also provides a practical route to reducing the number of variables presented to the subsequent coordinate-learning and symbolic-regression stages. We have not yet completed those downstream stages for this new experiment, so it is not an end-to-end demonstration of the full pipeline at 100 parameters.

The existing hidden-banana experiment demonstrates coordinate discovery and downstream inference up to 32 ambient parameters (Fig. 2A; Appendix F.1). We will distinguish these results explicitly: successful input screening at 100 parameters, and the previously demonstrated discovery and inference pipeline at 32. Symbolic search complexity still depends on the retained input variables and expression complexity, as well as the number of latent coordinates.

Dear Reviewer MNz7, as the discussion closes shortly, we would be grateful to know whether the per-experiment simulation counts and the input-screening experiment address your concerns, or whether anything remains unclear that we can answer in the time left.

Reviewer_MNz7OpenReview ↗

Response

Hello, thank you for the detailed response to my review.

The ambiguity of simulator usage has been addressed which fully removes that weakness from the paper. As far as scalability, the new experiment does add some evidence for a better ability to handle larger sizes however it does not run the whole pipeline so concerns over end-to-end scaling still remain. The changes do improve the paper however not substantially and so I am keeping my recommendation of weak acceptance.

Thank you for your reply.

We have run the full pipeline (screening, neural coordinate learning, symbolic regression, downstream posterior inference) for three of the d=100 trials, each with 2,000 simulations. All three recover the two hidden Rosenbrock coordinates symbolically. Downstream NPE posterior log density in the discovered coordinates is 2.92, in line with the d=32 result (2.86) and close to that of the true coordinates (2.95). As anticipated with increased dimensionality, log density degrades when performing NPE in the 100 raw coordinates (2.21), which is worse than what raw θ\theta scored at d=32 (2.61). This means our discovery scaling plot (Fig. 2A) can be extended to d=100 to demonstrate the scalability of the method. We will include this end-to-end d=100 experiment in the camera-ready version.

Reviewer_SvrpOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper deals with simulation based inference (SBI). We imagine there is a simulator that can map parameter vector θ to observation vector x. Given an observed x_obs, we wish to find the θ so that, when it's input to the simulator, the output is as close as possible to x_obs. This paper addresses the situation where the problem is non-identifiable -- that is, there are multiple θ that all yield equally good approximations to x_obs.

To address this, the authors propose mapping θ to a new coordinate system in which the Fisher information matrix is the identity matrix, and then using symbolic regression to make the new coordinate system interpretable.

Critical strengths

Non-identifiability in simulation-based inference is a real challenge and too little work has addressed this; this paper is addressing an important problem.

The authors conduct experiments reflecting a broad range of application domains, and the use of calibration tests is good.

Critical weaknesses

EDIT: during the rebuttal, the authors offered a more precise explanation of a major point of confusion. While some of my concerns are still unanswered, the explanation helped and I am increasing my rating.

Most challengingly, this approach is only viable in very low-dimensional settings where the posterior p(θ|x) is unimodal. The need to estimate the Fisher information matrix presents a major challenge to higher-dimensional settings. Recall that the point of using neural networks within an SBI context is to ensure you can get good estimates of θ with few runs of the simulator. But if estimating the Fisher information matrix requires many different (x,θ) pairs, each of which requires running the simulator, then you wouldn't come out ahead.

Imagine x ∈ R and x = θ_1 + θ_2. Given x, there are many (θ_1,θ_2) pairs that fit exactly. The Fisher information matrix is rank deficient. Mapping to a new coordinate system will not change this. Later the authors say they are actually mapping to a latent low dimensional space. (However, the experiments often have the original and new dimensions the same, so it's not clear how this addresses the non-identifiability issue.) The low-dimensional mapping is written η(θ), and they need to match this to some function of the data \hat η(x), which is jointly learned with η. Ultimately, this looks like a variant of contrastive learning, which has been used previously within a simulation-based inference setting.

Why is reparameterization necessary if they are learning a posterior like in neural posterior estimation or embed and emulate? Doesn't the posterior address the non-identifiability?

Overall, the writing is very unclear. The Fisher information is a function of x instead of θ. Notation like \partial_a is never defined. Is θ^a supposed to be the a-th coordinate of θ? It's not defined. The connection to related literature is not clearly articulated. The experimental results are difficult to parse. For example, in Figure 5 we see posteriors in both θ space and η space overlapping, but these are entirely different domains. They say the fact that the η-space posteriors are rounder is a sign of success, but I don't understand why. What's wrong with an elongated posterior if it's correct? It's also not clear what "conventional coordinates" are and why they are different from θ.

Questions for authors

None

4 comments

Thank you for your review. We would like to clarify three points central to the assessment.

1. Rank preservation does not prevent learning lower-dimensional identifiable combinations.

We agree that an invertible reparameterization cannot increase Fisher rank. Indeed, Section 6, Limitation (iii), explicitly states that “an exactly singular F cannot be resolved by an invertible square map”. This is why Eq. (6) permits η:Rd→Rm\eta:\mathbb{R}^d\to\mathbb{R}^m with m<dm < d.

Your example illustrates this distinction. With fixed Gaussian observation noise,

x=θ1+θ2+ϵ,ϵ∼N(0,σ2),x=\theta_1+\theta_2+\epsilon,\qquad \epsilon\sim\mathcal N(0,\sigma^2),

the original Fisher matrix is

\begin{pmatrix} 1 & 1 \\\\ 1 & 1 \end{pmatrix}.$$ It remains rank one. However, the identifiable coordinate $\eta=(\theta_1+\theta_2)/\sigma$ parameterizes the same observation distributions and has scalar Fisher information $F_\eta=1$. The null direction is removed, not made identifiable. Eq. (6) admits such dimension-reducing maps, so the objection concerning invertible square maps does not apply to this formulation. Our heater experiment provides an analogous rank-one example, with observations depending on a product rather than a sum. It has an exactly rank-one Fisher matrix (Section 5.2, Eqs. 7–8; Appendix F.5), while the hidden Rosenbrock experiment addresses rank-two structure embedded in up to 32 dimensions (Fig. 2A; Appendix F.1). Experiments retaining the original dimensionality instead investigate improved conditioning. These are separate settings, and we will make the distinction clearer in our revised manuscript. **2. The learned, data-dependent matrix is distinguished from theoretical Fisher information.** Section 3 defines the likelihood Fisher information as a function of $\theta$. The network output $F_\phi(x)$ in Section 4 is a learned information proxy. For each training pair $(\theta_i,x_i)$, the network supplies a data-dependent information proxy $F_\phi(x_i)$, which is used when learning the parameter-coordinate map in Step 2. As explicitly stated there and proved in Appendix A, its population optimum is inverse posterior covariance. Its identification with likelihood Fisher information requires the stated asymptotic conditions. Its dependence on the data $x$ is therefore not a redefinition of theoretical Fisher information. Furthermore, Eq. (6) bypasses separate Fisher estimation entirely. **3. The dimensionality-reduction comparisons reuse discovery simulations for inference.** In Figs. 2A and 3 (right), the comparisons use shared simulation datasets, with discovery simulations reused downstream (Appendices F.1 and F.5). These experiments use Eq. (6). At the matched simulation budgets, inference in discovered coordinates tracks the oracle (for $d \ge 6$ in the heater, and up to $d = 32$ in the hidden Rosenbrock), while raw-parameter inference deteriorates with ambient dimension. A posterior can correctly represent non-identifiability. The question we ask is how efficiently it can be learned at a finite simulation budget, and whether the constrained combinations can be extracted symbolically. Roundness alone is not evidence of accuracy. Better-conditioned $\eta$ coordinates can facilitate efficient density estimation at finite simulation budgets. We assess the downstream benefits through inference accuracy and calibration. Regarding notation, $\theta^a$ denotes the $a$-th component of the parameter vector, and $\partial_a\equiv\partial/\partial\theta^a$. Repeated upper and lower indices are summed under the Einstein convention. We will make these definitions explicit in the revised manuscript. Three smaller points. In Fig. 5 the three estimators are trained in different coordinates, and their samples are mapped through the closed-form transformations into a common space for each panel, which is why they can be overlaid. “Conventional coordinates” are chirp mass and mass ratio $(\mathcal{M}_c,q)$, defined in the first gravitational-wave paragraph of Section 5.2. On the relation to contrastive learning: Eq. (6) has an alignment term but no negative pairs; constant collapse is penalised by the Jacobian volume term, and we will discuss this relation in the related work. Could you clarify whether these distinctions resolve the corresponding soundness concerns?
Reviewer_SvrpOpenReview ↗

Clarifications

I apologize that my original comments were not better articulated.

Yes, I recognize that a mapping to a new, lower-dimensional parameter space could make the Fisher information matrix full rank. This was not my concern. Rather, the issue is this. In my toy example, the Fisher matrix is rank-deficient. In Section 4.1 of the paper, the authors say the optimization objective (3) is proportional to - log det F(x), where F(x) is a model that approximates the Fisher information in theta space. First, if F(x) does accurately approximate the rank-deficient Fisher information, this term in the objective would be problematic. It seems that optimizing this objective must result in a learned F(x) that does NOT in fact approximate the rank-deficient Fisher information, right? Second, I still do not understand why one would learn a model to estimate the Fisher information as a function of x when the Fisher information is defined as a function of theta and the expected value over different x's (see line below eqn (1)). The response from the reviewers says that F(x) is an "information proxy" and not the same as the Fisher matrix except under some asymptotic conditions, but the paper says explicitly "[estimate of the] Fisher matrix, F(x)." The authors seem to perhaps think I'm concerned about population vs. sample properties, but in fact my concern is more basic than that -- both the actual Fisher information and any estimate of it should be functions of the same space (i.e., theta, not x). Perhaps I'm just missing something very basic here, but Section 4.1, which does not depend on the latent low-dimensional parameter estimation \eta, is quite unclear to me. Since this section sets the foundation for the method, I think this issue is critical.

I think Figure 5 is also still confusing. I now understand the authors are mapping from eta space to theta space, and it makes sense that roundness in eta space makes sense, but I still don't understand why roundness in the theta space is an indication of success. I thought the authors' stated goal was to learn the correct posterior in theta space, but to be more sample-efficient by working in eta space. Based on this, I would have expected to see the different posteriors in theta space to look more similar, at least for large numbers of training samples.

The basic idea of learning a lower-dimensional parameter representation for SBI is solid. It has been explored previously in the SBI literature (cf https://arxiv.org/abs/2409.18402, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6623978, https://arxiv.org/abs/2609.37381), and has strong connections to ideas such as active subspaces. It would be helpful to understand the contributions of this submission with respect to those ideas.

Thank you for clarifying. We agree that describing the network output simply as “the Fisher matrix” in Step 1 is imprecise. We will distinguish its exact statistical target from its interpretation as a likelihood-Fisher approximation more explicitly.

1. What Eq. (3) learns, and why its log-determinant is well defined.

Yes, you are right: the optimum of Eq. (3) is not the rank-deficient likelihood Fisher. Eq. (3) is a Gaussian negative log-likelihood, including both the residual quadratic term and the log-determinant. Under the assumptions in Appendix A, its population optimum is

Fφ∗(x)=Cov⁡(θ∣x)−1.F_\varphi^*(x)=\operatorname{Cov}(\theta\mid x)^{-1}.

This result is also stated immediately after Eq. (4). The distinction from likelihood Fisher information therefore persists at the population optimum. It is not merely a finite-sample estimation error.

For a concrete example, consider your sum model with Gaussian observation noise of variance σ2\sigma^2 and a Gaussian prior θ∼N(0,τ2I)\theta\sim\mathcal N(0,\tau^2 I), with both variances positive. Writing a=(1,1)⊤a=(1,1)^\top, we obtain

\qquad F_\varphi^*(x)=\tau^{-2}I+\sigma^{-2}aa^\top.$$ The likelihood Fisher is rank one, while the posterior precision is positive definite and has a finite log-determinant. Precision along the likelihood's null direction comes entirely from the prior since the data have not made that direction identifiable. Thus, the population target is posterior precision, which can be positive definite even when the likelihood Fisher is singular. Eq. (3) evaluates the log-determinant of the network's positive-definite output. **2. Why the network is written as a function of the data.** Eq. (3) writes the network output as $F(x)$. We write it here as $F_\varphi(x)$ to make explicit that it is the matrix-valued output of a neural network with weights $\varphi$ whose input is the data $x$, and we will adopt this notation in the paper. The output is a precision matrix for the parameters $\theta$. Its dependence on $x$ reflects its conditional-posterior target, whereas the expected likelihood Fisher is a function of $\theta$. We condition the network on $x$ because its target is the conditional posterior precision, learned from parameter–simulation pairs. In Step 2, each prediction $F_\varphi(x_i)$ is paired with its generating parameter $\theta_i$, and the coordinate map is learned by minimising the flattening loss over these pairs. Each $F_\varphi(x_i)$ is a noisy, single-realisation estimate of the precision associated with its generating $\theta_i$, and the loss averages over many such pairs, so the coordinate map it yields is a function of $\theta$ alone. The learned posterior precision can be identified with the likelihood Fisher only approximately, and only in the regular, identifiable, data-dominated regime described in Appendix A. This identification does not apply to unidentifiable directions. We will revise Step 1 to call the learned output a “posterior precision proxy” and state the conditions for its Fisher interpretation explicitly there. Finally, the dimensionality-reduction experiments in Fig. 2A and Fig. 3 use Eq. (6), which bypasses separate Fisher estimation. Its determinant is that of the latent-space matrix $JJ^\top$, rather than the singular ambient likelihood Fisher. We will make this separation between the two routes clearer. Could you clarify whether this addresses your remaining soundness concerns about Eq. (3)?
Reviewer_SvrpOpenReview ↗

Thank you

Thank you for this explanation; it helps quite a bit!

Reviewer_dThGOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

this paper presents degeneracy distillation, which aims to tackle the problem of parameter degeneracy in scientific models that can effect downstream simulation based inference. The authors propose to learn parameter transformations with neural networks using learned approximations of the Fisher Information, and then gain interpretable insight with symbolic regression on the parameter transformation. several symbolic and real-world scientific experiments are shown.

Critical strengths

generally I think this is a strong paper and it tackles and important problem in simulation based inference and model reduction, and makes a reasonable contribution to the field. the method is clearly described and components of the pipeline are explored and tested. To my knowledge, this approach of neural estimated Fisher information with flattening with learned transformations and symbolic interpretation is new. The experiments are useful and interesting. The authors give a clear and useful assessment of the limitations.

Critical weaknesses

the paper would improve with:

  • where possible, can you compare against a finite difference approximation of the Fisher Matrix as a baseline in scientific applications.

  • discovery is counted as one time costs, and not accounted for in the simulation budget. but this should be accounted for in overall simulation budget to give more clear picture of simulation cost.

Questions for authors

same points as weaknesses: the paper would improve with:

  • where possible, can you compare against a finite difference approximation of the Fisher Matrix as a baseline in scientific applications.

  • discovery is counted as one time costs, and not accounted for in the simulation budget. but this should be accounted for in overall simulation budget to give more clear picture of simulation cost.

2 comments

We thank Reviewer dThG for a constructive and technically informed review, and for recognising the methodological contribution, clarity of presentation, and value of the scientific experiments. We have run finite-difference (FD) comparisons at your suggestion and are making the simulation accounting explicit.

1. Finite-difference Fisher comparison.

We evaluated derivatives of the simulator mean by central differences. For a Gaussian observation model with parameter-independent covariance CC, the Fisher matrix is

Fab=∂aμ⊤C−1∂bμ.F_{ab}=\partial_a\mu^\top C^{-1}\partial_b\mu.

This applies to the TaylorF2 GW observation model. For SIR, we use the same mean-response expression with an effective covariance incorporating variability in the unobserved initial count. Since marginalising that count need not yield a Gaussian likelihood with parameter-independent covariance, we treat this as an approximate information diagnostic, rather than the exact Fisher matrix of the marginal likelihood.

We compare with the Fishnet ensemble mean at 500 held-out parameter points across the prior. The results are medians:

ProblemFrobenius cosineStiff-direction cosineTrace ratio: Fishnet / benchmark
SIR0.9960.9980.68
GW TaylorF20.9991.0000.00003
  • SIR (observation noise 0.05). The shape and dominant direction agree closely under these metrics, while the median trace ratio is 0.68. Both stiff directions follow the R0R_0 gradient. At noise 0.01, shape agreement remains similar and the median trace ratio is 0.44.

  • TaylorF2. Directional agreement is strong, but the absolute information scale is not recovered. The FD benchmark implies local Gaussian widths near one thousandth of the prior width. The Fishnet output is a learned posterior precision proxy, and here it does not recover the absolute likelihood-information scale. Nevertheless, the discovered coordinates reduce the FD-based loss of Eq. (5) by approximately 7-fold relative to the original mass coordinates and 8-fold relative to the conventional chirp-mass/mass-ratio coordinates. We will report the normalisation and absolute residual losses alongside these reductions, and state the amplitude-recovery limitation explicitly in Section 4 and the appendix.

  • Simulator cost. Our FD protocol required 12 simulator evaluations per parameter point, giving 6,000 perturbed-parameter evaluations for the 500-point comparison in each problem. The Fishnet estimator was trained on 500 prior-drawn simulations. These additional FD evaluations can be prohibitive when individual simulations are expensive. The comparison highlights the practical benefit of learning an information proxy from parameter–data pairs, alongside the limitations in absolute Fisher-scale recovery identified above. We will document the FD setup in the revised manuscript’s appendix.

2. Discovery cost in the simulation budget.

We agree that the discovery budget should be made more explicit in the main text. Discovery-training and discovery-validation simulations must both be counted, with reused simulations counted only once. At downstream budgets large enough to include the complete discovery dataset, those simulations are reused as NPE training data.

For the original SIR run, discovery used 500 training plus 500 validation simulations, so its discovery cost is 1,000 rather than 500. We also tested a separate 500-total-simulation configuration, which recovered an R0R_0-type coordinate successfully across three seeds. The per-experiment breakdown is discussed in our response to Reviewer MNz7.

For Rosenbrock, discovery uses 500 simulations in total. Restricting the learning-curve comparison to NPE training budgets at or above 500 gives approximately 4-fold and 9-fold differences in the budget required to reach matched validation log-probability, at η\eta-training budgets of 500 and 1,000 respectively. We will distinguish these metric-specific comparisons from complete simulator-call totals. Points in the figures below the discovery budget will be labelled as downstream-training-budget comparisons.

Thank you for the replies and for answering my questions. I think these additions will strengthen the paper. I will increase my score accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.