acceptodds
Withdrawn from ICLR 2027

Predators, Prey, and Sloppy Parameters: Fine-Tuning Amortized Inference for Generalized Lotka–Volterra

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Accurate parameters do not guarantee accurate trajectories in nonlinear ordinary differential equation inverse problems. We establish this distinction through a controlled generalized Lotka-Volterra benchmark with species, comparing amortized point estimators, neural posterior estimation, and test-time optimization on identical observations. Amortized regression achieves low parameter error but often produces inaccurate or invalid simulations. Fisher information and posterior diagnostics connect this mismatch to highly unequal sensitivity across parameter directions. We introduce optimizer-in-the-loop (OIL) tuning: a three-term objective trains a parameter initializer through finite trajectory-matching updates, and test-time trajectory matching refines its prediction. OIL improves trajectory recovery and simulation reliability while preserving competitive parameter accuracy. Matched-budget comparisons expose where the learned initialization helps and where ordinary warm starts remain competitive. Experiments with additive noise, FitzHugh-Nagumo and Lorenz-63 dynamics, and a data-conditioned Hudson Bay hare-lynx case study extend the comparison beyond clean gLV trajectories. Together, these results establish the need to evaluate parameter recovery, trajectory accuracy, numerical validity, and uncertainty as distinct properties of an inverse method.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
parameter inference, inverse problem, dynamical systems, ordinary differential equations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_sqZvOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies nonlinear dynamical inverse problems in which model parameters are estimated from observed trajectories. The focus is on settings with practically non-identifiable or "sloppy" parameters (meaning many parameter combinations produce similar observable behavior and parameter accuracy is not necessarily aligned with predictive performance). I am intimately familiar with this type of problem in the field of uncertainty quantification (UQ).

The proposed method is referred to as OIL (for "Optimizer-In-the-Loop"). As I understand it, the workflow is like this: a single NN is trained to predict parameters from observations by first predicting a parameter from an observed trajectory and then using this parameter estimate as the starting point for a small number of trajectory-matching optimization steps. These steps generate a sequence of refined parameter estimates and corresponding simulated trajectories. A loss is then computed from the resulting optimized trajectory, which is then backpropagated through the optimization process to update the NN weights. The idea is that this NN should learn to produce parameter estimates that are better starting points for subsequent trajectory matching (a downstream task).

From a UQ perspective, this resembles a multi-fidelity inverse-problem workflow in which a learned coarse inverse solver is followed by a more expensive optimization-based refinement stage. In classical multi-fidelity approaches, the lower-fidelity component is usually a surrogate for the forward model, whereas here where the NN approximates the inverse solution itself. Nevertheless, the workflow follows a familiar principle of obtaining a cheap approximate solution and then refining it through a more accurate optimization procedure.

The validation is done on the generalized Lotka-Volterra systems (of varying dimension) together with a couple of other systems. The evaluation considers parameter-estimation accuracy, trajectory-reconstruction accuracy, forward-simulation validity, posterior diagnostics, and Fisher-information-based sensitivity geometry. The empirical findings show that these metrics can disagree substantially in "sloppy" inverse problems, which is consistent with expectations from the inverse-problems and UQ literature.

Overall, the paper's primary contribution is not a new inverse-problem formulation, but rather an optimizer-aware amortized estimator that learns how to initialize a trajectory-matching inverse solver. In that sense, its strongest contribution is the extensive empirical studies provided (most of which are found in the appendix). Its principal limitation is that the underlying methodology is largely a synthesis of existing ideas from learned initialization, differentiable optimization, and amortized inference rather than a fundamentally new theoretical framework, but that is mostly coming from my UQ background. I think the paper would be of interest to the ICLR crowd.

Critical strengths
  1. I believe the strongest aspect of the paper is the empirical study rather than the OIL method itself. While the disagreements among the metrics is widely expected by UQ practitioners, it does highlight an important practical issue that accurate parameter estimates from NNs are likely insufficient for many downstream tasks.

  2. OIL is a principled approach for training a NN estimator to act as an effective initializer for any subsequent trajectory matching. I believe the approach is also straightforward enough for most people to incorporate.

Critical weaknesses
  1. The methodological novelty is a bit limited. The core idea can largely be interpreted as a learned warm-start strategy for trajectory matching that combines many existing ideas and is similar philosophically to approaches used in UQ for inverse problems.
Questions for authors

How much of the observed improvement comes specifically from training through the trajectory-matching optimization steps, as opposed to simply training a NN estimator that provides a good initial guess for trajectory matching?

Reviewer_yGZqOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper investigates parameter inference for nonlinear dynamical systems, focusing on generalized Lotka–Volterra models under practical non-identifiability. The authors compare amortized neural estimators, neural posterior estimation, and traditional test-time optimization methods, highlighting the differences between parameter accuracy, trajectory recovery, and numerical validity. They propose Optimizer-In-the-Loop (OIL), which trains a neural initializer by differentiating through several trajectory-matching optimization steps and then applies further optimization at inference time. The paper also uses Fisher Information Matrix (FIM) analysis to study parameter sensitivity and identifiability. Experiments are conducted across different system sizes, observation noise levels, and additional dynamical systems, including FitzHugh–Nagumo and Lorenz-63, as well as a real-world hare–lynx dataset. The results suggest that OIL can improve trajectory recovery and sensitivity-subspace alignment, although its advantages are not consistent across all settings and numerical failures remain.

Critical strengths
  1. The paper studies a meaningful problem in ODE parameter inference and clearly distinguishes parameter accuracy from trajectory recovery and numerical validity.

  2. The experimental evaluation is fairly extensive, covering different system sizes, noise levels, optimization budgets, and FIM-based diagnostics.

Critical weaknesses
  1. Limited novelty. This is my main concern. The proposed OIL method essentially combines learned warm starts, unrolled optimization, and trajectory-based supervision, all of which are well-established ideas. I do not see a sufficiently clear methodological contribution beyond applying this combination to ODE parameter inference. The FIM analysis is also largely based on existing sloppiness theory.

  2. Missing important classical baselines. The generalized Lotka–Volterra equations have a useful linear structure in their unknown parameters. For example, integrating the log-transformed dynamics gives a linear relationship involving the growth rates and interaction coefficients. This naturally suggests integral matching, regularized least squares, or related approaches followed by trajectory optimization. The paper includes gradient matching, but does not adequately compare against these structure-aware alternatives. Without such comparisons, it is difficult to justify the need for a much more complicated learning-based approach.

  3. The improvements are not very convincing. In Appendix Table 3, OIL and MPNN+TM achieve almost identical results at S=3, while OIL is actually worse at S=6 (3.61e-4 vs. 2.90e-4). The advantage becomes more noticeable at S=10, but it is not consistent across settings. Under higher observation noise, the differences in median trajectory recovery also become quite small. Considering the additional training cost, the practical benefit of OIL remains unclear.

  4. Insufficient ablation studies. The current experiments do not clearly establish which part of OIL is responsible for the improvements. In particular, a properly matched comparison with ordinary joint parameter-and-trajectory fine-tuning followed by the same optimizer is missing. The individual contributions of the three loss terms and the number of unrolled optimization steps also need more systematic investigation.

  5. Numerical reliability remains a concern. Although OIL improves over raw MPNN predictions, it still produces 6.38% invalid trajectories at S=10, whereas direct TM achieves 100% validity. Its full-cohort upper-tail errors remain problematic. This weakens the claim that OIL provides substantially more reliable trajectory recovery than conventional optimization.

  6. Limited real-world validation. Most experiments are conducted on synthetic, fully observed dynamical systems. The real-data evaluation uses only one short hare–lynx time series, where OIL improves RMSE from 4.74 to 4.38 over TM. Moreover, the synthetic training distribution is constructed using a posterior fitted to the same real observations. Therefore, this experiment provides limited evidence of generalization to genuinely unseen real-world systems.

Questions for authors

See weaknesses

Reviewer_jQyGOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This work demonstrates across various inverse methods for recovering the parameters of a Lotka–Voltera system that if a model is accurate in parameters, it still may not be accurate in terms of the trajectories it predicts. It also proposes a new inverse method, Optimizer-In-the-Loop (OIL) that provides a warm start for the parameters that are subsequently optimised by a trajectory matching method. They compare OIL to other inverse methods on a synthetic system with 3, 6, and 10 species, and a real-world dataset.

Critical strengths
  • The notion that parameter accuracy is not the same as trajectory-level accuracy is good to emphasise.
  • Limitations of the method are openly discusses, including some remaining prediction of invalid trajectories.
Critical weaknesses
  • The paper is lacking a strong contribution. While it is true that parameter accuracy is not equivalent to trajectory accuracy for inverse methods like these, this is not a novel observation, nor is demonstrating this in the synthetic experiments very useful. The contribution of the work then rests on OIL, which has limited empirical evidence to demonstrate its worth.
  • The results for OIL are generally unremarkable in comparison to the included baselines. Using OIL to warm start TM does not convincingly beat the base TM method in terms of parameter or trajectory recovery, and it also does not consistently beat using MPNN as the warm start instead. Furthermore, OIL+TM requires much more training time than MPNN+TM, with limited return.
  • Only one real-world dataset is used, and the results on this are not very comprehensive, with a very marginal difference in terms of MSE and trajectory visualisation between TM and OIL+TM. It is also unclear why why MPNN is reported as a “catastrophic failure” in this setting.
  • To claim that OIL's advantage is in "difficult high-dimensional regimes" (L459) therefore seems unsupported by these empirical results.
Questions for authors
  • Can the authors elaborate on or clarify the worth of the contributions of this work?
  • Can more real world data analyses be included?
  • Can the comparison between OIL+TM vs. MPNN+TM be moved to the main body of the paper, and discussed more thoroughly to elucidate any advantages for OIL? Can confidence intervals be included for this analysis as well, as this seems to be the most relevant comparison to make, not OIL+TM vs TM, or OIL+TM vs MPNN as is currently in Figure 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.