acceptodds
Under review as a conference paper at ICLR 2027

Simulation-Free Learning of Population Dynamics with Wasserstein Lagrangian Residuals

Abstract

The dynamics of cells, organisms, and fluids are often modeled as probability distributions evolving over time. Reconstructing and extrapolating this evolution from unpaired snapshots requires assumptions about the underlying process. Wasserstein gradient flows are a common choice, but they cannot describe conservative or periodic dynamics. Lagrangian mechanics in Wasserstein space covers both, but existing methods for learning it are simulation-based: they run a numerical solver at every training step, which makes training expensive. We propose Double-Stitch, a simulation-free method that learns these mechanics by penalizing the residual of the equation of motion along a learned population path. We derive this equation from a Clebsch variational principle that does not require gradient velocities, and show that the residual vanishes exactly when the equation holds. We test Double-Stitch on synthetic, single-cell and ocean vortex datasets and find that it matches or outperforms gradient-flow methods and simulation-based WLM on most tasks, while training 4 -14 times faster than WLM.

Primary area:
learning on time series and dynamical systems
Keywords:
population dynamics, simulation-free, Lagrangian mechanics, Wasserstein gradient flow

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_V3HLOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Double-Stitch, a simulation-free method for learning population dynamics from unpaired temporal snapshots. Building on Stitching, it jointly optimizes a KDE population trajectory, potential energy, and friction using a data-fitting loss and a second-order mechanical residual. Training avoids differentiating through sequential simulations, while the learned dynamics can be integrated for prediction. Experiments cover synthetic oscillators, embryoid-body single-cell data, and ocean vortices, with substantial training speedups over simulation-based methods.

Critical strengths
  • The collocation-vs-shooting framing is well-motivated and practical. Bringing simulation-free residual training to second-order population dynamics is a natural extension, and the speedup mechanism is well explained, namely the fact that the residual parallelizes over time nodes, while a rollout is sequential. Extending simulation-free residual training to second-order dynamics is well motivated. Moreover, the mathematical derivations are correct and rigorous.

  • The training speedups in the reported settings are substantial (Table 4) over WLM in the evaluated settings.

Critical weaknesses
  • Limited methodological novelty. Double-Stitch closely follows Stitching’s framework of jointly fitting a population path and an energy model, retaining the KDE representation, data-fitting objective, and energy form. Its principal algorithmic change is the second-order residual, supplemented by the non-gradient-velocity derivation and initial-velocity constraint. These are useful extensions, but the existing ablations do not establish whether learning the force improves held-out interpolation beyond a simpler acceleration-smoothing penalty. Such a comparison would strengthen the evidence for the dynamical modeling component.

  • Limited evidence that the learned law is accurate, especially for extrapolation. Extrapolation is the paper’s stated advantage over flow-based methods. On large-vortex forecasting, static Double-Stitch obtains W1=0.134W_1=0.134 under pushforward but 1.8391.839 under rollout, raising concerns about the accuracy of its learned dynamics over longer trajectories. Pushforward also uses observed velocities at its restart time, so this gap does not isolate extrapolation error. Nevertheless, it raises a substantive concern about the learned dynamics over longer uninterrupted trajectories. The time-dependent variant substantially improves rollout error to 0.2220.222, close to WLM’s 0.1940.194, but forecasting is evaluated at only one future snapshot. Although the paper acknowledges these limitations, the reported results leave unclear when the static model’s error accumulates and provide limited evidence of forecasting reliability beyond this setting.

Questions for authors
  • Residual and self-consistency. Could you report the empirical residual in Equation (12) at convergence, together with the W1W_1 between the fitted path and a rollout of the learned law from the fitted initial state, at the training times. Evaluating this at the training times would clarify how closely the learned law reproduces the optimized path.

  • Static ocean rollout. What explains the large static rollout error? Could you provide a per-snapshot rollout W1W_1 for the static and time-dependent models at t=1,…,10t=1, \dots, 10? This would show whether the discrepancy builds up within the observed interval, only beyond it, or both.

  • Ablations of the dynamics. Could you compare held-out interpolation accuracy against the same trajectory model with an acceleration-only penalty, obtained by setting U≡0,γ=0U\equiv0, \gamma=0, while retaining the residual weight? These would separate the contributions of path fitting, the smoothness prior and the learned force.

4 comments

Thank you for the detailed review! We appreciate you noting that the derivations are rigorous and the training speedup is useful.

We numbered the weaknesses and questions and grouped them as {W1/Q3, W2/Q2, and Q1} and addressed them below. Also please see the updated manuscript: the edits addressing and hopefully resolving your concerns are highlighted in green (some edits that answer several reviewers may have another color, we give their specific sections below).

 

W1. Limited methodological novelty. Double-Stitch closely follows Stitching’s framework of jointly fitting a population path and an energy model, retaining the KDE representation, data-fitting objective, and energy form. Its principal algorithmic change is the second-order residual, supplemented by the non-gradient-velocity derivation and initial-velocity constraint. These are useful extensions, but the existing ablations do not establish whether learning the force improves held-out interpolation beyond a simpler acceleration-smoothing penalty. Such a comparison would strengthen the evidence for the dynamical modeling component.

Q3. Ablations of the dynamics. Could you compare held-out interpolation accuracy against the same trajectory model with an acceleration-only penalty, obtained by setting U≡0U\equiv 0, γ≡0\gamma \equiv 0 while retaining the residual weight? These would separate the contributions of path fitting, the smoothness prior and the learned force.

We agree that Double-Stitch builds on Stitching and keeps its KDE path and data loss. Our contribution is the extension to second-order mechanics. First, we derive the dynamical law for general (non-gradient) velocities (see Appendix B.4) (when fitting paths directly, velocities are not guaranteed to be gradient, so WLM’s derivation can’t be used). Second, we derive the corresponding residual and show what a small residual does and does not guarantee (see Appendices C.3 and C.5). Third, we show theoretically and empirically that the simulation-free method is generally faster to train because the residual is evaluated at all time nodes at once (see Section 4.4 and Proposition 1).

We add the requested acceleration-only ablation on EB and oceans datasets (see Table 21 in Appendix E.6). We fix all potential, entropy and interaction force coefficients and damping to zero, keeping the other training settings unchanged. Both models use direct prediction on the same held-out data and the same three seeds (mean ±\pm std below; see Table 14 for Direct evaluation mode for oceans). For L1O, we average the three holdouts within each seed.

ModelEB LPO (W1W_1)EB L1O (W1W_1)EB L2O (W2W_2)Ocean interp. (W1W_1)Ocean forecast (W1W_1)
Double-Stitch0.6184 ±\pm 0.00080.7329 ±\pm 0.00350.7947 ±\pm 0.00640.0710 ±\pm 0.01900.2092 ±\pm 0.0133
Acceleration-only0.6880 ±\pm 0.00260.7850 ±\pm 0.00610.8026 ±\pm 0.00550.1884 ±\pm 0.00820.2559 ±\pm 0.0041

We find lower error with the full residual on LPO, L1O and both ocean tasks in every seed. On L2O, the difference is small and mixed across seeds. This shows that the mechanical terms contribute more than just smoothing, although the benefit depends on the task.

W2. Limited evidence that the learned law is accurate, especially for extrapolation. Extrapolation is the paper’s stated advantage over flow-based methods. On large-vortex forecasting, static Double-Stitch obtains W1=0.134W_1 = 0.134 under pushforward but 1.8391.839 under rollout, raising concerns about the accuracy of its learned dynamics over longer trajectories. Pushforward also uses observed velocities at its restart time, so this gap does not isolate extrapolation error. Nevertheless, it raises a substantive concern about the learned dynamics over longer uninterrupted trajectories. The time-dependent variant substantially improves rollout error to 0.2220.222, close to WLM’s 0.1940.194, but forecasting is evaluated at only one future snapshot. Although the paper acknowledges these limitations, the reported results leave unclear when the static model’s error accumulates and provide limited evidence of forecasting reliability beyond this setting.

Q2. Static ocean rollout. What explains the large static rollout error? Could you provide a per-snapshot rollout W1W1 for the static and time-dependent models at t=1,…,10t=1, \ldots, 10? This would show whether the discrepancy builds up within the observed interval, only beyond it, or both.

Direct prediction scores the optimized particles, while rollout simulates the learned law, so a poor rollout does not imply a poor direct prediction. We report both in Table 14.

To demonstrate extrapolation across multiple future time steps, we add an undamped-orbit experiment to Appendix E.2. We generate 300 unpaired samples per snapshot from a Gaussian population moving around a circle. We train on the first rotation (t=0,…,12t=0,\ldots,12), learning a static quadratic potential and damping, and simulate the next rotation from the final fitted positions and velocities. The learned stiffness is 0.2727±0.00010.2727\pm0.0001 (true 0.27420.2742) and the damping decreases from 0.1 to 0.0072±0.00030.0072\pm0.0003 (true γ=0\gamma=0). The table compares the fitted particles at three training snapshots with simulated predictions at three future snapshots (W1W_1, mean ±\pm std over three seeds). We see that the extrapolation error remains close to the fitting error.

Snapshot tt4812162024
Double-Stitch0.0621 ±\pm 0.00180.0606 ±\pm 0.00230.0556 ±\pm 0.00140.0695 ±\pm 0.00200.0782 ±\pm 0.00030.0826 ±\pm 0.0021
TypeTrainingTrainingTrainingForecastForecastForecast

Note that in general for any method different laws can match the same snapshots and diverge after the last one (see our non-identifiability discussion in Section 2.1 and Appendix E.1).

We also provide the requested per-snapshot rollout errors on the large ocean vortex (Appendix E.4, Figure 12). We simulate from the observed initial positions and velocities without restarting, using the existing fits. The table reports W1W_1 over three seeds (training snapshots t=0,…,9t=0,\ldots,9, forecast at t=10t=10). We see that the static model's error already grows within the observed interval, while the time-dependent model stays closer to the data.

tt12345678910
V(x)V(x)0.040 ±\pm 0.0040.120 ±\pm 0.0660.073 ±\pm 0.0400.056 ±\pm 0.0220.130 ±\pm 0.0970.254 ±\pm 0.2080.447 ±\pm 0.3610.750 ±\pm 0.5891.306 ±\pm 0.7971.824 ±\pm 0.562
V(x,t)V(x,t)0.101 ±\pm 0.0030.163 ±\pm 0.0250.084 ±\pm 0.0240.107 ±\pm 0.0090.131 ±\pm 0.0570.164 ±\pm 0.0670.207 ±\pm 0.0520.183 ±\pm 0.0190.144 ±\pm 0.0070.223 ±\pm 0.008

From t=4t=4 the static model's error grows by a factor of about two with every snapshot. The same happens when we start from the fitted initial state and compare with the fitted path (Figure 12, left), so the initial condition is not the cause. Both models have a similar particle residual (see Appendix C.5.3: 0.0860.086 and 0.0830.083 RMS), so the static model does not fit the law worse locally. Instead its small local error accumulates along the trajectory, which the residual does not control (Appendix C.5). The time-dependent model does not show this growth.

We agree that our introduction stated extrapolation too categorically, so we rewrote that sentence (Section 1) and added the rollout drift to the Limitations.

Q1. Residual and self-consistency. Could you report the empirical residual in Equation (12) at convergence, together with the W1W_1 between the fitted path and a rollout of the learned law from the fitted initial state, at the training times. Evaluating this at the training times would clarify how closely the learned law reproduces the optimized path.

To illustrate how residual and prediction quality change during fitting, we show residual RMS and prediction W1W_1 at initialization and after training (see Table 6 in Appendix C.5.4; same evaluation modes as in the paper; mean ±\pm std over 3 seeds).

SyntheticEB LPOOceans interpolation
MetricResidual RMSPrediction W1W_1Residual RMSPrediction W1W_1Residual RMSPrediction W1W_1
Double-Stitch initialization20.7490.4103.988 ±\pm 0.0511.970 ±\pm 0.0051.214 ±\pm 0.0020.231 ±\pm 0.000
Double-Stitch fitted0.7550.3890.163 ±\pm 0.0010.618 ±\pm 0.0010.093 ±\pm 0.0140.065 ±\pm 0.004

We see that training reduces both residual RMS and prediction error in these settings (training lowers the residual RMS by more than ten times). Note, however, that in general, low residual value does not necessarily mean that the dynamical law is satisfied (see the discussion in Appendix C.5).

We also provide the requested fit-to-rollout comparison. Specifically, we simulate from the fitted initial positions and velocities without restarting and compare with the fitted particles at the training snapshots (see Table 7 in Appendix C.5.4; raw Eq. (12) loss at convergence and the fit-to-rollout W1W_1).

Dataset (training snapshots after t0t_0)Eq. (12) loss1st2nd3rd4thMean
EB LPO (t=1,2,3,4t=1,2,3,4)0.107 ±\pm 0.0010.026 ±\pm 0.0010.169 ±\pm 0.0030.434 ±\pm 0.0040.683 ±\pm 0.0100.328 ±\pm 0.004
Oceans interpolation (t=2,4,6,8t=2,4,6,8)0.071 ±\pm 0.0190.185 ±\pm 0.0840.141 ±\pm 0.0700.383 ±\pm 0.0800.484 ±\pm 0.1520.298 ±\pm 0.022

We see that the rollout follows the fitted path over a short horizon and then drifts. On EB the gap is 0.0260.026 after one snapshot interval and 0.6830.683 after four, which is as large as the prediction error (0.6180.618). On oceans it grows from 0.1850.185 to 0.4840.484. So the learned law reproduces the fitted path locally, but not over a long uninterrupted simulation: small local errors accumulate, and the residual does not control this (see Appendix C.5). This is why direct and pushforward predictions, which do not need a long simulation, are more accurate than rollouts (Table 14). We added this to the Limitations.

 

In summary, we clarified how learned-particle predictions and rollouts are evaluated, reported empirical residual values, reported static ocean rollout error, and added an ablation of the dynamics. If we managed to clarify the points you raised, we would appreciate it if you could consider updating the score. In any case, thank you once again for the detailed review.

Thank you for the thorough and candid response. The acceleration-only ablation shows that the mechanical terms contribute beyond smoothing on EB LPO, L1O and both ocean tasks, and the requested residual and rollout diagnostics make clear what the residual does and does not guarantee. Together with the softened extrapolation claim and the added limitation, the paper now states its trade-off openly: a simulation-free method that trains 4–14 times faster than WLM at some cost in accuracy. On that basis I am raising my score to 3.

Reviewer_apoJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents Double-Stitch, a simulation-free approach to learning second-order population dynamics from unpaired temporal snapshots. Based on the Clebsch variational principle, the authors derive equations of motion that allow non-gradient velocity fields and incorporate their squared residual into a loss for fitting observed distributions. The method uses kernel density estimation (KDE) to represent population distributions and directly optimizes particle trajectories on a discrete time grid. Velocities and accelerations are estimated using finite differences, while the potential energy and damping coefficient are learned jointly. This avoids trajectory simulation during training updates. Experiments cover synthetic datasets with conservative, damped, and gradient-flow dynamics, along with single-cell and ocean vortex data. The results show comparable or better performance than WLM in single-cell distribution reconstruction and ocean vortex interpolation, along with substantially faster training under the reported experimental settings.

Critical strengths
  • The paper presents a simulation-free approach to learning second-order population dynamics. Training does not require integrating the ODEs governing the particle population, as WLM does, resulting in a substantial reduction in training time.

  • The empirical evaluation covers single-cell and ocean vortex data and includes both interpolation and extrapolation tasks. The method achieves performance comparable to or better than WLM in several experimental settings.

Critical weaknesses
  • The forecasting evaluation is limited to the final time point of the ocean vortex dataset. This provides limited evidence of extrapolation performance and leaves open whether the proposed method can match the extrapolation capabilities claimed in the original WLM paper, particularly over longer time horizons.

  • The potential energy includes an entropy term, while the density ρ\rho is estimated by placing a Gaussian kernel at each particle. However, KDE may become unreliable in high-dimensional settings, raising concerns about the accuracy of this entropy contribution when applying the method to high-dimensional single-cell data.

Questions for authors
  • Regarding Weakness 1, could the authors evaluate long-horizon extrapolation on synthetic or real-world datasets?

  • Regarding Weakness 2, how can the accuracy of KDE be ensured in high-dimensional settings? Are there alternative approaches that would allow the entropy contribution to the potential energy to be estimated more accurately?

  • (Minor) Finally, I have a further concern about the conceptual contribution of the method. At a high level, the approach can be understood as parameterizing trajectories to match the observed distributions at the observation times, while minimizing a dynamical residual along those trajectories. Both components are relatively natural, and their combination, as currently presented, does not offer a particularly compelling conceptual advance. This makes it difficult for me to justify a score of 4. I would appreciate deeper insights into what makes the formulation distinctive and why it works, beyond demonstrating that a straightforward combination of these components yields a sound and effective method.

4 comments

Dear reviewer,

Thank you for the thoughtful review of the paper. We appreciate your assessment of the method's soundness and computational advantage.

We grouped the weaknesses and questions you pointed out and numbered them as {W1/Q1, W2/Q2, and Q3} and addressed them below. Also please see the updated manuscript: the edits addressing and hopefully resolving your concerns are highlighted in blue (some edits that answer several reviewers may have another color, we give their specific sections below).

 

W1: The forecasting evaluation is limited to the final time point of the ocean vortex dataset. This provides limited evidence of extrapolation performance and leaves open whether the proposed method can match the extrapolation capabilities claimed in the original WLM paper, particularly over longer time horizons.

Q1: Regarding Weakness 1, could the authors evaluate long-horizon extrapolation on synthetic or real-world datasets?

On the large ocean vortex WLM is indeed better at forecasting (see Table 3). Under rollout the time-dependent Double-Stitch is close to it (0.222±0.0080.222\pm0.008 against 0.194±0.0480.194\pm0.048), while the static one performs poorly (see the error at every snapshot in Figure 12 in Appendix E.4). This benchmark has only one future snapshot.

To demonstrate Double-Stitch forecasting over multiple steps, we add a synthetic dataset: an undamped orbit (γ=0\gamma=0), where we observe the first rotation (13 snapshots) and forecast the second (12 snapshots). We use a static quadratic potential and learn the potential, particle paths and damping jointly, starting γ\gamma at 0.1. We learn γ=0.0072±0.0003\gamma=0.0072\pm0.0003 (true value 00). Simulating from the final fitted positions and velocities gives mean future W1=0.0723±0.0014W_1=0.0723\pm0.0014 over three seeds, compared with 1.26491.2649 for persistence (see Figure 8 in Appendix E.2). In the table below we show the error at three training and three future snapshots (W1W_1, mean ±\pm std over three seeds).

Snapshot tt4812162024
Double-Stitch0.0621 ±\pm 0.00180.0606 ±\pm 0.00230.0556 ±\pm 0.00140.0695 ±\pm 0.00200.0782 ±\pm 0.00030.0826 ±\pm 0.0021
TypeTrainingTrainingTrainingForecastForecastForecast

Note that for any method, different laws can match the same snapshots and diverge after the last one (see our non-identifiability discussion in Section 2.1 and Appendix E.1).

In general we expect a simulation-based method such as WLM to be more reliable for long forecasts: it enforces the law along the whole simulated trajectory, while our residual enforces it only approximately. This is essentially the trade-off that the simulation-free method makes to offer faster training. We added the rollout drift to the Limitations.

W2: The potential energy includes an entropy term, while the density is estimated by placing a Gaussian kernel at each particle. However, KDE may become unreliable in high-dimensional settings, raising concerns about the accuracy of this entropy contribution when applying the method to high-dimensional single-cell data.

Q2: Regarding Weakness 2, how can the accuracy of KDE be ensured in high-dimensional settings? Are there alternative approaches that would allow the entropy contribution to the potential energy to be estimated more accurately?

The entropy term enters the dynamical law only as a force, −cH∇log⁡ρ-c_H\nabla\log\rho. Our density ρ\rho is a mixture of Gaussian kernels, so ∇log⁡ρ\nabla\log\rho has a closed-form expression (see Appendix C.2) and the entropy term adds no error beyond that of the mixture itself. On EB the entropy term plays almost no role: we report the learned cHc_H in Table 22, and refitting with cHc_H fixed to zero does not change the error (see Appendix E.6).

To test higher dimensions, we refit the same model on d=10d=10, 5050 and 100100 principal components of the EB dataset in LPO mode (each component standardized to unit variance, as in Table 2), with the same cells, split and settings (see Table 22 in Appendix E.6). For reference we report persistence (the preceding training snapshot) and the error on the first five components, which is in the units of Table 2 (W1W_1, mean ±\pm std over three seeds). For comparison, we also refit WLM at each dd with its EB settings unchanged (as in Table 2, it pushes forward the held-out cells of the previous snapshot). Note that Double-Stitch uses P=100P=100 particles here (as in Table 20), so its error at d=5d=5 differs from Table 2.

In the full space Double-Stitch degrades more than WLM and loses to persistence for d≥50d \ge 50.

W1W_1 on all dd componentsd=5d=5d=10d=10d=50d=50d=100d=100
Double-Stitch0.666 ±\pm 0.0161.736 ±\pm 0.0178.563 ±\pm 0.03114.724 ±\pm 0.015
WLM0.683 ±\pm 0.0091.622 ±\pm 0.0246.594 ±\pm 0.01910.720 ±\pm 0.037
Persistence1.3452.2557.23011.379

The added components change little between snapshots, but after standardization they dominate W1W_1. The KDE spreads the predicted cells in these components (see the spread rows in Table 22). On the first five components both methods degrade in the same way and still beat persistence.

W1W_1 on the first 5 componentsd=5d=5d=10d=10d=50d=50d=100d=100
Double-Stitch0.666 ±\pm 0.0160.728 ±\pm 0.0090.896 ±\pm 0.0231.018 ±\pm 0.004
WLM0.683 ±\pm 0.0090.782 ±\pm 0.0310.906 ±\pm 0.0131.068 ±\pm 0.072
Persistence1.3451.3451.3451.345

So this degradation is not specific to the KDE.

As for alternative approaches, the entropy term can be dropped (cH=0c_H=0), or its gradient ∇log⁡ρ\nabla\log\rho can be supplied by an estimator built for high dimension, such as sliced or denoising score matching. Both plug into the residual unchanged, since it only needs ∇log⁡ρ\nabla\log\rho at the particles. We added this discussion to the Limitations.

Q3. (Minor) Finally, I have a further concern about the conceptual contribution of the method. At a high level, the approach can be understood as parameterizing trajectories to match the observed distributions at the observation times, while minimizing a dynamical residual along those trajectories. Both components are relatively natural, and their combination, as currently presented, does not offer a particularly compelling conceptual advance. This makes it difficult for me to justify a score of 4. I would appreciate deeper insights into what makes the formulation distinctive and why it works, beyond demonstrating that a straightforward combination of these components yields a sound and effective method.

We agree that path fitting and residual minimization are established ideas. Our contributions are two-fold. First, we propose the residual formulation for second-order population mechanics: the Clebsch derivation allows general velocities, and the resulting residual lets us fit a population path and its mechanics directly from unpaired snapshots. Second, we demonstrate the computational advantage of the simulation-free method: the residual can be evaluated across time nodes without sequential integration.

We additionally show that the performance gain from the residual goes beyond simple smoothness (see the acceleration-only control requested by another reviewer in Table 21). The residual is also only an approximate constraint at finite grid resolution and bandwidth. We describe the relationship between dynamical law, continuous residual, and approximated residual in Appendix C.5.

 

In summary, we added a longer-horizon forecasting experiment, per-time rollout diagnostics on the ocean data, and the cH=0c_H=0 and higher-dimensional EB experiments, and clarified the scope of the KDE parametrization. Please let us know if you have any follow-up questions. If we managed to clarify the points you raised, we would appreciate it if you could consider updating the score. In any case, thank you once again for the thoughtful review.

Thank you for the detailed responses, additional experiments, and careful revisions. I appreciate the authors’ efforts to address my questions and clarify the method’s strengths and limitations. I maintain my positive assessment of the paper and will keep my score of 3 (weak acceptance). Thank you again for the constructive discussion.

Reviewer_8DvNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The goal of this paper is a simulation-free method to learn non-gradient population dynamics, going beyond classical Wasserstein gradient flows. The proposed approach based on Lagrangian residuals: a simulation-free algorithm that learns the dynamics by optimizing the trajectory to minimize the distance between the learned and observed snapshots (data loss) as well as the residual of the underlying dynamical law (residual loss).

Critical strengths
  • Modeling non-gradient population dynamics is an important and well-motivated problem and the paper makes a meaningful contribution through a simulation-free method to fitting second-order population mechanics.
Critical weaknesses
  • Authors state ”To compute the residual loss, we approximate the population average in Equation (7) by a weighted sum over the learned center paths.” This is clearly an approximation of the residual loss, but then the question is how well does this loss capture the claimed mechanical law? This could possibly be evaluated empirically.

  • Novelty and framing is questionable. In my opinion, the novelty is not so much learning non-gradient dynamics but rather a specific way of doing so through a second-order model. This difference is important, because there are other established ways of learning non-gradient dynamics. The most important predecessor seems Curly-FM, but its framing is often lost in the paper. For example, statements such as “Flow-based methods do not make explicit assumptions” and “Unlike flow-based methods, this method allows extrapolation” are too categorical, and clearly misplace flow-based Curly-FM. Marking Curly-FM as X* (no extrapolation) is misleading in Table 1: It can in principle extrapolate, as acknowledged in the footnote by authors. I believe authors should present this method as explicitly an extrapolation case. To the best of my understanding, there is no theoretical argument as to why and how one approach would extrapolate better or worse in this case. The dividing line is one needs a reference process, whereas this approach relies on a second-order method.

  • On the empirical side, I find the results rather mixed and mostly insufficient. On the Ocean’s dataset, Curly-FM performs substantially better. On the cell experiments, it is unclear to me why the datasets and cell experiments from the Curly-FM paper are not included. It seems like the closest related baseline, and we are lacking a comparison except on Oceans dataset, where Curly-FM remains strong.

Questions for authors

Please refer to weaknesses.

6 comments

Thank you for the careful review! We appreciate you noting that you found our second-order simulation-free method a meaningful contribution to modeling population dynamics.

We numbered the weaknesses you pointed out as {1, 2(a-c), 3} and addressed them below. Also please see the updated manuscript: the edits addressing and hopefully resolving your concerns are highlighted in purple.

 

  1. Authors state ”To compute the residual loss, we approximate the population average in Equation (7) by a weighted sum over the learned center paths.” This is clearly an approximation of the residual loss, but then the question is how well does this loss capture the claimed mechanical law? This could possibly be evaluated empirically.

This is indeed an interesting question. We added Appendix C.5 to explain how the dynamical law relates to the population residual and its discrete approximation. Satisfying the law gives zero population residual; the converse holds on the population support under continuity (and everywhere if the density is strictly positive). Relating the discrete loss to the population residual additionally requires controlling time discretization, variation of the dynamical term across each kernel, and local particle-velocity dispersion. We measure the two kernel terms on the large ocean vortex fits (see Appendix C.5.3): the equation error of the KDE population is larger than the particle residual (2.953 against 0.086 RMS for the static model).

To illustrate what changes during fitting, we report Residual RMS and Prediction W1W_1 at initialization and after training (see Table 6 in Appendix C.5.4, using the same evaluation protocols as in the paper). Residual RMS is the square root of the raw loss divided by the total time-quadrature weight.

SyntheticEB LPOOceans interpolation
MetricResidual RMSPrediction W1W_1Residual RMSPrediction W1W_1Residual RMSPrediction W1W_1
Double-Stitch initialization20.7490.4103.988 ±\pm 0.0511.970 ±\pm 0.0051.214 ±\pm 0.0020.231 ±\pm 0.000
Double-Stitch fitted0.7550.3890.163 ±\pm 0.0010.618 ±\pm 0.0010.093 ±\pm 0.0140.065 ±\pm 0.004

Training reduces both residual RMS and prediction error in these settings. Initialization includes both the particle paths and the dynamical-law parameters. The fitted prediction scores reproduce the rounded values in Section 4.1 and Tables 2-3; the EB diagnostic uses three of the five seeds in the main table.

We separately evaluate how closely the learned law reproduces the fitted path (see fit-to-rollout W1W_1 at every training snapshot in Table 7 in Appendix C.5.4). This is distinct from prediction error against the data. The fit-to-rollout W1W_1 grows with the horizon (Table 7), so a small residual does not guarantee that a long rollout reproduces the fitted path.

2a. Novelty and framing is questionable. In my opinion, the novelty is not so much learning non-gradient dynamics but rather a specific way of doing so through a second-order model. This difference is important, because there are other established ways of learning non-gradient dynamics.

Indeed, learning non-gradient dynamics is an established field, and our contribution is a specific way of doing it: a simulation-free method for second-order dynamics. The existing method for this setting, WLM, simulates the particle trajectories at every training step, which makes it computationally expensive. Double-Stitch instead optimizes the particle paths directly and evaluates the residual of the dynamical law at all time nodes at once. As a result it trains 44–1414 times faster than WLM on EB and oceans (see Table 4 in Section 4.4). Flow-based methods are also simulation-free, but they learn a first-order velocity field and not a second-order law (see Table 1). We derive the dynamical law for general (non-gradient, unlike WLM) velocities, derive the corresponding residual (generalizing Stitching's approach), and implement and test the resulting method. We compare to the results reported for a wide range of methods, including ones based on flow-matching, WGF, and WLM. We believe our abstract and introduction already reflects that, but would be happy to correct if you see a place where we overclaim on this point.

 

2b. The most important predecessor seems Curly-FM, but its framing is often lost in the paper. For example, statements such as “Flow-based methods do not make explicit assumptions” and “Unlike flow-based methods, this method allows extrapolation” are too categorical, and clearly misplace flow-based Curly-FM.

We agree that the two sentences were too categorical and rewrote them (Section 1, purple highlighted parts in the updated manuscript). However, the closest predecessors are Stitching, which we extend, and WLM, which learns the same law (although with gradient velocities) by simulation. Curly-FM uses more inputs: its reference process is built from the velocities observed at all training snapshots (see Curly-FM paper Appendix G.4; and WLM paper Section 4.2), while Double-Stitch and WLM train on positions and the initial velocity. We added a clarification in our Section 4.3 and Table 1.

 

2c. Marking Curly-FM as X* (no extrapolation) is misleading in Table 1: It can in principle extrapolate, as acknowledged in the footnote by authors. I believe authors should present this method as explicitly an extrapolation case. To the best of my understanding, there is no theoretical argument as to why and how one approach would extrapolate better or worse in this case. The dividing line is one needs a reference process, whereas this approach relies on a second-order method.

Thank you, we double-checked this point. Curly-FM learns a velocity field between observed snapshots and reports interpolation only, and WLM notes that flow-based methods, including Curly-FM, cannot forecast beyond the observed horizon (their Sections 1 and 4.2). Our footnote meant that a reference process known beyond the last snapshot could extend it. We changed the Table 1 entry to "Conditional" and rewrote the footnote to say this. Our orbit experiment (point 3 below) shows both cases: as published, with time as an input, the learned field does not continue the rotation after the last snapshot, while a variant without the time input does on the training support. We agree that no theoretical argument shows which approach extrapolates better. For any method, different laws can match the same snapshots and diverge after the last one (see our non-identifiability discussion in Section 2.1 and Appendix E.1). Curly-FM's reference process averages the velocities of nearby observed particles, so it carries no information away from the observed positions; we therefore expect Curly-FM to degrade further when the forecast leaves the training support. Our orbit experiment (point 3 below) stays on the training support, the favorable case for Curly-FM. As you put it, the dividing line is the assumption added: a reference process or a second-order law.

  1. On the empirical side, I find the results rather mixed and mostly insufficient. [...] On the cell experiments, it is unclear to me why the datasets and cell experiments from the Curly-FM paper are not included. [...]

We agree that Curly-FM performs better on oceans, but this is not surprising: it has additional inputs (observed velocities at all training snapshots, from which it builds its reference process). The WLM paper arrives at a similar conclusion (see WLM paper Section 4.2).

We add a comparison to Curly-FM on a synthetic extrapolation dataset, the undamped orbit (see Table 8 in Appendix E.2). Both methods train on the first rotation (t=0,…,12t=0,\ldots,12) and extrapolate to the second rotation from t=12t=12: Double-Stitch from its fitted positions and velocities, Curly-FM from the observed particles. We run Curly-FM as published (time is an input to its velocity field), without the time input (the same field then applies after t=12t=12), and as its reference process alone (no learned field). "With velocities" means that the reference process uses the velocities observed at all 13 training snapshots; "without velocities" means positions only, as for Double-Stitch. The table reports W1W_1 mean ±\pm std over three seeds.

Snapshot tt4812162024
Double-Stitch0.0621 ± 0.00180.0606 ± 0.00230.0556 ± 0.00140.0695 ± 0.00200.0782 ± 0.00030.0826 ± 0.0021
Curly-FM, with velocities0.0726 ± 0.00190.1164 ± 0.01060.0704 ± 0.02341.3853 ± 0.07374.8802 ± 0.13447.1051 ± 0.5872
Curly-FM, without velocities0.1191 ± 0.00650.1784 ± 0.03370.1389 ± 0.02702.0594 ± 0.13696.2706 ± 0.28389.1340 ± 0.2292
Curly-FM without time input, with velocities0.0869 ± 0.01150.1589 ± 0.04340.2253 ± 0.06090.0965 ± 0.00780.1566 ± 0.01950.2180 ± 0.0329
Curly-FM without time input, without velocities0.1246 ± 0.02630.2645 ± 0.06400.3878 ± 0.06450.1246 ± 0.02550.2584 ± 0.04350.3887 ± 0.0239
Curly-FM reference process, with velocities0.1521 ± 0.02190.3045 ± 0.03870.4453 ± 0.05120.1436 ± 0.01190.2933 ± 0.01790.4421 ± 0.0216
TypeTrainingTrainingTrainingForecastForecastForecast

Double-Stitch manages to accurately extrapolate the trajectory: its error stays below 0.09 through t=24t=24, a full rotation past the last observation. From t=15t=15 on it has the lowest mean error of all rows; at t=13t=13 (not shown) the last three Curly-FM rows have slightly lower error and at t=14t=14 they are comparable. As published, Curly-FM's field is unconstrained after t=12t=12 and drives the population off the orbit (7–9 times the orbit radius at t=24t=24). Without the time input, the field gives one velocity to each position, so the population stays on the orbit but spreads to 2–3.5 times its observed width by t=24t=24. In our second-order law each particle carries its own velocity, which is one reason for the lower error of Double-Stitch.

The other single cell datasets that Curly-FM uses do not fit our problem setting. They have only two training snapshots and mainly score velocity reconstruction instead of coordinates (see Curly-FM Section 4.3 and Appendix E). The single-cell datasets differ as follows:

DatasetTemporal setupFitting inputsEvaluation
EBFive time bins; 3–5 used for trainingCell states, no RNA velocitiesPopulation prediction (W1W_1 or W2W_2)
FibroblastOne cell population reused at two endpoints (p0=p1p_0=p_1)Cell states and RNA velocitiesVelocity alignment and endpoint W2W_2; endpoint matching alone does not distinguish identity from cycling
ErythroidThree bins of inferred time; middle bin held outEndpoint cell states and RNA velocitiesMiddle-bin W2W_2 and velocity agreement

So instead we focus on the datasets from the Stitching and WLM papers: 6 tasks in total (1 synthetic Gaussian mixture, the EB single cell dataset with 3 scoring versions, and 2 ocean datasets), compared with flow-based, WGF, and WLM methods (see Tables 2 and 3). Compared to WLM, Double-Stitch trades off precision for speed, but still achieves the best performance on some tasks, in particular for interpolation. The multiple-shooting method we mention in the future work would give further control over this tradeoff, but is beyond the scope of our work.

 

In summary, we added the discussion of the dynamical law/residual/residual loss relationship and clarified the relationship and differences with the Curly-FM. Please let us know if you have any follow-up questions. If we managed to clarify the points you raised, we would appreciate it if you could consider updating the score. In any case, thank you once again for the detailed review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.