acceptodds
Under review as a conference paper at ICLR 2027

DisKO: Deep Koopman Learning in Distribution Space from Unpaired Snapshots

Abstract

Many complex systems are observed only through temporally unpaired distribution snapshots, making trajectory-based dynamical learning difficult without additional assumptions. We therefore formulate the problem directly in distribution space, treating the distribution itself as the dynamical state. The challenge is that distribution space is infinite-dimensional, making compact and approximately closed representations difficult to learn from finite snapshots. We introduce DisKO, which extends deep Koopman learning to distribution dynamics by jointly learning predictive distributional observables, a finite-dimensional Koopman representation, and a generative map back to the full distribution. Across seven diverse benchmarks, DisKO achieves state-of-the-art extrapolation performance, with substantially slower error accumulation on long-horizon prediction tasks. DisKO further recovers leading Koopman eigenvalues and eigenfunctions on systems with analytic spectra, revealing meaningful dynamical structure in the learned representation.

Primary area:
learning on time series and dynamical systems
Keywords:
Distribution Dynamics, Koopman Learning, Unpaired Snapshots

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_BbPzOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes to learn distribution dynamics of autonomous dynamical systems (deterministic or stochastic). Conceptually, it follows the idea of Distributional Koopman Operator (Oprea et al., 2025), but adds deep components to learn statistics of empirical distribution and to reconstruct distribution from them. Practically, the paper proposes a conventional Encoder - Finite Koopman Evolution - Decoder model, but adapted in dynamic of distributions: the encoder is implemented by a permutation-invariant network and takes empirical samples as input; the decoder is implemented by conditional flow matching. Evaluated on different wasserstein distances, the experimental results show that the proposed DisKO learn distribution dynamics better than baselines. No theoretical results are extablished to support the performance of DisKO. Overall, the paper nicely combines distributional koopman operator and conditional flow matching, and proposes a new conceptual framework to learn distribution dynamics.

Critical strengths
  1. DisKO consists of a new conceptual framework to learn distribution dynamics and it nicely combines distributional koopman operator and conditional flow matching.
  2. Compared to transfer operator which is heavy to approximate, DisKO use a more compact representation to learn distribution dynamics
Critical weaknesses
  1. No theoretical result is established to support experimental results: it's hard to know whether the claimed distributional koopman operator is actually learned or well approximated
Questions for authors
  1. The Encoder - Finite Koopman evolution - Decoder pipeline approximates the transfer operator, why the comparison to the transfer operator is not discussed at all?
  2. Can the authors provide insights on how theoretical guarantee can be established?
5 comments
AuthorsOpenReview ↗

Response to Q1 (Transfer operators, 1/1) — DisKO and methods for learning the forward transfer operator

Thank you for pointing out this connection. For time-homogeneous Markov dynamics, DisKO's end-to-end predictor D(Ft(E(μ)))D(F_t(E(\mu))) approximates the action of the forward transfer operator, PtμP_t\mu, on the family of distributions we model. Our original discussion focused on how distributional observables evolve under the Koopman operator (Oprea et al., 2025) and left this connection implicit.

Karimi and Georgiou (2022) use the Wasserstein distance to fit a common state map whose pushforward of each observed distribution approximates the next snapshot. Beier et al. (2025) estimate a transition kernel from batches of particles observed at two times, with the individual pairings within each batch unknown. For spectral analysis, BSO (Xu et al., 2026) learns the backward operator dual to the forward Markov operator. DisKO learns Ft(E(μ))≈E(Ptμ)F_t(E(\mu))\approx E(P_t\mu) from marginal distributions sampled separately at each time, without matching individuals across time.

WOT-iPSC measures different cells on different days. We compared DisKO directly with dynamic distribution decomposition (DDD), which can also learn from these snapshots (Taylor-King et al., 2020). DDD uses a continuous-time Markov rate matrix to evolve the mixture weights of Gaussian basis densities. We trained both methods on the 33 snapshots from days 0 to 15. Each forecast was a single rollout through day 18 from the same 1,024 cells observed on day 0. In this run (seed 0), DisKO had lower W1W_1 at all six held-out time points, with a mean W1W_1 32.4% lower than DDD's (Table 1.1).

Table 1.1. Extrapolation error at the six held-out time points on WOT-iPSC. We fit the SVD using only the training data. At each time point, we computed W1W_1 by optimal assignment between 512 predicted and 512 observed samples in the 50-dimensional SVD coordinates on their original scale. The final row gives the equally weighted mean over the six time points.

Evaluation timeDDD W1W_1 ↓DisKO W1W_1 ↓
Day 15.524.67616.521
Day 16.024.16016.773
Day 16.525.50916.500
Day 17.025.62217.119
Day 17.526.91418.360
Day 18.026.69518.540
Mean over six time points25.59617.302

References

  1. Oprea et al. (2025). The Distributional Koopman Operator for Random Dynamical Systems.
  2. Karimi and Georgiou (2022). Data-Driven Approximation of the Perron-Frobenius Operator Using the Wasserstein Metric. IFAC-PapersOnLine.
  3. Beier et al. (2025). Transfer Operators from Batches of Unpaired Points via Entropic Transport Kernels. Information and Inference.
  4. Xu et al. (2026). Learning Kolmogorov Backward Operators for Spectral Analysis and Forecasting: A Data-Driven Variational Galerkin Framework.
  5. Taylor-King et al. (2020). Dynamic Distribution Decomposition for Single-Cell Snapshot Time Series Identifies Subpopulations and Trajectories during iPSC Reprogramming. PLOS Computational Biology.
AuthorsOpenReview ↗

Response to Q2 (Theoretical guarantees, 1/3) — I. (a) Existence of approximately invariant subspaces and (b) Encoder approximation of distributional observables

Thank you for asking what theoretical guarantees DisKO admits. We give two results. (I) With the encoder fixed and under the assumptions of Proposition 1, the propagator fitted by least squares to nn exact snapshot pairs converges as n→∞n\to\infty to the optimal projection of the true distributional Koopman operator onto the chosen observable space. (II) We give conditions under which future distributions are identifiable in a class of linear Gaussian systems.

I. Finite-dimensional approximation of the distributional Koopman operator and approximately invariant subspaces

TL;DR. Closure relations in state space carry over to distribution space, where nonlinear observables can yield additional finite-dimensional closures (a). Existing results show that, under suitable compactness assumptions, a sufficiently expressive DisKO encoder can uniformly approximate any finite collection of continuous distributional observables to arbitrary accuracy (b). With the encoder fixed, we fit the dynamics to nn snapshot pairs. If the starting distributions represent the reference measure and the observables are nondegenerate, the least-squares fit converges to the optimal Koopman projection on this space as n→∞n\to\infty (c). This motivates DisKO's three stages. We learn observables by reconstruction, fix them to fit the dynamics, then jointly optimize all components to reduce the closure residual while retaining distributional information (d).

(a) Existence of approximately invariant subspaces

If state observables satisfy an exact or approximate closure relation under the underlying Markov generator, their expectations satisfy the same evolution equations with the same bound on the closure error. Nonlinear combinations of moments can yield additional closure relations for the same system.

Let G\mathcal G be the generator of the underlying Markov dynamics. Suppose qq state observables ϕ=(ϕ1,…,ϕq)⊤\phi=(\phi_1,\ldots,\phi_q)^\top satisfy

Gϕ=Cϕ+d+r,∥r∥∞≤ε0.\mathcal G\phi=C\phi+d+r, \qquad \lVert r\rVert_\infty\leq\varepsilon_0.

Here ε0\varepsilon_0 bounds the closure error at the generator level, with exact closure when ε0=0\varepsilon_0=0. Define the distributional observables by h(ν)=∫ϕ(x) dν(x)h(\nu)=\int\phi(x)\,d\nu(x). The distributional Koopman generator L\mathcal L satisfies Lh(ν)=∫Gϕ(x) dν(x)\mathcal Lh(\nu)=\int\mathcal G\phi(x)\,d\nu(x). Taking expectations therefore gives

Lh=Ch+d+rˉ,∥rˉ∥∞≤ε0,\mathcal Lh=Ch+d+\bar r, \qquad \lVert\bar r\rVert_\infty\leq\varepsilon_0,

where rˉ(ν)=∫r(x) dν(x)\bar r(\nu)=\int r(x)\,d\nu(x). Thus the distributional observables inherit the closure relation and its error bound.

Nonlinear distributional observables can have closed equations of their own. Consider the discrete-time doubling map T(x)=2x mod 1T(x)=2x\bmod 1 on [0,1)[0,1). For a single particle, cos⁡(2πx)\cos(2\pi x) evolves into cos⁡(4πx)\cos(4\pi x) after one step and cos⁡(8πx)\cos(8\pi x) after two. Each step introduces a new frequency. More generally, the doubling map is mixing with respect to Lebesgue measure. Its Koopman operator on L2([0,1],dx)L^2([0,1],dx) is an isometry whose only eigenfunctions are constants. Its finite-dimensional invariant subspaces therefore contain only constant functions.

Now consider the same map acting on the family of distributions νa\nu_a with densities ρa(x)=1+a(2x−1)\rho_a(x)=1+a(2x-1), where 0≤a≤10\leq a\leq1. At a=0a=0 the density is uniform. Larger values of aa place more mass toward the right end of the interval. A particle at yy after one step can come from y/2y/2 or (y+1)/2(y+1)/2, so the new density is 12[ρa(y/2)+ρa((y+1)/2)]=ρa/2(y)\tfrac12[\rho_a(y/2)+\rho_a((y+1)/2)]=\rho_{a/2}(y). Each step halves the slope of the density and keeps it in the same family.

Let H(ν)=112−Var⁡ν(X)H(\nu)=\tfrac1{12}-\operatorname{Var}_\nu(X), the variance of the uniform distribution minus the variance of the current particle distribution. We can estimate HH from the sample variance of a single snapshot. Integrating against ρa\rho_a gives

H(νa)=a236,H(νa/2)=14H(νa).H(\nu_a)=\frac{a^2}{36}, \qquad H(\nu_{a/2})=\frac14 H(\nu_a).

On this family, the expectation of any fixed state observable ϕ\phi, ∫01ϕ(x)ρa(x) dx\int_0^1\phi(x)\rho_a(x)\,dx, is affine in aa. No such expectation can therefore represent H(νa)H(\nu_a), which is quadratic in aa. The nonlinear observable HH spans an exact one-dimensional distributional Koopman invariant subspace. Since a≥0a\geq0, its value uniquely determines a=6H(νa)a=6\sqrt{H(\nu_a)} and hence the full density ρa/2n\rho_{a/2^n} after any number nn of steps.

(b) Encoder approximation of distributional observables

DisKO averages learned sample features before a neural readout (Zaheer et al., 2017). On a weakly compact family of distributions over a compact state space, such encoders with sufficient capacity can uniformly approximate any finite set of continuous observables to arbitrary accuracy (Pevný & Kovařík, 2019).

AuthorsOpenReview ↗

Response to Q2 (Theoretical guarantees, 2/3) — I. (c) Convergence to the optimal Koopman projection from finite snapshots and (d) How the theory informs three-stage training

(c) Convergence to the optimal Koopman projection from finite snapshots

Given a set of observables, such as those defined by a fixed encoder, we show when fitting the dynamics from snapshots converges to the optimal Koopman projection. The one-step propagation residual provides a way to assess approximate invariance of their span.

Fix the encoder EθE_\theta. Write Vθ=span⁡{1,Eθ,1,…,Eθ,m}V_\theta=\operatorname{span}\{1,E_{\theta,1},\ldots,E_{\theta,m}\} for its observable space and h(ν)=[Eθ(ν)⊤,1]⊤h(\nu)=[E_\theta(\nu)^\top,1]^\top for the augmented observable vector. Let Π\Pi be a reference probability measure on the family of distributions M\mathcal M. It determines how starting distributions are weighted in the population prediction error. Suppose we observe nn pairs of exact distribution snapshots (νi,TΔνi)(\nu_i,T_\Delta\nu_i) at a fixed lag Δ\Delta. The starting distributions define the empirical measure Πn=1n∑i=1nδνi\Pi_n=\frac1n\sum_{i=1}^n\delta_{\nu_i}. Averaging with respect to Πn\Pi_n gives each pair weight 1/n1/n. The matrix MnM_n minimizes the mean squared one-step error.

Mn=arg⁡min⁡M Eν∼Πn∥h(TΔν)−Mh(ν)∥2.M_n=\arg\min_M\,\mathbb E_{\nu\sim\Pi_n}\lVert h(T_\Delta\nu)-Mh(\nu)\rVert^2.

Thus MnM_n represents the optimal finite-dimensional projection of the true Koopman action on VθV_\theta under Πn\Pi_n, extending the usual Koopman projection to distribution space.

Two sampling issues arise in practice. First, a finite snapshot gives an empirical distribution. For independent, identically distributed samples, the law of large numbers ensures that its encoding under a fixed encoder converges to that of the underlying distribution as the sample size grows. Second, even with exact snapshots, the propagation rule we can identify depends on which starting distributions we observe. Proposition 1 extends the classical projection consistency result for extended dynamic mode decomposition (EDMD) to this setting.

Proposition 1 (Consistency of Empirical Distributional Koopman Projection). Assume that M\mathcal M is compact, EθE_\theta and TΔT_\Delta are continuous, and the following conditions hold.

  1. The empirical measure of the starting distributions converges weakly to the reference measure, Πn⇀Π\Pi_n\rightharpoonup\Pi.
  2. The Gram matrix GΠ=EΠ[h(ν)h(ν)⊤]G_\Pi=\mathbb E_\Pi[h(\nu)h(\nu)^\top] is positive definite.

Then, as n→∞n\to\infty,

M_\Pi^\ast=\mathbb E_\Pi[h(T_\Delta\nu)h(\nu)^\top]G_\Pi^{-1}.$$ Here $M_\Pi^\ast$ represents the $L^2(\Pi)$-orthogonal projection of the true distributional Koopman action on $V_\theta$ back onto $V_\theta$. **Remark (What the Gram condition requires of the snapshots).** More samples within a snapshot give a more accurate estimate of the distribution at that time. Identifying the propagator also requires enough variation across snapshots to distinguish different propagation rules. If all snapshots come from the same stationary distribution $\nu_\star$, even exact observations give the same vector $v=h(\nu_\star)$ each time. Then $G_\Pi=vv^\top$ has rank one, and every matrix satisfying $Mv=v$ fits the data. These snapshots cannot tell us whether a perturbation away from the stationary distribution would persist or decay. Thus, when the starting distributions represent the reference measure and the observables are nondegenerate, the fit from finite snapshots consistently estimates the optimal finite-dimensional projection of the true Koopman operator. #### (d) How the theory informs three-stage training Extending classical Koopman learning theory to distribution space raises questions about the existence of invariant subspaces and gives the Gram condition a concrete interpretation in terms of variation across distributions. **These results motivate DisKO's three-stage training procedure.** During **representation pretraining** in Stage 1, the distribution reconstruction objective encourages the encoder to learn observables that retain enough information to reconstruct a snapshot. In Stage 2, we fix the encoder and fit Koopman dynamics in the observable space it defines. For the discrete model at a fixed lag, suppose the ridge coefficient in Eq. (25) of the original paper is held fixed and the conditions of Proposition 1 hold. As the number $n$ of snapshot pairs tends to infinity, the fitted propagator converges to the matrix representing the optimal projection of the distributional Koopman action on this space under the reference measure. In Stage 3, **joint training** updates the observables, Koopman dynamics, and decoder together. The latent consistency loss reduces the Koopman closure residual, while the reconstruction and distribution prediction losses encourage the observables to retain the information needed for both tasks.
Reviewer_QQ5iOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DisKO, a deep Koopman learning framework for predicting distribution dynamics from temporally unpaired snapshots. The method combines a permutation-invariant distribution encoder, linear latent Koopman dynamics, and a conditional generative decoder. Experiments on seven benchmarks demonstrate strong long-horizon extrapolation performance, while additional spectral analyses show that the learned representations can recover meaningful Koopman eigenvalues and eigenfunctions.

Critical strengths
  • The paper extends deep Koopman learning to distribution-valued dynamics without requiring individual trajectory correspondences. Combining learned distributional observables with linear latent evolution and generative reconstruction provides a coherent and useful framework.

  • DisKO achieves the lowest W1 errors across all seven benchmarks, including real single-cell datasets. The recovery of analytical Koopman eigenvalues and eigenfunctions further supports the interpretability of the learned dynamics.

Critical weaknesses
  • Although the framework is motivated by distributional Koopman theory, the paper does not establish when the learned finite-dimensional observables form an approximately invariant subspace or when the underlying dynamics are identifiable from unpaired snapshots. Consequently, the theoretical connection between the learned representation and the true distributional Koopman operator remains insufficiently established.

  • The framework combines established set encoders, linear latent dynamics, and conditional flow matching. While the overall performance is strong, the main ablation comparing Koopman dynamics with a nonlinear Neural ODE is restricted to the OU benchmark. Additional controlled comparisons are needed to determine which components drive the improvements across more complex systems.

The paper presents an interesting extension of deep Koopman learning to distribution dynamics, supported by strong experimental results and meaningful spectral analyses. However, the theoretical justification for learned Koopman closure remains insufficient, and the limited component-level validation raises concerns about the distinct contributions of the proposed framework.

Questions for authors

1 Under what conditions can the learned observables approximate an invariant Koopman subspace from unpaired distribution snapshots?

2 Can the authors provide additional ablations isolating the contributions of Koopman dynamics, distribution encoding, and the generative decoder on more challenging datasets?

3 How robust is DisKO to changes in snapshot frequency, sample size, and initial distributions, particularly for datasets containing only a single observed sequence?

16 comments
AuthorsOpenReview ↗

Response to Q1 (Koopman invariance, 1/3) — TL;DR, (a) Existence of approximately invariant subspaces and (b) Encoder approximation of distributional observables

Thank you for asking when the learned observables can span an approximately Koopman-invariant subspace. Our argument has four parts.

TL;DR. Moving from state space to distribution space preserves existing closure relations. Nonlinear distributional observables can also admit finite-dimensional closures unavailable in state space (a). Existing results show that, under suitable compactness assumptions, a sufficiently expressive DisKO encoder can uniformly approximate any finite collection of continuous distributional observables to arbitrary accuracy (b). With the encoder fixed, we fit the dynamics to nn pairs of distribution snapshots without pairing individual samples. If the starting distributions are representative of the reference measure and the observables are nondegenerate, the least-squares fit converges as n→∞n\to\infty to the optimal projection of the distributional Koopman action onto that observable space under the reference measure (c). These results motivate DisKO's three training stages. We first learn observables by reconstructing distributions, then hold them fixed to fit the dynamics, and finally optimize all components jointly to reduce the closure residual while retaining the information needed to represent the distributions (d).

(a) Existence of approximately invariant subspaces

A distributional Koopman operator inherits invariant subspaces from the generator of the underlying dynamics. If a set of state observables satisfies an exact or approximate closure relation under that generator, their expectations satisfy the same evolution equations with the same bound on the closure error. Nonlinear combinations of moments can also yield new closure relations for the same system.

Let G\mathcal G be the generator of the underlying Markov dynamics. Suppose qq state observables ϕ=(ϕ1,…,ϕq)⊤\phi=(\phi_1,\ldots,\phi_q)^\top satisfy

Gϕ=Cϕ+d+r,∥r∥∞≤ε0.\mathcal G\phi=C\phi+d+r, \qquad \lVert r\rVert_\infty\leq\varepsilon_0.

Here ε0\varepsilon_0 bounds the closure error at the generator level, with exact closure when ε0=0\varepsilon_0=0. Define the distributional observables by h(ν)=∫ϕ(x) dν(x)h(\nu)=\int\phi(x)\,d\nu(x). The distributional Koopman generator L\mathcal L satisfies Lh(ν)=∫Gϕ(x) dν(x)\mathcal Lh(\nu)=\int\mathcal G\phi(x)\,d\nu(x). Taking expectations therefore gives

Lh=Ch+d+rˉ,∥rˉ∥∞≤ε0,\mathcal Lh=Ch+d+\bar r, \qquad \lVert\bar r\rVert_\infty\leq\varepsilon_0,

where rˉ(ν)=∫r(x) dν(x)\bar r(\nu)=\int r(x)\,d\nu(x). Thus the distributional observables inherit the closure relation and its error bound.

Nonlinear distributional observables can have closed equations of their own. Consider the discrete-time doubling map T(x)=2x mod 1T(x)=2x\bmod 1 on [0,1)[0,1). For a single particle, cos⁡(2πx)\cos(2\pi x) evolves into cos⁡(4πx)\cos(4\pi x) after one step and cos⁡(8πx)\cos(8\pi x) after two. Each step introduces a new frequency. More generally, the doubling map is mixing with respect to Lebesgue measure. Its Koopman operator on L2([0,1],dx)L^2([0,1],dx) is an isometry whose only eigenfunctions are constants. Its finite-dimensional invariant subspaces therefore contain only constant functions.

Now consider the same map acting on the family of distributions νa\nu_a with densities ρa(x)=1+a(2x−1)\rho_a(x)=1+a(2x-1), where 0≤a≤10\leq a\leq1. At a=0a=0 the density is uniform. Larger values of aa place more mass toward the right end of the interval. A particle at yy after one step can come from y/2y/2 or (y+1)/2(y+1)/2, so the new density is 12[ρa(y/2)+ρa((y+1)/2)]=ρa/2(y)\tfrac12[\rho_a(y/2)+\rho_a((y+1)/2)]=\rho_{a/2}(y). Each step halves the slope of the density and keeps it in the same family.

Let H(ν)=112−Var⁡ν(X)H(\nu)=\tfrac1{12}-\operatorname{Var}_\nu(X), the variance of the uniform distribution minus the variance of the current particle distribution. We can estimate HH from the sample variance of a single snapshot. Integrating against ρa\rho_a gives

H(νa)=a236,H(νa/2)=14H(νa).H(\nu_a)=\frac{a^2}{36}, \qquad H(\nu_{a/2})=\frac14 H(\nu_a).

On this family, the expectation of any fixed state observable ϕ\phi, ∫01ϕ(x)ρa(x) dx\int_0^1\phi(x)\rho_a(x)\,dx, is affine in aa. No such expectation can therefore represent H(νa)H(\nu_a), which is quadratic in aa. The nonlinear observable HH spans an exact one-dimensional distributional Koopman invariant subspace. Since a≥0a\geq0, its value uniquely determines a=6H(νa)a=6\sqrt{H(\nu_a)} and hence the full density ρa/2n\rho_{a/2^n} after any number nn of steps.

(b) Encoder approximation of distributional observables

DisKO averages learned sample features before a neural readout (Zaheer et al., 2017). On a weakly compact family of distributions over a compact state space, such encoders with sufficient capacity can uniformly approximate any finite set of continuous observables to arbitrary accuracy (Pevný & Kovařík, 2019).

AuthorsOpenReview ↗

Response to Q1 (Koopman invariance, 2/3) — (c) Convergence to the optimal Koopman projection from finite snapshots

(c) Convergence to the optimal Koopman projection from finite snapshots

Given a set of observables, such as those defined by a fixed encoder, we show when fitting the dynamics from snapshots converges to the optimal Koopman projection. We then use the one-step residual to quantify how close the observable space is to being invariant.

Fix the encoder EθE_\theta. Write Vθ=span⁡{1,Eθ,1,…,Eθ,m}V_\theta=\operatorname{span}\{1,E_{\theta,1},\ldots,E_{\theta,m}\} for its observable space and h(ν)=[Eθ(ν)⊤,1]⊤h(\nu)=[E_\theta(\nu)^\top,1]^\top for the augmented observable vector. Let Π\Pi be a reference probability measure on the family of distributions M\mathcal M. It determines how starting distributions are weighted in the population prediction error. Suppose we observe nn pairs of exact distribution snapshots (νi,TΔνi)(\nu_i,T_\Delta\nu_i) at a fixed lag Δ\Delta. The starting distributions define the empirical measure Πn=1n∑i=1nδνi\Pi_n=\frac1n\sum_{i=1}^n\delta_{\nu_i}. Averaging with respect to Πn\Pi_n gives each pair weight 1/n1/n. The matrix MnM_n minimizes the mean squared one-step error.

Mn=arg⁡min⁡M Eν∼Πn∥h(TΔν)−Mh(ν)∥2.M_n=\arg\min_M\,\mathbb E_{\nu\sim\Pi_n}\lVert h(T_\Delta\nu)-Mh(\nu)\rVert^2.

Thus MnM_n represents the optimal finite-dimensional projection of the true Koopman action on VθV_\theta under Πn\Pi_n, extending the usual Koopman projection to distribution space.

Two sampling issues arise in practice. First, each snapshot gives an empirical distribution based on finitely many samples. For independent, identically distributed samples within a snapshot, the law of large numbers gives convergence of the fixed encoder's output to the encoding of the underlying distribution as the sample size grows. Second, even with exact snapshots, the propagation rule we can identify depends on which starting distributions we observe. Proposition 1 extends the classical projection consistency result for extended dynamic mode decomposition (EDMD) to this setting.

Proposition 1 (Consistency of Empirical Distributional Koopman Projection). Assume that M\mathcal M is compact, EθE_\theta and TΔT_\Delta are continuous, and the following conditions hold.

  1. The empirical measure of the starting distributions converges weakly to the reference measure, Πn⇀Π\Pi_n\rightharpoonup\Pi.
  2. The Gram matrix GΠ=EΠ[h(ν)h(ν)⊤]G_\Pi=\mathbb E_\Pi[h(\nu)h(\nu)^\top] is positive definite.

Then, as n→∞n\to\infty,

M_\Pi^\ast=\mathbb E_\Pi[h(T_\Delta\nu)h(\nu)^\top]G_\Pi^{-1}.$$ Here $M_\Pi^\ast$ represents the $L^2(\Pi)$-orthogonal projection of the true distributional Koopman action on $V_\theta$ back onto $V_\theta$. **Remark (What the Gram condition requires of the snapshots).** More samples within a snapshot give a more accurate estimate of the distribution at that time. Identifying the propagator also requires enough variation across snapshots to distinguish different propagation rules. If all snapshots come from the same stationary distribution $\nu_\star$, even exact observations give the same vector $v=h(\nu_\star)$ each time. Then $G_\Pi=vv^\top$ has rank one, and every matrix satisfying $Mv=v$ fits the data. These snapshots cannot tell us whether a perturbation away from the stationary distribution would persist or decay. Proposition 1 establishes consistency of the fitted projection. Approximate invariance also depends on how far the observables move outside $V_\theta$ under $\mathcal K_\Delta$. The latent consistency loss in Stage 3 measures this through the one-step residual. For the learned propagator $\widehat M_\Delta$, define its population residual by $$\varepsilon_{\mathrm{lat}}^2=\mathbb E_{\nu\sim\Pi}\lVert h(T_\Delta\nu)-\widehat M_\Delta h(\nu)\rVert^2.$$ If $\lambda_{\min}(G_\Pi)\geq\gamma>0$, the invariance error satisfies $$\left\lVert(I-P_{V_\theta})\mathcal K_\Delta\vert_{V_\theta}\right\rVert \leq\frac{\varepsilon_{\mathrm{lat}}}{\sqrt{\gamma}}.$$ Thus when the population residual is small relative to $\sqrt{\gamma}$, the learned space is approximately Koopman-invariant at lag $\Delta$. **Remark.** The projection and operator norm are taken in $L^2(\Pi)$. For any unit-norm observable $f=a^\top h\in V_\theta$, the Gram condition gives $\lVert a\rVert\leq1/\sqrt{\gamma}$. Approximating $\mathcal K_\Delta f$ by $a^\top\widehat M_\Delta h\in V_\theta$ gives error at most $\lVert a\rVert\varepsilon_{\mathrm{lat}}$. The orthogonal projection has at most this error. The same criterion applies to jointly learned observables. To assess closure beyond the training snapshots, one can freeze the model and estimate the residual on held-out pairs representative of the target distributions.
AuthorsOpenReview ↗

Response to Q1 (Koopman invariance, 3/3) — (d) How the theory informs three-stage training

(d) How the theory informs three-stage training

Extending classical Koopman learning theory to distribution space raises questions about the existence of invariant subspaces and gives the Gram condition a concrete interpretation in terms of variation across distributions. These results motivate DisKO's three-stage training procedure.

During representation pretraining in Stage 1, the distribution reconstruction objective encourages the encoder to learn observables that retain enough information to reconstruct a snapshot.

In Stage 2, we fix the encoder and fit Koopman dynamics in the observable space it defines. For the discrete model at a fixed lag, suppose the ridge coefficient in Eq. (25) of the original paper is held fixed and the conditions of Proposition 1 hold. As the number nn of snapshot pairs tends to infinity, the fitted propagator converges to the matrix representing the optimal projection of the distributional Koopman action on this space under the reference measure.

In Stage 3, joint training updates the observables, Koopman dynamics, and decoder together. The latent consistency loss reduces the Koopman closure residual, while the reconstruction and distribution prediction losses encourage the observables to retain the information needed for both tasks.

Reviewer_fRzJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DisKO, a deep Distributional Koopman framework for learning population dynamics from temporally unpaired distribution snapshots. DisKO uses a permutation-invariant set encoder to learn distributional observables, propagates them with finite-dimensional affine Koopman dynamics, and reconstructs future distributions using a conditional flow-matching decoder. The model is trained with prediction, reconstruction, latent-consistency, and distribution-level losses. The paper reports strong extrapolation results on seven benchmarks and additionally evaluates Koopman eigenvalue/eigenfunction recovery on synthetic Circle and Torus systems.

Critical strengths
  1. The problem setting is relevant, and directly modeling distribution dynamics is well motivated for unpaired snapshot data.
  2. The experimental coverage is broad, including synthetic systems, physical simulations, cosmology, and single-cell datasets.
  3. The reported long-horizon forecasting results are strong across several benchmarks.
  4. The affine latent dynamics provide a clean continuous-time propagation mechanism and naturally support irregular observation intervals.
  5. The paper is generally clearly written and provides extensive supplementary material.
Critical weaknesses
  1. The methodological contribution is largely compositional. The main framework combines three established ingredients: permutation-invariant set encoders, affine/deep Koopman dynamics, and conditional flow matching. The integration is sensible, but I do not see a substantially new principle for learning Koopman observables, identifying the generator, or constructing invariant subspaces. The main novelty therefore appears to be applying and combining existing techniques in distribution space.

  2. The trajectory-identifiability motivation is not sufficiently connected to the empirical evidence. The introduction emphasizes that marginal distributions do not uniquely determine individual trajectories and uses this to motivate modeling distributions directly. However, the experiments do not specifically demonstrate when avoiding trajectory/coupling reconstruction provides an advantage over these alternatives. Moreover, Koopman modeling itself does not resolve the underlying observation-identifiability issue, which the paper explicitly acknowledges in Appendix G.3.

  3. It is difficult to attribute the strong prediction results specifically to Koopman dynamics. The complete objective is relatively complex, combining prediction and reconstruction flow matching, latent consistency, sliced Wasserstein, MMD, and moment matching, with several dataset-dependent weights and architecture choices. The conditional flow decoder is itself highly expressive. Consequently, the current results do not clearly separate gains from Koopman structure from gains due to decoder capacity, auxiliary losses, or optimization choices. Although the paper states that only one to three configurations were tested per dataset, a clearer account of model capacity, compute, and tuning budgets relative to baselines would improve comparison fairness.

  4. The spectral contribution remains restricted to toy examples. Eigenvalue and eigenfunction recovery is demonstrated on Circle and Torus, which are specifically constructed to have analytically known Fourier Koopman spectra. This is a useful sanity check, but no comparable analysis is provided on realistic datasets. In particular, the paper does not show whether learned Koopman modes on the single-cell datasets correspond to meaningful biological transitions, differentiation programs, or characteristic timescales. Thus, the practical value of the claimed spectral access remains unclear.

  5. The ablation study is too limited for the complexity of the proposed framework. The main Koopman-vs-Neural-ODE comparison is performed on OU, which is an especially favorable setting because the paper itself shows that Gaussian OU dynamics admit an exact finite-dimensional affine closure. This comparison should be repeated on more nonlinear and realistic systems. More importantly, there is no systematic ablation of JpredJ_{\mathrm{pred}}, JrecJ_{\mathrm{rec}}, JlatJ_{\mathrm{lat}}, and JdistJ_{\mathrm{dist}}. Without such experiments, it remains unclear whether the Koopman-related latent-consistency constraint is actually essential to the reported performance.

Questions for authors
  1. How much performance remains when the main architectural and objective components are ablated one at a time—including encoder/decoder capacity, latent dimension, Koopman dynamics, training stages, JpredJ_{\mathrm{pred}}, JrecJ_{\mathrm{rec}}, JlatJ_{\mathrm{lat}}, JdistJ_{\mathrm{dist}}, and the components within JdistJ_{\mathrm{dist}}—while keeping the rest of the pipeline fixed?
  2. Does the affine Koopman model still outperform matched Neural ODE or nonlinear latent dynamics on Duffing, Boids, or the single-cell datasets?
  3. Since DisKO does not resolve the fundamental non-identifiability of dynamics from marginal snapshots, can the authors clarify what concrete advantage is gained by avoiding explicit trajectory or coupling reconstruction?
  4. Can the learned Koopman spectrum be interpreted on at least one realistic dataset, particularly Pancreas or WOT iPSC?
  5. Can the authors report parameter counts, total training compute, and hyperparameter-search budgets for DisKO and the main baselines?
18 comments
AuthorsOpenReview ↗

Response to Q1 (Ablations, 1/4) — TL;DR and (a) Loss ablations

Thank you for asking about the ablations. The additional experiments give a more complete and convincing account of how each component contributes to DisKO's performance.

TL;DR. We ablated loss terms, model components, and training stages to measure how much extrapolation performance DisKO retained under each change. DisKO was generally robust to removing individual loss terms during joint training. On all three datasets, every such variant had lower mean extrapolation W1 and SW1 than the best baseline reported for that dataset in the original submission. Removing the prediction loss JpredJ_{\mathrm{pred}} had the largest effect, raising extrapolation SW1 on WOT-iPSC by 24.7% (a). On OU, Duffing, and WOT-iPSC, replacing the conditional flow matching decoder with a conditional Gaussian mixture model preserved DisKO's extrapolation advantage over the best baseline for each dataset (b). With the total number of updates held fixed, three-stage training yielded lower mean extrapolation SW1 than joint training from scratch on all five datasets (c).

(a) Loss ablations

We ran each configuration on Duffing, WOT-iPSC, and Boids with three training seeds per dataset. For each dataset and seed, all variants shared the weights learned in the first two stages. During joint training, we set one loss weight to zero at a time. DisKO was generally robust to removing individual loss terms. Most changes in extrapolation error were small and comparable to the variation across training seeds.

The largest relative increases among the four main loss ablations occurred on WOT-iPSC, where removing the prediction loss JpredJ_{\mathrm{pred}} raised mean extrapolation W1 by 8.1% and SW1 by 24.7%. On each of the three datasets, removing JrecJ_{\mathrm{rec}}, JlatJ_{\mathrm{lat}}, or JdistJ_{\mathrm{dist}} individually changed mean W1 by less than 3% and SW1 by less than 5% relative to the full model (Table 1.1). On Boids, removing JlatJ_{\mathrm{lat}} led to small increases in both errors for all three seeds.

These results are consistent with the roles of the three training stages. In Stage 1, the decoder learns to reconstruct a snapshot's distribution from its encoding. In Stage 2, we fix the encoder and fit affine dynamics to the observables by matching the propagated encoding to the target encoding. Removing JrecJ_{\mathrm{rec}} or JlatJ_{\mathrm{lat}} during joint training thus preserves the corresponding pretraining. Stage 3 introduces JpredJ_{\mathrm{pred}}, which uses samples from the target snapshot to train the decoder's flow field conditioned on the propagated latent state. At inference, the decoder likewise receives a latent state propagated from an earlier snapshot. Removing JpredJ_{\mathrm{pred}} eliminates flow matching supervision under that condition.

DisKO retained its forecasting advantage after any one loss term was removed, with lower mean extrapolation W1 and SW1 than the best baseline reported for each of the three datasets. Bold marks the lowest mean error in each column. Underlining highlights selected results.

Table 1.1. Extrapolation error after removing individual loss terms during joint training. DisKO results are means ± standard deviations over three training seeds.

Model / AblationDuffing W1 ↓Duffing SW1 ↓WOT-iPSC W1 ↓WOT-iPSC SW1 ↓Boids W1 ↓Boids SW1 ↓
DisKO (Full)0.1539 ± 0.00540.0613 ± 0.005017.611 ± 0.2770.8138 ± 0.06151.4066 ± 0.08940.6115 ± 0.0712
w/o JpredJ_{\mathrm{pred}}0.1531 ± 0.00490.0619 ± 0.003719.031±0.629‾\underline{19.031 \pm 0.629}1.0146±0.0773‾\underline{1.0146 \pm 0.0773}1.4351±0.2952‾\underline{1.4351 \pm 0.2952}0.6335±0.2381‾\underline{0.6335 \pm 0.2381}
w/o JrecJ_{\mathrm{rec}}0.1546 ± 0.01020.0622 ± 0.008517.686 ± 0.2820.8104 ± 0.05681.3877 ± 0.07790.5961 ± 0.0623
w/o JlatJ_{\mathrm{lat}}0.1543 ± 0.00490.0616 ± 0.004617.611 ± 0.2820.8131 ± 0.06081.4284 ± 0.08580.6302 ± 0.0668
w/o JdistJ_{\mathrm{dist}}0.1583 ± 0.01070.0642 ± 0.008817.314 ± 0.2080.7863 ± 0.03521.3860 ± 0.11440.5967 ± 0.0896
Best baseline0.318 ± 0.028 (WLM)0.160 ± 0.019 (WLM)23.328 ± 0.086 (PRESCIENT)1.883 ± 0.013 (PRESCIENT)1.595 ± 0.047 (WLM)0.656 ± 0.050 (WLM)
AuthorsOpenReview ↗

Response to Q1 (Ablations, 2/4) — (a) Loss ablations (continued) and (b) Model component ablations

We also removed the SW1, MMD², and moment-matching terms from JdistJ_{\mathrm{dist}} one at a time. Relative to the full composite loss, mean extrapolation W1 changed by less than 2% and SW1 by less than 4% on each dataset (Table 1.2).

Table 1.2. Extrapolation error after removing individual components of the endpoint distribution loss. DisKO results are means ± standard deviations over three training seeds.

Model / AblationDuffing W1 ↓Duffing SW1 ↓WOT-iPSC W1 ↓WOT-iPSC SW1 ↓Boids W1 ↓Boids SW1 ↓
DisKO (Full)0.1555 ± 0.00550.0625 ± 0.004917.611 ± 0.2770.8138 ± 0.06151.4066 ± 0.08940.6115 ± 0.0712
w/o SW10.1583 ± 0.01010.0643 ± 0.008517.301 ± 0.1750.7825 ± 0.03411.4014 ± 0.09740.6106 ± 0.0731
w/o MMD²0.1553 ± 0.00560.0623 ± 0.005017.617 ± 0.2890.8142 ± 0.06101.3963 ± 0.08470.6037 ± 0.0677
w/o Moment0.1559 ± 0.00570.0627 ± 0.005117.552 ± 0.2660.8099 ± 0.05681.4075 ± 0.08750.6138 ± 0.0698
Best baseline0.318 ± 0.028 (WLM)0.160 ± 0.019 (WLM)23.328 ± 0.086 (PRESCIENT)1.883 ± 0.013 (PRESCIENT)1.595 ± 0.047 (WLM)0.656 ± 0.050 (WLM)

(b) Model component ablations

On Duffing and WOT-iPSC, we varied one aspect of model capacity at a time. We halved or doubled the width of either the encoder or the decoder, or changed the latent dimension from its default of 64 to 32 or 128 (Table 1.3). Relative to the default model, the largest increases in mean extrapolation W1 were 7.1% on Duffing and 5.7% on WOT-iPSC. Some variants improved on the default on individual metrics, suggesting room to improve accuracy by choosing the capacity for each task. On both datasets, every configuration had lower mean extrapolation W1 and SW1 than the best baseline reported for that dataset.

Table 1.3. Effect of encoder width, decoder width, and latent dimension on extrapolation.

ConfigurationDuffing W1 ↓Duffing SW1 ↓WOT-iPSC W1 ↓WOT-iPSC SW1 ↓
DisKO (Full)0.1569 ± 0.00830.0638 ± 0.007217.6000 ± 0.30790.8148 ± 0.0612
Encoder width ×0.50.1566 ± 0.00890.0635 ± 0.007717.8090 ± 0.31940.8511 ± 0.0366
Encoder width ×20.1642 ± 0.02000.0692 ± 0.015418.1245 ± 0.37780.8632 ± 0.0367
Decoder width ×0.50.1508 ± 0.01040.0591 ± 0.008618.5254 ± 0.23400.7931 ± 0.0423
Decoder width ×20.1680 ± 0.01180.0719 ± 0.009218.5433 ± 0.54010.9874 ± 0.0687
Latent dim. = 320.1624 ± 0.01120.0677 ± 0.008917.9023 ± 0.13600.8780 ± 0.0383
Latent dim. = 1280.1555 ± 0.00850.0623 ± 0.006818.6060 ± 0.54050.9379 ± 0.0661
Best baseline0.318 ± 0.028 (WLM)0.160 ± 0.019 (WLM)23.328 ± 0.086 (PRESCIENT)1.883 ± 0.013 (PRESCIENT)
AuthorsOpenReview ↗

Response to Q1 (Ablations, 3/4) — (b) Model component ablations (continued)

DisKO represents each distribution by the values of a set of observables, learns their linear evolution in augmented coordinates, and decodes the propagated coordinates into a predicted distribution. The encoder and decoder can each be replaced within this framework. To assess the encoder's contribution, we kept the latent dimension and the architectures of the dynamics and decoder fixed and replaced the features learned by DeepSets with one of two fixed feature dictionaries. The first uses statistics such as the mean and covariance. The second combines multiscale kernel features with the average soft assignments of the samples in each snapshot to the k-means centroids. Both use a trainable linear readout (Table 1.4).

For linear OU dynamics, the mean and covariance obey a known closed system of equations, providing a basis for choosing distributional observables in advance. With the default 2,500 joint training steps, DeepSets had the lowest mean extrapolation W1 and SW1 on OU. When we increased the joint training steps to 5,000 for all three models, the kernel dictionary had the lowest mean error on both metrics. The kernel dictionary also had the lowest errors on Duffing. On WOT-iPSC, where no closed set of observables is known a priori for forecasting, DeepSets had the lowest mean extrapolation errors.

Table 1.4. Learned vs. fixed distributional features. Each entry gives W1 / SW1. The column headings indicate the number of joint training steps.

EncoderOU · 2,500 (default)OU · 5,000Duffing · 15,000WOT-iPSC · 4,000
DeepSets (Full)0.2560 ± 0.0121 / 0.1393 ± 0.00980.2365 ± 0.0313 / 0.1242 ± 0.02480.1569 ± 0.0083 / 0.0638 ± 0.007217.6000 ± 0.3079 / 0.8148 ± 0.0612
Fixed statistical features0.7523 ± 0.1005 / 0.4708 ± 0.06550.5243±0.1111/0.3196±0.0741‾\underline{0.5243 \pm 0.1111 / 0.3196 \pm 0.0741}0.1543 ± 0.0044 / 0.0635 ± 0.004418.3532 ± 0.8260 / 0.9247 ± 0.1158
Fixed kernel features + soft cluster assignments0.3171 ± 0.0371 / 0.1817 ± 0.02580.2055±0.0058/0.1014±0.0033‾\underline{\boldsymbol{0.2055 \pm 0.0058 / 0.1014 \pm 0.0033}}0.1431 ± 0.0017 / 0.0529 ± 0.001517.6973 ± 0.4613 / 0.8444 ± 0.0784
Best baseline0.602 ± 0.032 / 0.362 ± 0.020 (WLM)0.602 ± 0.032 / 0.362 ± 0.020 (WLM)0.318 ± 0.028 / 0.160 ± 0.019 (WLM)23.328 ± 0.086 / 1.883 ± 0.013 (PRESCIENT)

To test how much DisKO's extrapolation advantage depends on its conditional flow matching (CFM) decoder, we replaced it with a conditional Gaussian mixture model (GMM). We kept the encoder architecture and the linear latent dynamics parameterization fixed and used the GMM negative log-likelihood for both prediction and reconstruction. On OU, Duffing, and WOT-iPSC, DisKO with the GMM decoder still had lower mean extrapolation W1 and SW1 than the best baseline reported for each dataset in the original submission (Table 1.5).

The benefit of CFM emerged beyond the training window. We forecast from the first snapshot of each sequence, propagating continuously to each evaluation time. Within the training window, the GMM decoder had lower mean W1 and SW1 than CFM on all three datasets. Beyond that window, CFM had lower errors on both metrics. DisKO retained its advantage over the baselines after the decoder was replaced, and joint training with CFM further improved forecasts of future distributions.

Table 1.5. Effect of decoder architecture on prediction within and beyond the training window.

DatasetDecoderTraining-window W1 ↓Training-window SW1 ↓Extrapolation W1 ↓Extrapolation SW1 ↓
OUCFM (Full)0.2763 ± 0.00120.1320 ± 0.00110.2560 ± 0.01210.1393 ± 0.0098
GMM0.2593 ± 0.00250.1284 ± 0.00100.3449±0.0293‾\underline{0.3449 \pm 0.0293}0.2047±0.0199‾\underline{0.2047 \pm 0.0199}
Best baseline (WLM)0.709 ± 0.0060.428 ± 0.0030.602 ± 0.0320.362 ± 0.020
DuffingCFM (Full)0.1015 ± 0.00230.0372 ± 0.00180.1569 ± 0.00830.0638 ± 0.0072
GMM0.1012 ± 0.00500.0359 ± 0.00360.2412±0.0105‾\underline{0.2412 \pm 0.0105}0.1206±0.0069‾\underline{0.1206 \pm 0.0069}
Best baseline (WLM)0.116 ± 0.0040.042 ± 0.0020.318 ± 0.0280.160 ± 0.019
WOT-iPSCCFM (Full)16.7357 ± 0.09340.5882 ± 0.017417.6000 ± 0.30790.8148 ± 0.0612
GMM15.7502 ± 0.04700.2288 ± 0.024621.6508±1.3242‾\underline{21.6508 \pm 1.3242}1.3130±0.2339‾\underline{1.3130 \pm 0.2339}
MFA14.9659 ± 0.06580.2046 ± 0.016420.8675±0.8758‾\underline{20.8675 \pm 0.8758}1.2625±0.1335‾\underline{1.2625 \pm 0.1335}
Best baseline (PRESCIENT)25.992 ± 0.0462.379 ± 0.00823.328 ± 0.0861.883 ± 0.013

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.