- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes DOODL, a framework for learning a representation of a population of related dynamical systems using Koopman operators, under the assumption that estimated Koopman operators of systems share a low-dimensional manifold. Its key components include: (1) Spectral decompositions are treated as points on a smooth manifold, quotiented by eigenvector rescaling, and equipped with a scale-invariant Riemannian metric and a retraction. (2) Dictionary learning: atoms and simplex codes are obtained by minimizing the SGOT divergence between each operator and its reconstruction B^p, which is a convex combination of the atoms’ eigenvalues and eigenvectors followed by a bi-orthogonality projection. (3) Dictionary-constrained estimation: from a new short trajectory, only the code is fit via a one-step feature-prediction loss. Theorem 1 bounds the excess risk with a complexity term in the intrinsic dimension rather than the feature dimension.
Critical strengths
The problem is well motivated and studies an interesting problem. Population-level priors for operator learning are understudied in the literature, and the paper makes a meaningful conceptual contribution towards this goal. The idea of learning a population-level prior directly on spectral decompositions is also interesting and useful in practice. The proposed part pipeline is sound and also constitutes an innovative technical contribution.
Critical weaknesses
(1) The central hypothesis is that operators of related systems concentrate near a low-dimensional manifold, and that convex combinations of atoms remain physically meaningful. However, for a parametrized family of SDEs, how strong is this assumption? When a parameter crosses a bifurcation, for example, a single-well dynamics becomes a double well after parameter changes, a second near-zero eigenvalue separated by the spectral gap appears. In its current form of the paper, the estimator always returns a point in the convex hull of the atoms, and nothing flags a system that lies off the learned manifold. Some intuition or experimental checks would be helpful. (2) Koopman estimation is often characterized w.r.t where is the invariant measure of the system. Appendix A.1 states its error bounds in exactly this space. However, in a population, every system has its own invariant measure . It would be helpful to clarify this. (3)Theorem 1 assumes a stationary test trajectory. Section 4, however, advertises operator estimation from short non-equilibrium trajectories, and windows of 10–100 samples are unlikely to be sampled according to the invariant measure. The guarantee therefore does not cover the regime the experiments highlight. (4) The paper takes the per-system spectral decompositions as given. Spectral estimation from data, however, is a hard problem itself, as finite-rank truncations can produce spurious eigenvalues, known as spectral pollution, that do not correspond to the underlying operator. The paper cites ResDMD, it would be helpful to discuss how robust the method is. (5) The Langevin dynamics is a one-parameter, one-dimensional family. A more convincing evaluation would include a multidimensional or non-reversible system. The text also missed a dt in the description of the SDE.
Questions for authors
(1) Under what conditions on the parameter family is the manifold hypothesis expected to hold? How does DOODL behave on a system that lies off the learned manifold? (2) Theorem 1 assumes a stationary test trajectory, while the experiments target short non-equilibrium windows. Can the guarantee be extended to non-stationary trajectories? (3) How sensitive are the learned atoms to spurious eigenvalues?
6 comments
Q1: Formalism of related dynamical systems
- The central hypothesis is that operators of related systems concentrate near a low-dimensional manifold, and that convex combinations of atoms remain physically meaningful. However, for a parametrized family of SDEs, how strong is this assumption? When a parameter crosses a bifurcation, for example, a single-well dynamics becomes a double well after parameter changes, a second near-zero eigenvalue separated by the spectral gap appears.
- Under what conditions on the parameter family is the manifold hypothesis expected to hold?
We thank the reviewer for raising bifurcations, which are the right stress test for the hypothesis. The new Appendix G.5 (general comment R3) gives sufficient conditions: smooth dependence on the parameters, and a uniform gap between the leading eigenvalues and from the rest of the spectrum (Proposition 9). We clarify below what this implies for reviewer's example, and where the hypothesis genuinely fails.
Single-well to double-well. A bifurcation of the potential does not, by itself, create a singularity in operator space. For one-dimensional reversible diffusions, the generator is a Sturm–Liouville operator: its eigenvalues are simple, cannot cross, and depend analytically on the parameters of the potential. The near-zero eigenvalue you describe is the first non-trivial eigenvalue , which decreases continuously and becomes exponentially small in the barrier height (Kramers regime), rather than appearing at the bifurcation. For with and , decreases smoothly from to , while the relative gaps among the first four eigenvalues stay above (Table 7). The population therefore remains a smooth curve. Its curvature, however, concentrates around the transition, where more atoms are needed: the constants of Proposition 9 scale with the inverse gap.
Our Langevin family. For , varying is a pure time rescaling, with the spatio-temporal rescaling (Proposition 10). Hence and , and the population is an exact one-dimensional curve. On your question of whether convex combinations remain physically meaningful: in generator (time-scale) coordinates, any convex combination of the atoms' eigenvalues is exactly the spectrum of another member of the family, with . The approximation error then only stems from the eigenfunctions, and is .
Where your concern applies. We agree that the hypothesis is not automatic, and Appendix G.5 now lists the cases where the gap condition fails:
- crossings in multi-parameter families, where eigenvectors have non-trivial monodromy and our ordering-dependent decoder needs atoms on each side;
- exceptional points in non-reversible families, e.g. two real eigenvalues merging into a complex pair at the onset of oscillations;
- an eigenvalue entering the retained rank- block, e.g. when a third metastable state appears.
In these regimes, DOODL returns the closest point of the learned manifold, and the remaining error does not decrease with the trajectory length (Remark 6). We discuss how to flag such systems from data in our answer to Q2 and in general comment R2.
Q2: DOODL behavior for operators outside of the learned manifold
- In its current form of the paper, the estimator always returns a point in the convex hull of the atoms, and nothing flags a system that lies off the learned manifold. Some intuition or experimental checks would be helpful.
- How does DOODL behave on a system that lies off the learned manifold?
We thank the reviewer for encouraging us to discuss the behavior of DOODL when estimating an operator outside of the learned manifold. In the general response R2, we provide a detailed discussion and an additional experiment on that topic.
With a dictionary of atoms , the DOODL spanning manifold is the set of all operators reconstructed from coefficients on the simplex.
For a new operator , DOODL returns an operator on the envelope of that is closest to in Hilbert–Schmidt norm. Thus, for a system outside the spanning manifold, the estimate is biased by a term proportional to the Hilbert–Schmidt distance from to .
We illustrated DOODL's behavior when estimating new operators outside the spanning manifold in the Langevin experiment. The results in R2 show that for systems outside the spanning manifold, the coefficients concentrate on one corner of the 1D manifold, and the Hilbert–Schmidt error increases as the operator gets further away from the spanning manifold.
Q3: Estimation from short non-stationary trajectories
Theorem 1 assumes a stationary test trajectory, while the experiments target short non-equilibrium windows. Can the guarantee be extended to non-stationary trajectories?
Theorem 1 assumes a stationary test trajectory. Section 4, however, advertises operator estimation from short non-equilibrium trajectories, and windows of 10–100 samples are unlikely to be sampled according to the invariant measure. The guarantee therefore does not cover the regime the experiments highlight.
The new Appendix G.4 extends Theorem 1 to test trajectories started from an arbitrary initial distribution for geometrically ergodic chains satisfying
Proposition 8 retains the stationary risk as the target and adds a term of order
See Eq. (139). The proof retains the -mixing blocking argument of Theorem 1, which also applies to non-stationary trajectories. The empirical risk is now centered around a time-averaged risk, and the block variances also change. Geometric ergodicity controls both changes. The additional term vanishes as grows and is most relevant for windows shorter than the mixing time . The results for the shortest windows in the regime-switch experiment (Figure 3) illustrate the theory empirically.
W1: Clarification on invariant measure
The per-system spaces Lπi2 only appear in Appendix A.1, which recalls the guarantees for estimating each training operator individually. DOODL itself does not compare operators in these spaces. By assumption (A1), all operators act on a common finite-dimensional feature space H (e.g. shared Random Fourier Features). The SGOT loss compares their spectral decompositions in the geometry of H, which does not depend on any πi. In Theorem 1, the only measure involved is the invariant measure π of the test system, through the population risk Eπ[ℓt(G)]. The training systems enter only through the learned dictionary, which is fixed and independent of the test trajectory.
W2: Langevin increase
We volontarily kept the Langevin example simple so that we can better interpret the embeddings. We would like to highlight that we did a very high dimensional and non-reversible system as asked by the reveiwer with the plasma experiments and we did recover quite well physical parameters from the DOODL embeddings.