acceptodds
Withdrawn from ICLR 2027

Scalable Learning of Drift and Diffusion in High-Dimensional Stochastic Differential Equations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Stochastic differential equations (SDEs) provide a fundamental framework for modeling high-dimensional stochastic dynamical systems. Their drift and diffusion functions govern the deterministic dynamics and stochastic fluctuations of the underlying system, respectively. Learning these functions from observed data is crucial for understanding and predicting system behavior. Despite the importance of high-dimensional SDEs () in practical applications, existing methods mostly rely on prior knowledge of the functional forms or are difficult to scale to the high-dimensional case because of prohibitive time and space complexity. To address these limitations, we propose KoopSDE, an algorithm that uses the stochastic Koopman operator to learn drift and diffusion functions from a single discretely observed trajectory without prior knowledge of functional forms. To address the curse of dimensionality, we incorporate Hutchinson trace estimation (HTE), reducing the time and space complexity from to . We also establish a set of theoretical guarantees for KoopSDE. Experiments demonstrate that KoopSDE can learn -dimensional SDEs on a single NVIDIA A800 GPU. This work opens up the possibility for learning the drift and diffusion functions of high-dimensional SDEs.

Primary area:
learning on time series and dynamical systems
Keywords:
Stochastic Differential Equations, Stochastic Koopman Operator, Hutchinson Trace Estimation, Neural Network

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7JsJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes KoopSDE, a two-stage framework for learning drift and diffusion functions of high-dimensional stochastic differential equations. It combines stochastic Koopman eigenfunction learning with Hutchinson trace estimation to reduce computational complexity. Experiments on synthetic systems and real-world traffic data demonstrate scalability up to 100,000 dimensions with reasonable estimation accuracy.

Critical strengths

The proposed framework effectively combines Koopman-based learning with efficient trace estimation, enabling SDE coefficient learning at dimensions that are computationally prohibitive for the evaluated baselines. The experiments demonstrate substantial memory savings while maintaining reasonable estimation accuracy.

Critical weaknesses

Although KoopSDE learns drift and diffusion in the original high-dimensional space, it relies on a limited number of Koopman eigenfunctions and a low-rank-plus-diagonal diffusion parameterization. It remains unclear whether these representations sufficiently capture complex high-dimensional dynamics, or what practical advantages they offer over directly learning lower-dimensional latent SDE dynamics. The current experiments do not adequately establish this distinction.

Questions for authors

Could the authors further clarify the advantages of recovering drift and diffusion in the original high-dimensional space over using latent SDE models? A comparison on a common forecasting task, together with experiments involving more complex high-dimensional dynamics, would help establish the practical benefits and limitations of the proposed approach.

Reviewer_vXNjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper tackles the problem of learning drift and diffusion functions in stochastic differential equations (SDEs) when the dimension of the state is very high (d>>100d>>100). This paper utilizes concepts from the stochastic Koopman operator and tries to train MLPs with two different losses that can approximate the drift and diffusion well. Using this training, it can learn up to 100,000-dimensional SDEs, which many other methods fail to do due to out-of-memory issues.

Critical strengths
  • I think the motivation of the paper is good and the idea of using the stochastic Koopman operator has its novelty.
Critical weaknesses
  • I think the paper is a bit blurry in explaining the approximation using MLPs. The paper conveys the story as if everything still lies in the theoretical regime even after approximation but I'm not fully convinced of that. Also, we need more discussion on what kind of MLPs the paper used. Also, without theoretical analysis of the approximations using the specific MLPs used, it is hard to fully believe this method will work universally.
Questions for authors
  • In line 46. Can you add a definition for small xx or xlx_l? I'm not sure what the bold variable means here.
  • After Eq. 5, what is ϕ~(x;θ)\tilde{\phi}(x;\theta)? Is it just a neural network-parameterized version of the ϕ\phi defined above?
  • When you parameterize using an MLP, ϕ~k\tilde{\phi}_k is not in L2L^2, no? Can you still use the logic to get Eq. 6 to parameterize Λ\Lambda as well after that?
  • What happens when Ldyn\mathcal L_{dyn} is not zero? In practice, there is always an approximation error. In that case, how should we still interpret the meaning of ϕ~\tilde{\phi} and Λ\Lambda?
  • In line 222. Continuing from the previous question, is Φ~∈L2\tilde{\Phi}\in L^2?
  • In lines 224- 225. Are you assuming KτΦ~(xl;θ)=Φ~(xl+1;θ)\mathcal K^\tau \tilde{\Phi}(x_l;\theta) = \tilde{\Phi}(x_{l+1};\theta) here without any expectation?
  • The loss function in line 211 cannot be exactly the same as the loss function defined in line 235, no? Since the left-hand side of the two equations is exactly the same, are you assuming the two right-hand sides are exactly equal?
  • Do you have theoretical analysis on how MLPs will converge to the optimal ϕ\phi, except just mentioning the universal approximation theorem? I'm not fully convinced that any MLPs can easily approximate those functions/values.
  • For tasks like PEMS-BAY (which I think is the only real-data), what is the reason we need to use KoopSDE over any NeuralSDE method?
Reviewer_iNLcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

KoopSDE learns the drift and diffusion term of SDE from a single discretely observed trajectory. It first learns functions that describe the system's evolution, then uses these functions to fit the underlying coefficients. Theoretically, the analysis covers the trace estimator and eigenvalue estimation when the eigenfunctions are given. Empirically, a structured diffusion model and stochastic trace estimation reduce the memory and computation required for training. Synthetic experiments reach 10510^5 dimensions, and a traffic experiment provides an additional application.

Critical strengths
  1. The paper makes a substantial computational contribution. Combining a structured diffusion model with stochastic trace estimation avoids storing large matrices and makes experiments at 100,000 dimensions feasible.

  2. The presentation of this paper is good.

Critical weaknesses
  1. It is not sufficiently clear why the first stage learns the eigenfunctions needed by the second stage. Fitting each function's one-step evolution and encouraging orthogonality do not generally establish this for the SDEs considered.

  2. The paper does not adequately explain why fitting a small number of learned functions should recover the high-dimensional drift and diffusion accurately. It would be better to add a theoretical guarantee to elucidate when a small training loss implies accurate coefficients.

Questions for authors

Please see the weakness part.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.