acceptodds
Under review as a conference paper at ICLR 2027

Identifiability Guarantees for Drivers and Dynamics of Delayed Physical Systems

Abstract

Learning the dynamics of a physical system directly from observations is a key problem in natural sciences, where physical consistency and interpretability are essential. A wide range of methods have been proposed, including physics-informed neural networks, which are powerful but do not guarantee identifiability of the dynamics, symbolic regression, which requires a set of precomputed operations, and causal discovery, which is more principled but usually relies on strong assumptions that physical systems may violate. In this work, we develop a theory-grounded method and prove that under a set of permissive assumptions, the structural drivers and drift of stochastic delayed differential equations are identifiable. Our method outperforms others on a benchmark for driver identifiability, and on a second benchmark to evaluate physical consistency of the learned dynamics.

Primary area:
learning on time series and dynamical systems
Keywords:
Dynamical systems, Identifiability, Climate science, Theoretical guarantees

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_7egFOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The authors propose a method for the identification of systems driven by stochastic delayed differential equations. They analyze identifiability conditions and compare their method against various competing baselines on two benchmarks.

Critical strengths

I found the paper very well written. It is relatively easy to follow despite the complex topic. The required assumptions are clearly stated, and the paper discusses how realistic they are. Overall, this appears to be a significant contribution that should interest many people in the community.

Critical weaknesses
  1. The results in Table 1, especially the upper part (SHD), don't seem that impressive. Several methods outperform the proposed identification method. I think this requires more attention in the results discussion. Given that the AUROC/AUPRC results are quite clear, the method still seems to perform well overall. However, explaining why it doesn't perform as clearly in terms of SHD could be insightful; the authors could also explain why it seems superior in some systems and less so in others.

  2. Some of the assumptions are relatively strong. This may be necessary, but the authors also clearly criticize the causality community for being able to derive guarantees only under restrictive assumptions. This could be phrased more weakly.

Questions for authors
  1. How would one find a good λ\lambda in practice?

  2. How restrictive is the invertibility assumption on the diffusion matrix? I can easily imagine a system where only some states are affected by noise. Then the matrix would be rank deficient.

  3. Could you explain your choice of baselines?

5 comments
AuthorsOpenReview ↗

Response to reviewer 7egF

Thank you for noting that the paper is very well written and relatively easy to follow despite the complex topic, and that it is a significant contribution that should interest many people in the community.

We would like to respond here to the weaknesses and questions that you raised.

W1. Our method does not significantly outperform all other methods on all evaluated datasets, but consistently ranks best and outperforms other methods on a large number of datasets and metrics. One reason why it appears to perform much better on AUROC and AUPRC than on SHD is that the selected hyperparameters often lead to too dense graphs in the "Coupled" datasets, meaning that AUROC (ranking among edges) can be very high while SHD performance is low because no edge is removed. This is because we follow the procedure of the CausalDynamics benchmark and use a single set of hyperparameters for all systems, although different hyperparameters would probably perform best on different systems, such as a simplified 10-dimensional coupled ocean-atmosphere model or the 3-dimensional Lorenz system.

W2. We agree that the comparison with prior causal-discovery work should be more balanced. Our goal is to establish identifiability for SDDEs with potential instantaneous and lagged drift dependencies, including cyclic interactions. We do not want to claim that our assumptions are uniformly weaker than those used in existing causal-discovery methods; rather, we argue that they may be more physically plausible. In particular, we had in mind the assumption that instantaneous connections must form a Directed Acyclic Graph (DAG), which is made by several methods (e.g., DYNOTEARS, VARLiNGAM, PCMCI+, GRaSP) but does not fit many physical systems.

We propose the following revisions:

  • In the abstract:
    • Before: "causal discovery, which is more principled but usually relies on strong assumptions that physical systems may violate"
    • After: "causal discovery, whose guarantees rely on assumptions about the graph and data-generating process that physical systems may violate"
  • In paragraph 4 of the introduction:
    • Before: "To identify the causal graph, these methods typically rely on strong constraints such as acyclicity or no instantaneous connections, which physical systems often violate."
    • After: "To identify the causal graph, some methods impose constraints on instantaneous connections such as acyclicity, which physical systems often violate."

Q1. In practice, when modelling a system, one could set the parameters of the model to reflect prior knowledge of the system. Depending on the system, one might have some prior knowledge of the true sparsity, which would allow setting λ\lambda accordingly. In that case, we also provide a Cooper [3] implementation of our method, which uses a constraint instead of a penalty, so that one can set the desired level of sparsity directly rather than setting λ\lambda. Also, depending on the task, one may want to sparsify more (to retain only the main drivers) or less (make sure there are no false negatives).

Q2. Invertibility of the noise is assumed throughout the paper for simplicity, but we believe it can be relaxed to a rank-deficient Σ\Sigma. The condition on the noise is only used twice: for full support of the joint occupation measure (Lemma 1) and for the ergodicity result (Lemma 5, used for Theorem 3).

  • Lemma 1. The inverse of the noise, Σ−1\Sigma^{-1}, is used to build the control that drives the process along a pilot path. If we use a pseudo-inverse instead, the proof carries over verbatim.
  • Lemma 5. Σ−1\Sigma^{-1} is used to steer one copy of the process onto another (which cannot be done in noise-free coordinates). We think this could be handled using the asymptotic coupling form of Harris's theorem from the paper on which we already build the proof. This would hold provided that the noise-free coordinates contract once the noisy coordinates agree via Girsanov, in the spirit of the hypoellipticity and controllability conditions of [4].

We will include the extension of Lemma 1 in the revision, and upgrade the ergodicity result once we confirm that a proof can hold without complicating the existing one too much.

Q3. For the CausalDynamics benchmark and driver identification, we report results for the main causal discovery methods. For both CausalDynamics and Dysts, we compare our method to the Neural ODE variants that use a sparsity penalty for regularization or performing driver selection. Among all baselines, only AGL provides an identifiability result for SDEs (not for delayed systems). Since regular NODE does not perform driver selection, we do not include it in the CausalDynamics benchmark.

[3] Cooper: A Library for Constrained Optimization in Deep Learning, Jose Gallego-Posada et al., NeurIPS 2025

[4] Ergodicity for SDEs and approximations: Locally Lipschitz vector fields and degenerate noise, J. Mattingly et al., 2002

Thank you for the replies; those mostly address my concerns. It would be great if the upgraded ergodicity result could be included before the rebuttal ends. However, also without it, I remain positive about the paper.

Dear reviewer 7egF, thank you very much for being positive about the paper.

We've worked on the upgraded ergodicity result, and have updated the pdf accordingly. We haven't proved the finite-sample identifiability theorem under degenerate noise yet, so we just added a section in the appendix (Section A.9, in blue in the pdf) where we show ergodicity under degenerate noise. The details of the results and proofs can be found there.

Reviewer_4ipJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work studied driver and drift identification for SDDEs, aiming to learn interpretable and physically consistent models of delayed systems such as climate dynamics. Their insight is that, under non-degenerate noise, an L0-penalized sparse mask over instantaneous and lagged inputs makes both the drivers and the drift identifiable, without acyclicity assumptions. To address the lack of guarantees for delayed systems, they proposed L0-NDDE, masked neural drifts trained with a relaxed L0 penalty, and proved well-posedness, population and finite-sample identifiability, and ergodicity. In the experiment, they showed the best average rank for driver recovery on CausalDynamics, and strong long-term statistics on 35 Dysts chaotic systems, although the gain over a plain Neural ODE was not significant.

Critical strengths
  • Complete theory for an under-studied setting. It is the first work to connect well-posedness, identifiability, ergodicity and finite-sample recovery for delayed SDEs; earlier delayed-system work had no guarantees.
  • Assumptions are well motivated and fairly mild. Cycles, instantaneous effects and chaotic behaviour are all allowed, which fits continuous-time physics better than DAG-based causal discovery.
  • Broad comparison. It compares against 13 causal discovery baselines and 4 neural baselines, uses 6 complementary dynamics metrics, and reports significance tests.
Critical weaknesses
  • The experiments barely test delayed systems. All 35 Dysts test systems are ODEs, and only one CausalDynamics column involves time lags; the authors admit both benchmarks are "mostly SDE-driven". Please add controlled SDDE simulations and check whether edges are placed at the correct lag.
  • Identifiability relies on noise, but the test systems are near-deterministic. Full support comes from invertible noise. With little noise, the data stay near a low-dimensional attractor, so the signal strength smin⁡s_{\min}, and with it the identifiability margin, can be close to zero. With dense sampling, lag-0 and lag-1 inputs are also hard to tell apart.
  • Gains are modest and the tuning is inconsistent. AUROC is mostly about 0.6, and differences from C-NODE are often not significant. On Dysts, the method is no better than a plain NODE. The gradient-penalty settings differ across methods and benchmarks.
  • Missing baselines. There is no Neural DDE and no SINDy-with-delays, and it is unclear how lagged masks are mapped onto the benchmark graphs.
  • Limited practical scope. Cost grows quadratically with dimension, lags must be known in advance, and noise must be additive. Please report runtimes and test sensitivity to wrongly specified lags.
Questions for authors

See weakness.

3 comments
AuthorsOpenReview ↗

Response to reviewer 4ipJ

Thank you very much for noting that this is a complete theory for an under-studied setting, that the assumptions are well motivated and fairly mild, and that the comparison to other methods is broad.

We would like to respond here to the weaknesses and questions that you raised.

The experiments barely test delayed systems. We understand that this is a limitation of our experiments. However, as you mentioned, this is an under-studied setting, and there are no available benchmarks with more delayed systems. We tested our method on two benchmarks and reported multiple metrics. We plan to develop appropriate benchmarks and evaluations, and test our method as well as the baselines on them. We leave this for future work, since such a benchmark could, in our opinion, be a standalone paper.

Identifiability relies on noise, but the test systems are near-deterministic. Yes, the method requires invertible noise to obtain full support. In the CausalDynamics benchmark, systems are either noisy or deterministic, and we show results for both settings (in the noisy setting, the SNR is 1/2, meaning that the noise level is very high). The Dysts benchmark trajectories were generated with a stochastic noise forcing of level 0.05. This is not very high, but it is sufficient for the chaotic trajectories to be very diverse. A higher noise level would lead to many diverging trajectories and to a lower number of systems on which to test our model.

Gains are modest and the tuning is inconsistent. While the gains appear modest, and the difference with NODE is not significant on the Dysts benchmark, L0L_0-NDDE consistently outperforms the other methods and ranks best on all metrics except NRMSE on the Dysts benchmark. We added an experiment (Figure 4 in the appendix) to highlight that overfitting can lead to lower short-term error even though it leads to worse long-term statistics [1, 2].

On the CausalDynamics benchmark, we use a single set of hyperparameters for all systems, although different hyperparameters would probably perform best on different systems, such as a simplified 10-dimensional coupled ocean-atmosphere model and the 3-dimensional Lorenz system. This is why the metrics are not always as good as one would expect; the same applies to all compared methods.

The gradient-penalty settings differ across methods and benchmarks because the parameters are not directly transferable from one method to another. We therefore perform a hyperparameter search that is consistent across methods, and the selected parameters are then fixed for all datasets in a given benchmark. The datasets used to select hyperparameters are excluded from the results. This procedure is detailed in Section 4 and Appendices D2 and E1. In practice, when modelling a system, one could set the parameters of the model to reflect prior knowledge of the system.

Missing baselines. We compare our method to Neural DDE, but call it Neural ODE in our paper to make it clear that we refer to the method presented by Chen et al. (2018), extended to model delayed systems. Regarding SINDy with delays, we did not include it in our baselines because we believe it would require substantial changes to integrate delays. We would need to add to the library of precomputed operations all operations involving inputs at different time lags, leading to an exponential number of candidate operations to feed to the model.

Limited practical scope. We agree that one limitation is that the cost grows quadratically with dimension. We hope that this work lays the theoretical foundation for fully data-driven modelling of lagged dynamical systems, but it is not directly applicable to high-dimensional systems. As mentioned in the Discussion section, we hope to extend this work to an identifiable representation learning method for high-dimensional systems, providing scientific value beyond the theoretical contribution. The lags must be known in advance, but this is often fine in practice: observations are available at a given temporal resolution and are not continuous, so the input time lags can match the observation time grid.

Regarding the additive noise model, we use the standard noise model of stochastic differential equations, which is a common assumption in stochastic and causal modelling. More details about the invertibility assumption can be found in our response to reviewer 7oXa (W2). In short, we assume in our paper that every equation in our model contains some randomness, possibly correlated across variables.

As mentioned in our response to the first weakness above, we plan to develop appropriate benchmarks and evaluations in the future, and test our method as well as the baselines on them. We will then be able to test the sensitivity to wrongly specified time lags. As the considered systems are all low-dimensional, training our model on CPU takes only a few minutes (2 min 34 s on average on the Dysts benchmark systems).

Thanks for the reply and my concern has been addressed. But I decided to keep my current score.

Thank you for your reply and for your positive assessment of the paper.

Reviewer_7oXaOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes L0-NDDE, a method that combines Neural Delay Differential Equations with L0 sparsity on input features to identify the causal drivers and drift of stochastic delayed differential equations (SDDEs). The authors prove that under assumptions including non-degenerate noise, local Lipschitz drift, ergodicity, and population risk minimization, the binary driver mask M and drift G are identifiable at both population and finite-sample levels. Experiments on the CausalDynamics and Dysts benchmarks show improvements over existing causal discovery and Neural ODE-based methods for structural driver recovery and dynamics reconstruction.

Critical strengths
  1. Well-structured theoretical contribution. The proof architecture from existence (Proposition 1) through population identifiability (Theorems 1, 2) to ergodicity (Theorem 3) and finite-sample recovery (Theorem 4) is clean and well-organized.

  2. Strong and comprehensive empirical performance on driver identification. L0-NDDE achieves the best average rank on SHD (2.50), AUROC (1.75), and AUPRC (2.10) across all CausalDynamics categories, evaluated against 10+ competing methods spanning constraint-based, score-based, noise-based, and neural approaches. The key comparison is against C-NODE, which the authors extend to the delayed setting with L1 instead of L0, same architecture, same lagged inputs, isolating the effect of the penalty choice. L0-NDDE achieves meaningfully lower average SHD, showing that hard binary masking recovers causal structure more reliably than soft weight shrinkage

Critical weaknesses

W1. The realizability assumption (Assumption 3) is never verified experimentally. The theory requires that the model class contains the true drift (Assumption 3(i)) and that the global optimum is found (Assumption 3(ii)). In practice, the drift is parameterized by small MLPs with tanh activations. Whether the MLP actually recovers the true governing equations of the tested systems is never checked. The Dysts benchmark evaluates downstream trajectory metrics, not whether the learned drift matches the analytical drift known for these systems. It is possible that the MLP produces correct short-term trajectories through error compensation rather than faithful recovery of the true dynamics.

W2. Too many assumptions without proof of where and when they hold. The paper does not discuss how realistic its assumptions are for the physical systems it claims to target. Non-degenerate noise (Assumption 1(ii)) requires independent stochastic forcing on every variable, but many physical systems have deterministic couplings, e.g., radiative balance in climate models or synaptic currents in neural populations. Population risk minimization (Assumption 3(ii)) assumes global optimality of a nonconvex objective. The paper motivates the work with climate and neuroscience applications but never clarifies whether any of these systems actually satisfy the required assumptions, or how the method degrades when they are violated. A systematic discussion of where each assumption (1–7) is commonly satisfied, where it is likely violated, and what the empirical consequences of violation are would significantly strengthen the paper and help practitioners judge when the method can be trusted

W3. No real-world application. All experiments are on synthetic data. The CausalDynamics “Climate” datasets are simplified simulations (a 10-dimensional coupled ocean-atmosphere model and a toy ENSO model), not real climate observations. The paper’s motivating examples such as climate teleconnections, neural population dynamics, ecological feedbacks, remain entirely untouched. Without a real-world demonstration, it is unclear whether the method provides scientific value beyond winning a synthetic benchmark. For example, applying L0-NDDE to real ENSO reanalysis data, where the delayed coupling between sea-surface temperature and atmospheric pressure is physically understood but debated in its structure, would directly test whether the recovered mask aligns with known physics and whether the identifiability assumptions hold in practice.

W4. Improvement is concentrated on driver identification, with a possible tradeoff on short-term prediction. L0-NDDE achieves clear gains on SHD, which is valuable. However, it lags behind vanilla NODE and PathReg on NRMSE. Masking off inputs removes information the network could otherwise exploit for short-term fitting, even if that variable is not a true structural driver. This suggests a tradeoff between causal correctness and predictive accuracy that the paper never acknowledges or analyzes. Understanding when and how much prediction performance is sacrificed for correct structure recovery would be practically important

Questions for authors
  1. The NRMSE results suggest that enforcing correct causal structure via L0 masking can hurt short-term prediction accuracy relative to unmasked baselines. Can you characterize this tradeoff. For instance, how prediction error scales with the number of masked-off inputs.

  2. Assumption 3(i) requires that the MLP model class contains the true drift. In practice you use small MLPs with tanh activations, which are bounded. For the benchmark systems where the analytical drift is known, can you verify that the learned MLP actually recovers the true governing equations, rather than producing correct short-term trajectories through error compensation?

1 comment
AuthorsOpenReview ↗

Response to reviewer 7oXa

Thank you for highlighting the well-structured theoretical contribution and the strong and comprehensive empirical performance on driver identification.

We would like to respond to the weaknesses and questions that you raised.

W1 & Q2: Assumption 3 indeed considers an idealized learner, and even if our method identifies the correct driver, it does not mean that the neural network function approximates the correct drift. This is a strong assumption, but it is necessary for a proof of identifiability and is standard in the literature.

We thus report an additional metric, called "Drift MSE", in Table 2 (the Dysts benchmark results table). For each pair (i,j)(i, j), we calculate the mean squared error between the learned and true marginal drift functions, ∥f~j(xi)−fj(xi)∥2\| \widetilde{f}_j(x_i) - f_j(x_i) \|^2, averaged across coordinates and over 200 equally spaced values of xix_i in the training domain.

L0L_0-NDDE achieves lower Drift MSE than the competing methods, although only the difference with AGL and Pathreg is significant (permutation test). We also show the learned vs. true drift for one of the Dysts systems in Figure 5 of the appendix. The method generally learns the right function within the training domain (highlighted in green in the figure), but not outside of it. In other words, once the method identifies the correct drivers, it can approximate the drift well where there are observations.

W2: This is a good point that we haven't stressed enough in our paper. While we make an effort in our manuscript to explain what the different assumptions entail and to make them interpretable, we provide additional explanations here.

There are two groups of assumptions: on the physical system (A1(ii), A2, A4, A6) and on the learner (A3, A5, A7).

  • A1 (ii), invertible noise. This only assumes that every equation in our model contains some randomness, possibly correlated across variables. Deterministic couplings are thus allowed within our framework as part of the drift.
  • A2 (Regularity). This assumption asks for smooth dynamics without blow-up. It holds for standard physical systems such as Lorenz, Lotka-Volterra, Mackey-Glass and ENSO oscillators. A failure case is discontinuous models (e.g., integrate-and-fire models in neuroscience, or tipping points in climate models), which are out of scope for SDDEs and are not considered.
  • A3 (True drift with known lags contained in the model class and attained). While the known-lags condition seems quite restrictive, it is not very constraining in practice: physical systems are often observed at a given, discrete resolution, and time lags can be defined on this resolution. See response to W1 & Q2 for attainment of the minimum.
  • A4 (Faithfulness). This requires that each edge has an effect. If a driver does not, it cannot be detected by any method. I practice, this is always the case.
  • A5 (Structural sparsity). This assumption is replaced by the L0L_0 penalty and serves as a warm-up for identifiability.
  • A6 (Delay-adapted dissipativity). To ensure ergodicity in the delayed case, one needs a damping stronger than the delayed feedback. This holds for damped systems with a bounded attractor, a general class that includes classical systems such as Lorenz, Lotka-Volterra, Mackey-Glass and ENSO oscillators.
  • A7 (Finite-sample model class). This assumption ensures that the model class contains the true drift while not being too rich. That is, on any bounded region, every model in the class can be approximated by finitely many representative models, whose predictions do not differ too much. This makes the empirical risk uniformly close to the population risk (Equation 17 in the paper).

W3: We acknowledge that there is no real-world application shown in the paper, even though this work is largely motivated by real-world problems. We hope that this work lays the theoretical foundation for fully data-driven modelling of lagged dynamical systems, but it is not directly applicable to high-dimensional systems. We evaluated it on the CausalDynamics benchmark, a very comprehensive benchmark that contains a 10-dimensional coupled ocean-atmosphere model and a toy ENSO model, albeit only simulated ones. As mentioned in the Discussion section, we hope to extend this work to an identifiable representation learning method for high-dimensional systems and enable, for example, modelling real ENSO reanalysis data.

W4 & Q1: Yes, there is a tradeoff between short-term accuracy and sparsity of the learned dynamical driver matrix. To address this concern, we have added Figure 4 to the appendix. When using a fully dense matrix, the model can overfit and thus achieve lower next-step error. However, such models diverge faster from the true trajectories, as VPT decreases when SHD is lower. Please find a complete analysis in the manuscript. This is in line with results found in previous literature ([1], [2]).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

Dear AC and reviewers,

We sincerely thank all reviewers for noting:

  • the well-structured, complete, and easy-to-follow theoretical contribution for an under-studied setting (reviewers 7oXa, 4ipJ, 7egF), with clearly stated, well-motivated and fairly mild assumptions (reviewers 4ipJ, 7egF);
  • the strong and comprehensive empirical performance on driver identification, and the broad comparisons (reviewers 7oXa, 4ipJ).

In particular, we thank reviewer 7egF for stating that this is a significant contribution that should interest many people in the community.

We thank the reviewers for their constructive feedback, which allowed us to strengthen the analysis of the paper's results. We highlight here the updates we made to the manuscript so they are easily visible:

  • Evaluation of the learned drift We added a Drift MSE metric to the Dysts benchmark results, to investigate how well the drift is actually approximated by our method and the competing methods. The corresponding changes are:

    1. a description of the Drift MSE metric in Section 4.2 (end of second paragraph);
    2. an additional figure in the appendix (Figure 5) to illustrate this metric.
  • How sparsity and SHD affect short-term vs long-term metrics We ran an experiment to evaluate how sparsity and SHD affect short-term prediction accuracy vs. approximation of long-term dynamics. This resulted in:

    1. an additional paragraph in Section 5.2 (last paragraph);
    2. an additional figure in the appendix (Figure 4).

This experiment is in line with other results from the literature ([1], [2]). It shows that overly sparse transition matrices lead to lower short-term accuracy but also lower valid prediction time, or in other words, faster divergence from the true trajectory.

  • Rewording of the causal assumptions. We rewrote two sentences in the abstract and in paragraph 4 of the introduction. They now say that physical systems may violate causal assumptions about the graph and data-generating process, instead of saying that the assumptions are too restrictive. Details can be found in our response to reviewer 7egF.

We answer each reviewer's concerns in the individual responses.

[1] When are dynamical systems learned from time series data statistically accurate?, J. Park, N. T. Yang, N. Chandramoorthy, NeurIPS 2024

[2] A Weak Penalty Neural ODE for Learning Chaotic Dynamics from Noisy Time Series, X. Li, J. Harlim, D. Chakraborty, R. Maulik, 2026

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.