acceptodds
Withdrawn from ICLR 2027

Dissipative Neural SDEs for Robust Dynamical Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

While Neural stochastic differential equations (SDEs) provide a flexible framework for learning stochastic dynamics, their training still lacks dynamical guarantees, and in particular, they often exhibit non-physical behaviors under distribution shifts. This work aims to mitigate such failures by emphasizing the role of the Foster-Lyapunov dissipativity condition, which is commonly assumed in the current literature but is not guaranteed by the standard Neural SDE architectures. In this work, we introduce a flexible unified framework of partially dissipative Neural SDEs, which accommodates a wide class of dynamics, such as globally stable, chaotic, and integrated dynamics. We translate the dissipativity condition into sufficient forms compatible with enforcement mechanisms: PINN-style regularization and structural projection. Theoretically, we show that structural projection is conditionally no-regret, revealing a stability–flexibility trade-off, while our analysis further supports the framework's pathwise approximation flexibility over a moment-bounded class of initial laws. Experimentally, we demonstrate that the methods considerably improve dynamical recovery and robustness. The practical relevance is demonstrated on real-world high-dimensional datasets in terms of forecasting and reconstruction performance under context missingness and noise corruption.

Primary area:
learning on time series and dynamical systems
Keywords:
Neural stochastic differential equations, Foster-Lyapunov methods, partial dissipativity, distribution shift, stochastic dynamical systems, time-series modeling.

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rPhqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses a critical gap in Neural SDE learning: the absence of dynamical guarantees, particularly the Foster–Lyapunov dissipativity condition, which is routinely assumed in theoretical analyses of stable Neural SDEs but is not automatically satisfied by standard architectures or training objectives. The proposed framework, PDNSDEs, translates the Foster–Lyapunov dissipativity condition into computable constraints and enforces them through structural projection or PINN-style regularization on the drift, thereby improving robustness in the extrapolation regime. The paper further introduces partial dissipativity to accommodate globally stable, chaotic, and integrated dynamics.

Critical strengths
  • For each PDNSDE, thia paper introduces a "counterpart with exactly the same structure but without the partial dissipativity enforcement," enabling a clean evaluation of the enforcement effect. Multiple training objectives (signature kernel score, SDE Matching, finite-dimensional matching, and standard objectives including 1-step MSE) are treated as hyperparameters selected via the validation set, and comprehensive ablations are performed across model architectures (General Neural SDE, Homogeneous Neural SDE, Langevin-type Neural SDE, and Integrated Neural SDE). The paper acknowledges that "no single configuration uniformly dominates across datasets and evaluation regimes," which reflects an honest treatment of the empirical results.
  • This paper identifies a genuine and practically important gap, and proposes a well-defined. The motivation is clear, grounded, and well-articulated.
Critical weaknesses
  • PINN-style regularization depends on a sampling distribution νsample\nu_{sample} over collocation points. For the shear and stochastic Lorenz–63 far-field experiments, the authors state that the evaluated initialization shells lie within the broader regularization collocation domains. Consequently, the far-field results for the regularized arm can be partially attributed to regions seen during regularization. They constitute extrapolation with respect to the training distribution but not with respect to the regularization support. Whether the regularized method remains effective beyond the regularization collocation domain is not tested, which limits the generality of the robustness claim for this enforcement mechanism.
  • Table 1 shows that on the stochastic Lorenz–63 system, PDNSDE achieves a higher ID MMD$^2$ (0.0127±0.01480.0127 \pm 0.0148) than NSDE (0.0024±0.00010.0024 \pm 0.0001), although PDNSDE outperforms NSDE from the 2×2\times shell onward. This raises the question of whether the ID accuracy loss is compensated by robustness gains even under modest distribution shifts. Furthermore, in Table 2, the structurally enforced lower-bound variant (Struct-Lower) consistently underperforms NSDE at low shift levels across multiple datasets. The authors acknowledge a stability–flexibility trade-off in Theorem 3.4, but the paper does not provide actionable guidance on when a given enforcement mechanism (upper vs. lower bound, regularization vs. structural projection) should be preferred, or whether enforcing dissipativity may be net harmful for certain dynamics.
  • The no-regret guarantee in Theorem 3.4 is conditional. Its practical scope depends on correct specification of the Lyapunov function, coordinate mask, and compensation parameters. The authors partially address this in the limitations section, which to some degree mitigates the concern.
Questions for authors

Please refer to 'critical weaknesses'.

Reviewer_n46sOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a Foster-Lyapunov type dissipativity condition to neural SDE training, with two variants: a regularization term that is computed from sampled state-time points drawn from dataset specific distributions; and a structural enforcement method that projects the model output at each step. In practice, the paper selects the methods to apply using a validation set. Overall, the experiment appears to show improved robustness under distribution shift, increased dimensions, and noises and missing context, against evaluated baselines.

Critical strengths
  1. The partial dissipativity condition widens the scope of systems the proposed framework can model.
  2. The experiment appears to show improved robustness under distribution shift, increased dimensions, and noises and missing context, against evaluated baselines, across synthetic and realistic datasets.
Critical weaknesses
  1. The prior given in the synthetic datasets appears to be kind of strong. The true functional class of V and the exact integration order are given to the model.
  2. The mask coordinates are selected to either match the ground truth or from domain knowledge, which appears to be a strong condition.
  3. While the framework selects the exactly method to use according validation performance, the paper lacks in-depth discussion over why a particular method underperforms against the other.
Questions for authors
  1. How does the proposed framework perform on the synthetic under more generic class of V such as the one used in the real datasets?
  2. This robustness to far-field shift can be intuitively attributed to the uniform-in-time Lyapunov-moment control, and hence marginal confinement induced by dissipativity.

  • can the authors elaborate/expand on the attribution here.
Reviewer_iuKnOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a framework for partially dissipative Neural SDEs that imposes a Foster–Lyapunov dissipativity condition on selected state coordinates. Given a prespecified coordinate mask and Lyapunov function, it constructs a sufficient condition on the drift that accounts for the diffusion contribution and enforces it through structural projection. The theoretical results show that projection does not increase the local generator discrepancy when the true drift belongs to the feasible set, and that, under certain conditions, convergence of the drift and diffusion implies finite-horizon trajectory convergence. Experiments evaluate the approach through out-of-distribution recovery of synthetic dynamical systems and forecasting and reconstruction on real data with missing or noisy observations.

Critical strengths
  1. The paper extends existing Lyapunov-constrained stabilization ideas to Neural SDEs with learned diffusion and dissipativity on a subset of the state, bringing regularization, structural projection, and a high-dimensional decomposition into a unified framework. In particular, controlling derivative states in integrated systems without imposing the same dissipativity condition on every state offers modeling benefits for suitable target systems.

  2. Comparisons with unconstrained models sharing the same integration structure attempt to distinguish the effects of the architecture from those of dissipativity enforcement. There are substantial reductions in distributional recovery error under strong distribution shifts in synthetic systems. The evaluation also includes real data with missing and noisy observations, demonstrating performance improvements from dissipativity constraints under the evaluated conditions.

Critical weaknesses
  1. The theoretical novelty is not sufficiently established. The approach appears largely to adapt Lyapunov-constrained, projection-based learning, as exemplified by Learning Stable Deep Dynamics Models, to particular stochastic systems and subsets of the state. Theorem 3.4 relies on standard properties of orthogonal projection, while Theorem 3.6 establishes trajectory convergence assuming coefficient approximation. The theoretical contribution beyond prior work and the geometric meaning of the guarantee in Theorem 3.4, including the equality conditions, remain unclear.

  2. The validity of the assumptions and sufficient conditions is not sufficiently evaluated. The theory relies on several conditions: the restriction from the original partial dissipativity condition to the conditions defining Dcomp\mathcal D_{\mathrm{comp}}, the growth bound on the diffusion contribution in Assumption 3.2, feasibility of the true drift in Theorem 3.4, and the common dissipativity and growth conditions and coefficient approximation assumed in Theorem 3.6. However, the paper does not sufficiently establish how restrictive these conditions are for the target systems or which systems they exclude from the guarantees. In particular, the relationship between the coefficient class realizable by the proposed enforcement mechanisms and the coefficient approximation assumed in Theorem 3.6 is unclear.

  3. The reduction to linear constraints is not sufficiently justified. Okamoto and Kojima, Learning Deep Input-Output Stable Dynamics, NeurIPS 2022 constructs an analytical projection under a chosen distance for the quadratic Hamilton–Jacobi constraint in deterministic input-output systems. Although the setting differs, linear constraints are not a prerequisite for constructing a projection. The paper therefore needs to justify the additional assumptions and the design choice of fixing diffusion and modifying only the drift, and explain the trade-offs in expressiveness and computational cost.

  4. The evidence supporting practical applicability is insufficient. Equation (20) is defined separately on the singular set, and Appendix A.6 discusses feasibility there. However, behavior near that set, where the denominator becomes small, also depends on how the numerator vanishes. Although divergence does not necessarily follow, large corrections or gradients remain an unresolved concern. The boundedness and regularity of the projected drift and the effects of these issues on training stability are not sufficiently established. Assuming well-posed solutions in Section 2 alone does not resolve these concerns about the construction and implementation. Accuracy improvements under the evaluated conditions are insufficient to establish broad practical applicability.

Questions for authors
  1. What is the additional theoretical contribution? The current construction appears to reduce to projecting the drift (the local mean rate of change) onto a half-space while holding diffusion fixed. Given this, please explain the essential difference from existing drift projection methods such as Learning Stable Deep Dynamics Models. Also clarify what Theorem 3.4 adds to standard properties of orthogonal projection from a geometric perspective, including the equality conditions in Eq. (25).

  2. How restrictive are the theoretical assumptions? Please identify the systems excluded by the restriction to Dcomp\mathcal D_{\mathrm{comp}} and the additional assumptions in each theorem, and provide conditions under which the proposed enforcement mechanisms can achieve the coefficient approximation assumed in Theorem 3.6.

  3. Are the assumptions introduced to obtain linear constraints actually necessary? Learning Deep Input-Output Stable Dynamics (NeurIPS 2022) constructs an analytical projection for quadratic constraints. Although that method may not apply directly to this setting, could any of the present assumptions be avoided or relaxed by handling quadratic constraints directly and also modifying diffusion? Please distinguish assumptions essential to the guarantees from choices made to simplify the projection construction.

  4. How is stability near the singular set in Eq. (20) ensured? Please explain the effects of small denominators on correction magnitudes, gradients, and training stability, and provide conditions ensuring regularity of the projected drift and well-posedness of the resulting SDE.

6 comments
AuthorsOpenReview ↗

Response 1

Before replying to your points, we would first like to raise the following questions:

  1. Why are you criticizing positioning, novelty, and theoretical claims that we have never made, while ignoring the claims that we have actually made?
  2. Is applying a well-established analytical framework to analyze a newly proposed methodological framework a problem, rather than an indication of principled and reliable analysis? Does applying existing tools as building blocks for a new framework mean that the entire framework lacks novelty?
  3. Is it not obvious that we are not aiming to develop a new projection principle, but rather trying to address the lack of enforcement of a critical dynamical condition and proposing a suitable framework?

We now reply to your questions.

To begin with, we answer first your puzzling question about the positioning next to the projection papers and Okamoto and Kojima, Learning Deep Input-Output Stable Dynamics, NeurIPS 2022, as clearly that is not the focus of our work. Regarding the linear constraints, if you have really read the paper, have you not realized that the reason for their quadratic term is because the Hamilton-Jacobi (HJ) function is quadratic but not because they involve a fundamentally different projection principle? They essentially formulate their enforcement problem as an inequality of the form HJ ≤ 0, which is in the same form as both our work and the projection literature as cited in our paper,. In fact, their drift-only projection in Corollary 1 uses essentially the same ReLU-based projection mechanism as ours. Then obviously, in our case, the condition enforced is linear and hence our projection also appears to be linear.

As we mentioned that you seem to be familiar with the paper Okamoto and Kojima, Learning Deep Input-Output Stable Dynamics, NeurIPS 2022, we next answer your question on the practicality of the region around the singular set. Have you not realized that their projection is of almost the same form and suffers from the same issues, which they have addressed simply by introducing a very small jitter (1e-10) in the denominator and standard gradient clipping (check their GitHub codebase if you really do not know about these)? If you are indeed familiar with this field, as your review suggests, why are such standard numerical treatments overlooked, and why is this issue considered a "critical weakness"? Moreover, in our own setup, as described in Appendix A.6, Eq. (65), we have introduced an additional strict feasibility margin through the softplus parameterization. Also, how are our experiments across a variety of stochastic dynamical models and real-world datasets not counted as evidence of practical applicability?

These projection methods are standard and in no way the focus of our work, and so we think we need to restate our focus and contributions, as your novelty question strangely places a considerable amount of emphasis on the projection methods. The whole goal of our work is to point out that the commonly assumed dissipativity condition is not automatically guaranteed in standard Neural SDE training and that requiring full-state dissipativity can also be unnecessarily restrictive, and so we propose the partially dissipative framework. We also show that the joint condition itself is ambiguous in terms of accommodating considerably different dynamics, which then motivates further separation of the drift and diffusion contributions. This is also why we have arrived at the the drift-only condition to be enforced and hence the projectino mechanism. As stated in the paper, that is a valid and sufficient way to achieve the condition and so we do not consider this as a problem. This then leads to a range of conditions to enforce, and we provide theoretical analysis (we also elaborate on this point more later) that matches our framework and experiments to provide insights into the enforcement mechanisms, which are also clearly stated in the paper, even in the Abstract. We therefore do not understand why the new insights and analysis specific to the framework do not count as contributions to this reviewer. Also, given that we have never claimed any novelty in the geometry of orthogonal projection itself, why are you evaluating our theoretical contribution based on whether we have established new geometric properties of a standard projection?

AuthorsOpenReview ↗

Response 2

Now, we answer your questions regarding the restrictions, which seems to have 2 directions, one regarding the framework coverage and one regarding the assumptions. We answer the framework coverage first. Regarding the diffusion growth, that is essentially a considerable generalization and more than sufficient for many practical applications. As you suggest high familiarity with this field, have you not realized that many works cited in our paper have assumed constant or bounded diffusion coefficients, while ours essentially generalize to allow unbounded diffusion with linear growth? Also, in practice, one would generally want to prevent excessively large diffusion outputs at inf and so a numerical cap is very commonly put, which directly leads to a uniform bound on the diffusion. Therefore, how would such a thing considered a critical limitation? Furthermore, as clearly stated in Proposition 3.3, we explicitly show that our framework includes deterministic dissipative dynamics with stochastic noise injection and further generalizes beyond them through the lower-bound case. This already covers a broad class of commonly considered stochastic dynamical systems. If you disagree with the applicability, could you please provide any concrete examples of common dissipative SDEs relevant to the scope of our work that are excluded (of course, not including purely conservative systems without any dissipative component)?

Regarding your question about the theoretical assumptions, this is another puzzling point, because the scope and purpose of our theorems are both clearly stated, which only suggests that you are confusing the assumptions in our framework with the assumptions in our theoretical analysis of the trade-off. Those theorems, as even suggested by their names, are used to show the stability–flexibility trade-off and the expressive power of the framework. The assumption in Theorem 3.4 is used as an indicator of what is driving the trade-off, as there is, of course, no free lunch, and such an implication is also clearly stated in the paper and even the Abstract. The coefficient approximation condition in Theorem 3.6 is nothing but a standard assumption motivated by the well-established expressive power of neural networks. The additional assumption on the unselected states is also chosen to match the integrated dynamics considered in this paper. The purpose of this theorem is to support the pathwise approximation capability of the framework under conditions compatible with the models considered in our experiments. Furthermore, the assumptions actually required for implementing our framework have already been explicitly addressed in our experiments and appendices.

In conclusion, we strongly disagree with this review, as many of its criticisms are based on either claims we have never made or concerns that are invalid, irrelevant, or overstated. We find it difficult to understand how the standard nature of the tools, basic treatments for numerical stability, and the nature of our theoretical analysis can be treated as critical weaknesses. We therefore question the basis on which these concerns justify such a strongly negative assessment of our work.

Reviewer_iuKnOpenReview ↗

Thank You for Your Responses

Thank you for your response. I understand that novelty in the projection principle itself is not the central focus of this work, and I understand your explanations regarding the assumptions and the class of systems considered.

My main concern, however, is the necessity of the adopted conditions and their implications for the scope of the guarantees. This also motivated my question in Q3 about the reduction to linear constraints. Projection methods have evolved beyond linear constraints to include analytic projections under quadratic inequality constraints. The proposed construction holds the diffusion coefficient fixed and projects only μ\mu under linear constraints. In light of these developments, I believe it is important to clarify whether the assumptions introduced in this construction are essential to the guarantees or are design choices made to simplify the projection construction, and what theoretical advances it provides. The motivation for this design has been explained, but I do not consider the response to have sufficiently clarified the need for these restrictions, their effects on the scope of the guarantees, or the construction’s theoretical contribution in the context of this broader development of projection methods.

Regarding Q4, I understand your explanations concerning numerical treatments used in other methods, the feasibility margin introduced through softplus, and the existing experimental results. My concern, however, is the behavior of the projection correction and its gradients near the singular set, and whether the projection can be computed in a numerically stable manner while remaining consistent with the theoretical guarantees. This is an important issue concerning both the theoretical guarantees of the method and the validity of its numerical implementation. The reported experiments support the method’s performance and computational usefulness, but they do not directly examine this issue, and I do not consider this concern to have been resolved by the response.

Your response has clarified the intent and positioning of this work. However, with regard to the technical concerns discussed above, I have not gained new insights that would warrant revising my initial assessment. Therefore, having considered your response, I maintain my current assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.