- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper addresses a critical gap in Neural SDE learning: the absence of dynamical guarantees, particularly the Foster–Lyapunov dissipativity condition, which is routinely assumed in theoretical analyses of stable Neural SDEs but is not automatically satisfied by standard architectures or training objectives. The proposed framework, PDNSDEs, translates the Foster–Lyapunov dissipativity condition into computable constraints and enforces them through structural projection or PINN-style regularization on the drift, thereby improving robustness in the extrapolation regime. The paper further introduces partial dissipativity to accommodate globally stable, chaotic, and integrated dynamics.
Critical strengths
- For each PDNSDE, thia paper introduces a "counterpart with exactly the same structure but without the partial dissipativity enforcement," enabling a clean evaluation of the enforcement effect. Multiple training objectives (signature kernel score, SDE Matching, finite-dimensional matching, and standard objectives including 1-step MSE) are treated as hyperparameters selected via the validation set, and comprehensive ablations are performed across model architectures (General Neural SDE, Homogeneous Neural SDE, Langevin-type Neural SDE, and Integrated Neural SDE). The paper acknowledges that "no single configuration uniformly dominates across datasets and evaluation regimes," which reflects an honest treatment of the empirical results.
- This paper identifies a genuine and practically important gap, and proposes a well-defined. The motivation is clear, grounded, and well-articulated.
Critical weaknesses
- PINN-style regularization depends on a sampling distribution over collocation points. For the shear and stochastic Lorenz–63 far-field experiments, the authors state that the evaluated initialization shells lie within the broader regularization collocation domains. Consequently, the far-field results for the regularized arm can be partially attributed to regions seen during regularization. They constitute extrapolation with respect to the training distribution but not with respect to the regularization support. Whether the regularized method remains effective beyond the regularization collocation domain is not tested, which limits the generality of the robustness claim for this enforcement mechanism.
- Table 1 shows that on the stochastic Lorenz–63 system, PDNSDE achieves a higher ID MMD$^2$ () than NSDE (), although PDNSDE outperforms NSDE from the shell onward. This raises the question of whether the ID accuracy loss is compensated by robustness gains even under modest distribution shifts. Furthermore, in Table 2, the structurally enforced lower-bound variant (Struct-Lower) consistently underperforms NSDE at low shift levels across multiple datasets. The authors acknowledge a stability–flexibility trade-off in Theorem 3.4, but the paper does not provide actionable guidance on when a given enforcement mechanism (upper vs. lower bound, regularization vs. structural projection) should be preferred, or whether enforcing dissipativity may be net harmful for certain dynamics.
- The no-regret guarantee in Theorem 3.4 is conditional. Its practical scope depends on correct specification of the Lyapunov function, coordinate mask, and compensation parameters. The authors partially address this in the limitations section, which to some degree mitigates the concern.
Questions for authors
Please refer to 'critical weaknesses'.
Response 1
Before replying to your points, we would first like to raise the following questions:
We now reply to your questions.
To begin with, we answer first your puzzling question about the positioning next to the projection papers and Okamoto and Kojima, Learning Deep Input-Output Stable Dynamics, NeurIPS 2022, as clearly that is not the focus of our work. Regarding the linear constraints, if you have really read the paper, have you not realized that the reason for their quadratic term is because the Hamilton-Jacobi (HJ) function is quadratic but not because they involve a fundamentally different projection principle? They essentially formulate their enforcement problem as an inequality of the form HJ ≤ 0, which is in the same form as both our work and the projection literature as cited in our paper,. In fact, their drift-only projection in Corollary 1 uses essentially the same ReLU-based projection mechanism as ours. Then obviously, in our case, the condition enforced is linear and hence our projection also appears to be linear.
As we mentioned that you seem to be familiar with the paper Okamoto and Kojima, Learning Deep Input-Output Stable Dynamics, NeurIPS 2022, we next answer your question on the practicality of the region around the singular set. Have you not realized that their projection is of almost the same form and suffers from the same issues, which they have addressed simply by introducing a very small jitter (1e-10) in the denominator and standard gradient clipping (check their GitHub codebase if you really do not know about these)? If you are indeed familiar with this field, as your review suggests, why are such standard numerical treatments overlooked, and why is this issue considered a "critical weakness"? Moreover, in our own setup, as described in Appendix A.6, Eq. (65), we have introduced an additional strict feasibility margin through the softplus parameterization. Also, how are our experiments across a variety of stochastic dynamical models and real-world datasets not counted as evidence of practical applicability?
These projection methods are standard and in no way the focus of our work, and so we think we need to restate our focus and contributions, as your novelty question strangely places a considerable amount of emphasis on the projection methods. The whole goal of our work is to point out that the commonly assumed dissipativity condition is not automatically guaranteed in standard Neural SDE training and that requiring full-state dissipativity can also be unnecessarily restrictive, and so we propose the partially dissipative framework. We also show that the joint condition itself is ambiguous in terms of accommodating considerably different dynamics, which then motivates further separation of the drift and diffusion contributions. This is also why we have arrived at the the drift-only condition to be enforced and hence the projectino mechanism. As stated in the paper, that is a valid and sufficient way to achieve the condition and so we do not consider this as a problem. This then leads to a range of conditions to enforce, and we provide theoretical analysis (we also elaborate on this point more later) that matches our framework and experiments to provide insights into the enforcement mechanisms, which are also clearly stated in the paper, even in the Abstract. We therefore do not understand why the new insights and analysis specific to the framework do not count as contributions to this reviewer. Also, given that we have never claimed any novelty in the geometry of orthogonal projection itself, why are you evaluating our theoretical contribution based on whether we have established new geometric properties of a standard projection?