acceptodds
Withdrawn from ICLR 2027

Watch your neighbors: Training statistically accurate chaotic systems with local phase space information

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Chaotic systems pose fundamental challenges for data-driven surrogate modeling, as small modeling errors lead to exponentially growing trajectory discrepancies. Since exact long-term prediction is unattainable, it is natural to ask what a good surrogate model for chaotic dynamics is. Prior work has largely focused either on reproducing the Jacobian of the underlying dynamics, which governs local expansion and contraction rates, or on reproducing the ground-truth dynamics' long-term statistical behavior. We propose a framework that bridges these two paradigms by training surrogate models with accurate Jacobians and long-term statistical properties. Our method constructs a local covering of a chaotic attractor in phase space and analyzes the expansion and contraction of these coverings under the dynamics. The surrogate model is trained by minimizing the maximum mean discrepancy between the pushforward distributions of the coverings under the surrogate and ground-truth dynamics. Experiments show that our method significantly improves Jacobian accuracy while remaining competitive with state-of-the-art statistically accurate dynamics learning methods, at a computational cost that is a constant factor independent of the system dimension. Our code is fully available at https://anonymous.4open.science/r/neighborwatch.

Primary area:
learning on time series and dynamical systems
Keywords:
Neural ordinary differential equations, dynamical systems, chaos

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tLTkOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents Neighborhood training for learning chaotic dynamics from noisy trajectories. The method constructs annular neighborhoods around observed states, evolves neighbor displacements through a second-order Taylor expansion of a Neural ODE, and matches predicted and observed neighborhood distributions using MMD. The objective combines this neighborhood regularizer with trajectory fitting. Experiments on a series chaotic problems demonstrate improved Jacobian accuracy relative to trajectory-only training and DySLIM, with mixed results for short-term forecasting and long-term statistical accuracy.

Critical strengths
  1. The method offers an interesting way to learn local dynamics from noisy trajectories through neighborhood distribution matching.
  2. The annular neighborhood design is well motivated: excluding overly close observations helps prevent measurement noise from overwhelming the relative displacements used to learn local dynamics.
  3. The method improves Jacobian accuracy over Vanilla and DySLIM using only noisy trajectory observations, avoiding ground-truth derivative supervision that is typically unavailable when the governing equations are unknown.
Critical weaknesses
  1. Scalability to spatiotemporal systems. Although the method performs well on low-dimensional ODEs, its applicability to more complex spatiotemporal systems, such as the Kuramoto–Sivashinsky equation and two-dimensional Navier–Stokes flows, remains unclear. Here, each state represents an entire spatial field, and neighborhoods are constructed in the full state space rather than physical space. As the dynamics’ intrinsic dimensionality increases, finite trajectory datasets may become too sparse to provide sufficiently close and directionally informative neighbors.
  2. Insufficient empirical validation of scalability. This concern requires empirical validation beyond qualitative discussion. Experiments on a spatially extended system, varying resolution, and local dataset size, would help establish whether informative local neighborhoods remain available.
  3. Limited justification for neighborhood matching. Proposition 1 concerns exact agreement of restricted measures, whereas the implemented loss uses finite samples and Taylor-approximated evolution. Small empirical MMD does not, by characteristicness alone, guarantee support agreement or Jacobian accuracy. The connection to local deformation and long-term statistical accuracy therefore needs stronger justification and more qualified discussion.
  4. Mismatch between formulation and implementation. Eq. (6) restricts global pushforward measures to target sets, whereas the algorithm evolves samples from source neighborhoods. These constructions may not be generally equivalent.
Questions for authors
  1. Related to Weakness 1 and 2, could the authors provide an experiment on one spatial system and assess how neighborhood quality and performance change with dimensionality and local dataset size?
  2. How do neighborhood quality and model performance change as the dynamics’ intrinsic dimensionality increases? An experiment on a spatially extended system would help further clarify the method’s practical scalability.
  3. What additional assumptions are needed to connect small empirical neighborhood MMD to accurate local deformation and long-term statistics, beyond the exact measure agreement in Proposition 1? Clarifying this would help readers understand the scope and limitations of the theoretical justification.
  4. Can the authors clarify how Eq. (6), which restricts global pushforward measures to target sets, corresponds to the implemented objective that evolves samples from source neighborhoods?
Reviewer_K6P1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a neighborhood-based training method for Neural ODEs that aims to achieve both accurate local dynamics and long-term statistics in chaotic systems. The method propagates local perturbations using a second-order Taylor expansion and matches predicted and observed neighborhood distributions using MMD. Experiments on three chaotic systems with varying noise levels demonstrate improved Jacobian and Lyapunov exponent accuracy over existing baselines, while maintaining competitive long-term statistical performance.

Critical strengths
  1. Meaningful problem and well-motivated approach. The paper addresses the important challenge of learning accurate Jacobians from noisy trajectories without access to ground-truth derivatives. Incorporating local neighborhood evolution into training is an interesting and practically relevant idea.

  2. Encouraging empirical results. The method consistently improves Jacobian accuracy across the evaluated systems and noise levels, with additional improvements in Lyapunov exponent estimation and limited computational overhead.

Critical weaknesses
  1. Insufficient validation of the proposed mechanism. The paper lacks key ablations comparing neighborhood MMD against simpler alternatives, such as directly minimizing relative displacement errors or computing MMD from independently integrated neighbors. Moreover, Proposition 1 establishes measure agreement but provides no direct guarantee of Jacobian accuracy. Consequently, the source of the reported improvements remains insufficiently isolated.

  2. Potential issue with the MMD estimator. Equation (8) appears to apply an unbiased two-sample MMD estimator to paired predicted and observed neighbor samples. Due to their dependence, the estimator is generally biased and can be negative even under exact pairwise matching. This raises concerns about the statistical interpretation of the proposed objective.

Questions for authors
  1. Does the implementation use the MMD estimator exactly as specified in Eq. (8)? Given the correspondence between predicted and observed neighbors, how is the dependence between paired samples handled? Would a standard V-statistic MMD estimator yield comparable results?

  2. How does neighborhood MMD compare against a direct relative-displacement MSE loss? Could the authors provide ablations comparing direct neighbor integration with Taylor-based propagation, as well as first-order versus second-order expansions? These comparisons would help establish which components are essential to the observed Jacobian improvements.

  3. Does the proposed neighborhood matching objective provide any theoretical guarantee on Jacobian recovery under additional assumptions, or is the connection supported purely by empirical evidence?

Reviewer_maHSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Neighborhood training for Neural ODEs to improve both local dynamical accuracy and long-term statistical fidelity from noisy trajectories. It constructs annular neighborhoods around observed states, evolves neighbor displacements using a second-order expansion of the learned vector field, and supplements trajectory loss with an MMD loss between predicted and observed neighborhood distributions. Experiments on Lorenz63, hyperchaotic Chen, and six-dimensional Lorenz96 report improved Jacobian estimates relative to Vanilla training and DySLIM, while generally sacrificing some statistical accuracy relative to DySLIM. Additional analyses examine Lyapunov spectra, neighborhood radii, alternative baselines, and computational cost.

Critical strengths

The paper addresses a meaningful distinction between matching trajectory distributions and learning accurate local dynamics. Neighborhood-based regularization offers a useful way to incorporate derivative-related information without observing ground-truth Jacobians. Table 6 reports lower mean log relative Jacobian error than both main baselines across all nine system/noise configurations.

The evaluation examines several complementary properties: short-term prediction, long-term distributions, vector fields, Jacobians, and Lyapunov spectra. The radius ablations, local quadratic estimator, Jacobian-oracle reference, and measured training times help characterize both the benefits and limitations of the approach.

Critical weaknesses

The connection between Proposition 1 and the practical objective is incomplete. The proposition correctly establishes equality of measures from exact restrictions on a common cover. However, the algorithm compares finite samples from normalized neighborhood distributions, evolved through a truncated Taylor system rather than the exact learned flow. Small empirical MMD does not by itself establish the claimed agreement of neighborhood supports or Jacobians. The paper should distinguish restrictions after pushforward from pushforwards of restricted measures, specify normalization and overlap handling, and justify the approximation connecting Eqs. (6)–(9). The empirical Jacobian gains remain valuable, but are not guaranteed by the proposition.

The experiments leave the source and generality of those gains uncertain. There are no direct comparisons against neighborhood MMD using exact neighbor rollouts, first-order displacement evolution, or a paired displacement loss. Moreover, all three specified ground-truth vector fields are quadratic, making them especially favorable to a second-order expansion. Hyperparameters are selected using noiseless validation data (Appendix C), so an entirely noisy-data workflow is not demonstrated. The statistical tradeoff is also material: at 10% noise on Lorenz63, MMD² is 0.0165 versus 0.00304 for DySLIM, and VPT is 1.33 versus 1.63. These limitations prevent a sufficiently clear assessment of the proposed mechanism’s advantage.

Questions for authors

Theoretical clarification: What precisely are the measures compared in Eqs. (6), (8), and (9)? How do normalization, overlapping neighborhoods, and Taylor truncation affect the application of Proposition 1? Which conclusions are exact, and which should be presented as empirical motivation?

Evaluation clarification: Do the principal improvements persist when hyperparameters are selected exclusively from noisy validation observations? The disclosed clean-validation protocol is shared by the methods, but does not establish performance when clean observations are unavailable.

Evidence needed to assess the contribution: Can you compare exact neighbor rollouts, first-order evolution, and a paired displacement loss under matched training budgets? A nonquadratic benchmark would additionally test whether the second-order approximation remains effective beyond the favorable structure of the current systems.

Implementation clarification: Please confirm the implemented Lorenz63 parameter assignment, since Appendix B.2 lists the purported canonical values as (ρ, σ, β) = (10, 8/3, 28). Also clarify whether Eq. (15) uses the Euclidean norm or its square: the displayed squared norm is inconsistent with the stated root-mean-square metric and affects interpretation of the VPT threshold.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.