acceptodds
Under review as a conference paper at ICLR 2027

Beyond Trajectories and Jacobians: Distilling Finite-Time Tangent Cocycles for Chaotic Forecasting

Abstract

State-space losses and local Jacobian objectives do not directly constrain how infinitesimal errors are transported over multiple steps in a chaotic system. We introduce BundleDistill, a training-only objective that distills low-rank finite-time tangent-cocycle geometry from controlled antithetic trajectory rollouts, without requiring governing equations or derivative labels. The student matches cocycle action, probed singular growth, and transported-subspace orientation using Jacobian–vector products and small endpoint factorizations. Under matched optimizer-update budgets across five locked seeds on Lorenz–63, Lorenz–96, and Kuramoto–Sivashinsky, BundleDistill has the smallest observed transported-subspace angle on all three systems. Against the strongest scalar-growth control, restricted-mean valid prediction time increases on Lorenz–96 by 0.018 Lyapunov times (95% percentile interval [0.016,0.021]) and by 0.180 on Kuramoto–Sivashinsky ([0.149,0.215]), while the Lorenz–63 interval spans zero. Five-seed ablations on Lorenz–96 and Kuramoto–Sivashinsky show that component effects depend on the system and endpoint. The auxiliary objective leaves model size unchanged and adds no inference operation, but it increases training cost; under a calibrated optimizer-update-time control, the faster Rand-Jac2 baseline is stronger on Lorenz–96 VPT and geometric endpoints. Formal out-of-regime tests yield no Holm-corrected rejection, and held-out long-run statistics vary by system. The evidence supports finite-time tangent transport as a useful, compute-dependent training target, with its clearest gains on the two higher-dimensional systems studied.

Primary area:
learning on time series and dynamical systems
Keywords:
chaotic forecasting, dynamical systems, finite-time tangent cocycles, geometric learning, knowledge distillation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7rFsOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces BUNDLEDISTILL, a training objective that supervises how small perturbations evolve over multiple forecasting steps. Controlled simulator rollouts provide finite-difference targets for perturbation action, growth and transported-subspace orientation. The method adds no inference operations. Its primary evaluation covers Lorenz–63, Lorenz–96 and spatially one-dimensional Kuramoto–Sivashinsky (KS), comparing forecasting accuracy and tangent geometry against trajectory- and derivative-based objectives.

Critical strengths

The finite-time supervision objective is well motivated and targets information beyond local Jacobian norms. Under equal optimizer-update budgets, the method achieves the smallest observed transported-subspace angles across all three systems, with a 23.2% improvement in valid prediction time over Growth-Match on KS. The five-seed evaluation and explicit reporting of negative results, computational overhead and objective trade-offs strengthen the empirical presentation.

Critical weaknesses
  1. The forecasting benefit relative to computational cost is insufficiently established. The main comparisons match optimizer updates rather than compute. Reported training time is 3.9–9.3 times that of the multistep baseline, while the prediction-time gain over Growth-Match is only 1.5% on Lorenz–96 and inconclusive on Lorenz–63. In the calibrated Lorenz–96 timing comparison, Rand-Jac2 achieves better prediction time and geometric accuracy, although realized times are not exactly matched and BUNDLEDISTILL retains lower mean normalized error. These results leave its practical advantage uncertain.

  2. Geometric improvements do not consistently translate into broader forecasting quality. On KS, removing the growth-matching term improves prediction time, and the full method produces worse long-run distributional and spectral results than the multistep baseline. Moreover, validation uses one GRU forecaster family and requires resettable, controlled trajectories; applicability to larger spatial systems, noisy observations or passive datasets remains unestablished. Although these limitations are acknowledged, stronger evidence is needed to identify when the additional supervision is worthwhile.

Questions for authors
  1. Compute-adjusted performance: Can the calibrated timing comparison be extended to all three systems with matched end-to-end budgets, including controlled teacher-rollout generation? Do the strongest KS gains survive this comparison?

  2. Geometry versus forecasting: What explains the improved KS prediction time after removing growth matching, and the worse long-run statistics despite improved tangent alignment? How should practitioners select the loss components for their intended forecasting objective?

Reviewer_MqAiOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper propose Bundledistill, a training objective for chaotic forecasting that matches finite time tangent dynamics rather than only trajectories or local Jacobians. It uses controlled perturbations, jacobian vector products, and low-rank subspace alignment. Experiments on Lorenz-63, Lorenz-96, and Kuramoto–Sivashinsky show improved geometric alignment and some gains in valid prediction time.

Critical strengths

The distinction between local Jacobian matching and finite time tangent transport is meaningful. The proposed objective captures directional instability and subspace rotation, which scalar growth measures can miss. The experiments include relevant baselines, component ablations, and uncertainty estimates. The improvement on Kuramoto–Sivashinsky is particularly encouraging.

Critical weaknesses

The main concern is computational efficiency. Training costs 3.9–9.3 times more than the multi-step baseline. More importantly, Tab. 4 shows that Rand-Jac2 substantially outperforms Bundledistill on Lorenz-96 when given a roughly comparable computation budget. This weakens the practical advantage suggested by equal-update comparisons.

The theoretical results largely follow established finite difference consistency, random- projection, and matrix perturbation arguments. Furthermore, improvements in forecasting are inconsistent across systems, out of regime results are statistically inconclusive, and long run statistics spoil substantially on Kuramoto–Sivashinsky. The requirement for a resettable simulator also limits applicability to ordinary observational time series.

Questions for authors

Can the authors clarify whether the method provides a consistent advantage over Rand-Jac2 under genuinely matched wall-clock budgets? Why does improved tangent-subspace alignment not translate into better long run stats on Kuramoto–Sivashinsky? How practical is the method when controlled perturbation rollouts are unavailable?

Reviewer_fz7oOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a novel set of losses for matching the behavior of a local tangent space as it is propagated by the chaotic dynamics. The authors compare performance on 3 toy chaotic systems.

Critical strengths

The paper provides a nice exposition for an interesting alternative loss that makes use of an available simulator to capture local dynamical structure.

Critical weaknesses

Unfortunately, the paper's experimental results are rather inconclusive. The results in the main text show relatively minor improvements to short-term prediction performance (hard to judge without error bars). Long-term statistical behavior, which is very important and difficult for chaotic systems, does not seem to improve at all with the new method and, in some cases, the performance is much worse.

Questions for authors
  1. There seems to be very little difference between including and not including components of the method (from the ablations in Table 3). To what extent do baselines already preserve the properties you are trying to target using the new losses?
  2. In theory, how does noise affect the loss terms? In practice, how does noise affect the performance for short-term prediction and long-term statistical behavior?
  3. How does the method behave in a more realistic high-dimensional setting (beyond 1D toy problems), like Kolmogorov flow?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.