acceptodds
Under review as a conference paper at ICLR 2027

Why Directly Learning Periodic Trajectories Can Fail

Abstract

Operator learning of periodic solutions requires deciding how simulation data should be recorded and represented. A natural choice is to integrate long enough for transients to decay and record a window wide enough to contain at least one full period of all trajectories. We find that these conservative choices can make the resulting trajectories difficult to learn, even when the underlying periodic orbits vary regularly with system parameters. Unaligned trajectories generalize poorly even within the training distribution. Phase alignment substantially improves in-distribution generalization, but models trained on a fixed physical-time window still have large errors on trajectories with periods outside the training range. We explain both failures through a common mechanism: frequency differences accumulate over time, so the target phase varies rapidly with the parameters. Predictors that cannot track this variation incur a population MSE floor in both settings; for fixed window prediction, we also derive a per-sample lower bound. We then study one of the simplest representations that escape these floors: learning an aligned, normalized waveform and its period separately. We establish regularity of the decoupled targets under ODE assumptions and show experimentally that this approach avoids both failures in ODE systems and a PDE case study.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Scientific machine learning, Operator learning, Periodic solutions, Dynamical systems, Neural operators

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jARyOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies periodic trajectories recorded after a long common delay or over a fixed physical-time window. Frequency differences produce increasing phase differences. The authors derive error bounds for predictors with limited parameter sensitivity and test a representation that learns the aligned waveform and period separately.

Critical strengths
  1. Theorem 1 relates the error floor to orbit variance and predictor sensitivity. Theorem 2 identifies the product of window length and frequency gap as a local scale for OOD error. The proofs in Appendices A–B support these conditional results. This adds understanding beyond the established use of phase alignment and time warping.

  2. The experiments separate alignment from period normalization and use five ODE training seeds. Appendix F.3 increases the grid size with the window and supplies spectral headroom. These controls help attribute the observed failures to the representation rather than a simple resolution limitation.

Critical weaknesses
  1. Table 2 may not evaluate the reconstructed physical-time predictor. Section 5.3 uses U_hat(phi_hat + t/T_hat). Appendix G, p.26, lines 1378–1383, instead uses U_hat(phi_hat + tau_j) against the reference at T(p) times tau_j. T_hat is absent. As written, this measures cycle shape with true-period normalization. Separate period error does not show the total trajectory error. Table 1 also uses different time coordinates, and Table 2 lacks OOD reconstruction at common physical times.

  2. The connection between the bounds and trained FNOs remains conditional. The assumption L_lambda = o(lambda) is not enforced or checked. The observed error floors are consistent with the theory, but do not establish that the trained predictors satisfy its assumptions. This affects how strongly the experiments support the proposed explanation.

Questions for authors
  1. Does Appendix G match the evaluation code? Using the existing predictors, please report reconstruction with T_hat at common physical times for IID and period-OOD samples, over one cycle and several cycles.

  2. Are the FNO experiments intended as illustrations of the conditional bounds, or as evidence that their assumptions hold? Please clarify the claim; a sensitivity check on the existing checkpoints could help.

1 comment
AuthorsOpenReview ↗

Response to Reviewer jARy

Dear Reviewer jARy,

Thank you for your valuable review, especially your careful observations about the reconstruction evaluation and the conditional experimental validation.

We address your weaknesses and corresponding questions together below.

Weakness 1 and Question 1: Physical-time reconstruction

We have rerun the reconstruction evaluation using the predicted period explicitly. For each validation parameter sample, we integrate past burn-in and select a snapshot at a random post-burn-in time. We then generate the reference trajectory over one or five true periods by integrating from this snapshot. Using the existing waveform FNO and period MLP, we predict the waveform and period, estimate the initial phase by matching the snapshot, and reconstruct

x^(t)=U^(ϕ^+t/T^(p);p).\widehat x(t)=\widehat U\left(\widehat\phi+t/\widehat T(p);p\right).

The reconstruction MSE compares this prediction with the reference at matching physical times, so it includes the effect of period prediction error.

Section 5.3 and Table 2 report the IID and period-OOD results. Appendix J provides the evaluation procedure and detailed per-seed results. We reproduce Table 2 below; entries are mean ± standard deviation across five training seeds.

SystemSplitReconstruction MSE: one true periodReconstruction MSE: five true periodsPeriod relative MSE
GoodwinIID(4.10±0.63)×10−4(4.10\pm0.63)\times10^{-4}(5.78±0.63)×10−4(5.78\pm0.63)\times10^{-4}(2.61±0.23)×10−6(2.61\pm0.23)\times10^{-6}
GoodwinOOD(3.61±0.56)×10−2(3.61\pm0.56)\times10^{-2}(6.0±1.2)×10−1(6.0\pm1.2)\times10^{-1}(1.86±0.42)×10−3(1.86\pm0.42)\times10^{-3}
Gold95IID(1.330±0.033)×10−3(1.330\pm0.033)\times10^{-3}(6.78±0.53)×10−3(6.78\pm0.53)\times10^{-3}(3.66±0.29)×10−5(3.66\pm0.29)\times10^{-5}
Gold95OOD(3.84±0.32)×10−2(3.84\pm0.32)\times10^{-2}(4.80±0.30)×10−1(4.80\pm0.30)\times10^{-1}(1.253±0.082)×10−3(1.253\pm0.082)\times10^{-3}

Period relative MSE is the sample mean of (T^−T)2/T2(\widehat T-T)^2/T^2.

Over five true periods, IID reconstruction errors remain small, while OOD errors increase substantially. Period prediction errors can accumulate into phase drift during reconstruction. Improving long-horizon reconstruction is outside the scope of this work.

Weakness 2 and Question 2: The assumption Lλ=o(λ)L_\lambda=o(\lambda)

We have added empirical Lipschitz estimates for both FM1 and FM2 in Section 4.1, with experimental details and visualization in Appendix G. For each final FNO checkpoint, we estimate the largest singular value of the parameter Jacobian at sampled validation points and take the maximum as L^λ\widehat L_\lambda. Both sweeps show a decreasing ratio L^λ/λ\widehat L_\lambda/\lambda, consistent with the expected trend.

In FM1, the ratio decreases from 1.171.17 to 0.01150.0115 for Goodwin as cc increases from 10210^2 to 10410^4, and from 1.771.77 to 0.003220.00322 for Gold95 as cc increases from 10210^2 to 10510^5. In FM2, over a 64-fold increase in WW, it decreases from 1.811.81 to 0.380.38 for Goodwin and from 1.521.52 to 0.230.23 for Gold95.

The ratio drops more in FM1, where we sweep a much wider range. Our ability to examine FM2 over a wider range is limited by computational cost: increasing WW requires proportionally more output grid points to maintain temporal resolution.

Reviewer_fsTnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on nonlinear ODE and PDE systems admitting a stable limit cycle. The problem of interest is to learn the mapping from the equation parameters to one cycle of the periodic orbit. The central point of the paper is that for a model to properly learn such a mapping, two things must hold: 1) the periodic orbits must all be aligned to some reference phase, 2) the time coordinate must be normalized by the period, and the parameter-to-rescaled waveform and parameter-to-period mappings must be learned separately.

The authors argue this point by deriving lower bounds on the population and per-sample waveform error of the trained models under regularity assumptions on both the model and the ground truth parameter-to-period/waveform mapping. Experiments using simulated data from two ODE and one PDE system confirm the theoretical error bound, with the caveat that these error bounds hold in the large parameter limit for periodic waveform preprocessing. The authors also experimentally demonstrate the efficacy of a decoupled training strategy, where two separate neural networks are used to learn the parameter-to-rescaled waveform and parameter-to-period mapping separately.

Critical strengths

1. Identification of two distinct types of failure modes stemming from an absence of phase alignment / absence of time rescaling.

As described in the Critical Weaknesses section, it is well-known from the relevant literature that to learn the mapping from a parameter vector to the limit cycle, phase alignment (otherwise the limit cycle is not uniquely specified) and time rescaling (from the limit cycle sensitivity analysis literature) are required.

What is not known, however, is how the lack of each preprocessing step affects the trained model. The findings of this paper that lack of phase alignment breaks both in-distribution (ID) and out-of-distribution (OOD) performances, whereas lack of time normalization only breaks OOD performance is somewhat expected (no phase alignment - limit cycle is not uniquely specified, so parameter to cycle mapping is ill-defined; no normalization - alignment gives well-defined mapping, limited generalization capabilities of neural networks make it hard to generalize to periods different from those seen in training) but still interesting to see, and potentially informative to future researchers facing similar problems.

2. Proposed error bounds seem novel, and are verified to experiments on simulated systems.

The derived error bounds of Theorem 1 and 2 seem to be novel, and while I have some questions regarding them (see Questions section), they seem to be backed up by experimental results of Fig 1-3. These results may be of use to theoreticians working in related fields.

Critical weaknesses

1. Improper positioning with respect to previous works in related fields.

This is the biggest concern I have with the paper. Currently, the paper's related works survey is quite thin, and all of the main findings of the paper (necessity of phase alignment & time normalization; amplified phase variation due to bad preprocessing; different failure modes stemming from lack of alignment or normalization; theoretical error bounds and experimental confirmation; proposed fix - to learn two separate mappings for normalized waveform and period) are presented as novel. This seems not to be the case.

  • Necessity of phase alignment & time normalization: This is very well-known in numerical dynamical systems literature, in shooting [1] or boundary value problem methods [2] for computing the limit cycle of a oscillator (which is essentially numerically constructing the mapping from parameters of a given ODE/PDE to its limit cycle). It is common knowledge that the limit cycle is not uniquely determined due to its phase degree of freedom (a time-shifted limit cycle is another limit cycle of the system), and thus both shooting and BVP methods require a phase condition to remove this ambiguity. BVP methods also set the period and the time-normalized limit cycle as the unknowns and explicitly solve for them, which is a direct analogue of the time normalization discussed in the paper.

  • Amplified phase variation argument in page 3: While stated a bit differently, this corresponds to the sensitivity analysis of the limit cycle waveform on the parameters, which has been investigated in prior works such as [3]. Note their result for the sensitivity (eq 2.14) splits into an unbounded time-proportional period-dependent term, and a "cleaned out" term related to the parameter sensitivity of the waveform shape. The paper also mentions rescaling time as a remedy for the unbounded sensitivity term.

  • Learning separate mappings for the time-normalized waveform and period: While there is no work using neural networks for limit cycle systems, this is also an idea found in different parts of the literature. For example, [4] uses three separate surrogate models to learn the parameter-to-inverse period / phase shift / normalized waveform mappings independently, and shows that this outperforms using a single non-normalized time-based surrogate.

In light of this literature landscape, I strongly believe that the paper's positioning and related works survey must substantially be improved. Particularly, the paper's novel contributions (differing effects of bad preprocessing; theoretical error bounds) must be clearly delineated to those that are more incremental.

  • [1] P. Deuflhard. Computation of periodic solutions of nonlinear ODEs. BIT Numerical Mathematics 24 (1984).
  • [2] Y. Kuznetsov. Elements of applied bifurcation theory. Chapter 10.1.4. Springer International Publishing (2023).
  • [3] A. Wilkins et al. Sensitivity analysis for oscillating dynamical systems. SIAM Journal on Scientific Computing 31, 4 (2009).
  • [4] C. V. Mai and B. Sudret. Surrogate models for oscillatory systems using sparse polynomial chaos expansions and stochastic time warping. SIAM/ASA Journal on Uncertainty Quantification 5, 1 (2017).

2. Theoretical error bounds confirmed only for a single model class.

As presented, the theoretical error bounds do not depend strongly on a particular model class, but experiments are performed with a single model - the Fourier neural operator (FNO). For a proper validation of the error bounds, I believe that few other model types (1D convolutional neural networks, or multilayer perceptrons) should be tested as well. This is particularly so due to the following reasons:

  • FNOs have a frequency cutoff, which directly affects the frequency bandwidth of the model outputs. For the prediction of periodic structures, this could have an unintended side-effect.
  • For ODE problems, the inputs fed into the FNO seem to be (t,p(t))(t, \mathbf{p}(t)), where the time-dependent parameter channels $ \mathbf{p}(t)$ are constant signals. This is far from the conventional usage of FNOs, which takes in time and/or space-varying signals, to transform them (in the frequency domain) into output predictions.
Questions for authors

Q1.

I feel like the paper could benefit from a stronger motivation for the problem. For a given ODE form, numerical dynamical systems techniques (shooting, BVP collocation) can compute the period and the normalized waveform. What are some practical situations one would want to use neural networks surrogates instead?

Q2.

The established error bounds hold in the settings of c,W→∞c, W \rightarrow \infty. How realistic are these limits? Do these limits adequately represent the preprocessing parameters encountered in typical training, and thus be able to explain the decreased neural network performance? For example, WW is the longest period in the training dataset, so W→∞W\rightarrow \infty is a situation where a training dataset contains some orbits with extremely large period than others. When would one encounter such datasets? Can authors provide a histogram of periods of some of their training datasets?

Q3.

Unlike Fig 2, where the error gradually increases, Fig 1 shows a sudden jump at some threshold cc value. Can the authors explain why?

1 comment
AuthorsOpenReview ↗

Response to Reviewer fsTn

Dear Reviewer fsTn,

Thank you for your valuable review. We particularly appreciate your detailed discussion of the relevant literature.

Weakness 1. Positioning with respect to previous work

We agree that the original manuscript did not clearly distinguish our contributions from established work. References [2] and [4] in your review were already cited in the original submission, but their relationship to our work needed clearer explanation.

We have expanded the related-work discussion to include all four references you suggested (Lines 46–63). We now explicitly position our study as motivated by the established issues of phase alignment and time normalization. Our contributions are the theoretical error bounds and their empirical validation through controlled ablations and sweeps over the recording time scales. The period–waveform decomposition is clearly identified as an established approach and presented as one example of target design guided by our theorem.

Weakness 2. Validation beyond a single model class

We have repeated both the cc- and WW-sweeps with a one-dimensional CNN and an MLP, using five training seeds for each setting. The results are reported in Section 4.4 and Appendix F. Both architectures exhibit behavior consistent with the error-floor results observed for the FNO, providing evidence that the findings are not specific to its architecture or Fourier-mode cutoff.

Q1. Motivation for neural surrogates

Our motivation is to benefit from operator learning, particularly rapid, repeated predictions at new parameter values after offline training. This does not imply that operator learning outperforms shooting or BVP methods on a given problem; such a comparison is outside the scope of this work. We study how recording and representation choices affect prediction accuracy when an operator-learning approach is used.

Q2. Practical relevance of the asymptotic bounds and period distributions

Theorem 1 provides an asymptotic bound, but our ablation studies demonstrate substantial errors at practical window lengths. Specifically, we set WW to the largest period in the training set, and the fixed-window OOD validation MSE already reaches approximately 0.20.2 on both ODE systems (Lines 162–186, Table 1, section 2.1). Thus, substantial error can occur without an extremely large observation window.

This choice represents a natural recording rule: given a training dataset, choose a common window long enough to contain one full period of every training trajectory. Also, increasing WW in the sweep does not require introducing trajectories with increasingly large periods; it means observing the same periodic trajectories over longer windows.

The period distributions of the full Goodwin and Gold95 datasets are provided in Appendix A.4.

Q3. Apparent sudden jump in the cc-sweep

The apparent sharpness of the transition is partly attributable to the spacing and range of the sampled sweep values. The cc-sweep spans values from 11 to 10510^5, whereas the WW-sweep covers only a 64-fold range. The more widely spaced cc values can therefore make the transition appear abrupt.

The difference in sweep ranges has a computational reason. Increasing WW requires proportionally more output grid points to maintain the same solution resolution. Increasing cc shifts the starting phase without increasing the length of the recorded interval, so it permits a much wider sweep. The plotted jump should therefore not be interpreted as evidence of a discontinuous transition.

Reviewer_NTeAOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors are interested in learning periodic orbits of dynamical systems from their ``waveforms'' (which may be the same as orbits in the language of the paper) for a parameterized class of dynamical systems. The derive lower bounds on errors in learning periodic orbits and from experiments conclude that the time over which training data are present and how fast the period and phase of the periodic orbits vary with the parameter determine whether they can be learned accurately.

Critical strengths

The central motivation of the paper: which is use to learn the parameters-to-observations map in the case of periodic orbits is interesting to study both theoretically and empirically.

Critical weaknesses

The paper is written in a way that makes it difficult to understand what the main contributions are. Here are some helpful questions that can hopefully clarify the contributions.

  1. When the targets vary smoothly in parameter space, we can learn them? If that is the case, this has nothing to do with dynamics: this is a standard result in operator approximations.

  2. How are decoupled targets defined? Why does decoupling help one learn periodic orbits?

  3. What happens near bifurcations, when the targets presumably vary nonsmoothly with parameters?

Questions for authors
  1. What are the specifics of the learning problem? Why are the results not loss dependent?

  2. In general, there are many sentences that sound mathematically plausible but stated without explanation: every theorem and proof are examples. Could you please clarify the meaning behind them in a way human ML audience would understand? e.g., hyperbolicity, Poincare maps etc from dynamics, C^k continuations, local continuations etc from analysis make the presentation very opaque.

1 comment
AuthorsOpenReview ↗

Response to Reviewer NTeA

Dear Reviewer NTeA,

Thank you for your review. We address your concerns below.

1. When the targets vary smoothly in parameter space, can we learn them?

Our finding is that smoothness alone does not prevent failure. Even when a target varies smoothly with the parameters, it can become increasingly sensitive to them. A predictor whose sensitivity grows more slowly cannot track this variation and incurs the error floor established in Section 3.2.

2. How are decoupled targets defined? Why does decoupling help one learn periodic orbits?

As described in Section 5.1, the decoupled targets are the period and the phase-aligned, normalized waveform. We use an MLP to learn the parameter-to-period map and an FNO to learn the parameter-to-waveform map. This representation removes the large time-scale parameter from the targets, thereby avoiding the error-floor mechanism identified in Section 3.2. Furthermore, Section 5.2 establishes continuity and neural-network approximability of these targets under the stated ODE assumptions.

3. What happens near bifurcations, when the targets presumably vary nonsmoothly with parameters?

Our study is restricted to parameter regimes without bifurcations, where the period and aligned waveform vary continuously with the parameters. This setting allows us to isolate the error caused by accumulating phase differences from difficulties associated with bifurcations or loss of regularity. The key point is that the identified failure can occur even on a regular periodic branch. Behavior at bifurcations is outside the scope of our results.

4. What are the specifics of the learning problem? Why are the results not loss dependent?

We study the mapping from system parameters to periodic trajectories under different recording and representation choices. Our error bounds concern squared L2L^2 prediction error, and the proofs use the associated inner-product structure. They apply to any predictor satisfying the stated assumptions, regardless of the loss used to train it. Thus, the results are independent of the training loss.

5. Could the mathematical terminology and explanations be made more accessible to an ML audience?

We agree that additional background is needed for readers whose expertise is primarily in machine learning. We have added a dedicated terminology appendix (Appendix C), which explains the dynamical-systems and analysis terminology used in the proofs, including hyperbolicity, Poincaré maps, local CkC^k continuations, and so on. We have also edited intuitive interpretations alongside the main theorems to explain their meaning and the reasoning behind them.

To further improve readability, we now explain the intuition behind the error-floor mechanism at the beginning of Section 3 and present the setup and problem formulation in a separate subsection (Section 3.1).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.