acceptodds
Under review as a conference paper at ICLR 2027

EchoKoop: Koopman Forecasting with Learned Lifting and Frozen Multiscale Dynamics

Abstract

Neural Koopman forecasters typically learn both nonlinear observables and a finite-dimensional latent transition operator. While expressive, the learned operator can be parameter-intensive and difficult to stabilize under repeated long-horizon rollout. We introduce EchoKoop, a forecasting framework based on the principle of learning the lift while freezing the flow. Instead of optimizing Koopman transition matrices, EchoKoop learns nonlinear observables that are propagated by fixed multiscale normal operators. These operators generate flows with different memory ranges, while the learned encoder, modal projections, selectors, and decoders adapt observed histories to this dynamical basis. Removing trainable transition matrices eliminates their parameter cost and spectral drift during training. Across ten long-term forecasting benchmarks, EchoKoop attains the lowest dataset-averaged MSE on six; controlled ablations show no consistent gain from training its transitions.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series forecasting, Koopman operator learning, dynamical systems, latent linear dynamics, multiscale modeling

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_qn2yOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2(−1)
Soundness
2
Read review
Paper summary

EchoKoop investigates whether neural Koopman forecasters need trainable latent transition matrices. The method learns the encoder and decoder while using fixed multiscale transition operators built from damped rotations, together with a phase-aligned initialization scheme. The paper evaluates this design on forecasting benchmarks, chaotic systems, and physical systems, with ablations examining each component's contribution.

Critical strengths
  1. A clear and worthwhile research question. The paper asks if learning the latent transition is needed when the surrounding representation is learned. Exploring fixed dynamics is a bold and interesting direction, with a natural connection to reservoir computing. The controlled comparisons make this question scientifically useful, even without consistent improvements across every benchmark.

  2. A simple and effective initialization scheme. The phase-aligned initializer is a particularly appealing component. It provides a concrete way to connect the observed history to the prescribed dynamics, and the ablations help establish its contribution. The detailed description in the appendix is also valuable.

Critical weaknesses
  1. The central claim needs stronger support, and the paper's breadth makes it harder to assess. The paper asks a clear and worthwhile question: does the latent transition operator need to be learned? Table 13 shows near-parity: training the operators improves average MSE. This supports the competitiveness of the fixed bank in the tested configuration, but leaves uncertainty about when learning the operator is beneficial. A nonsignificant difference also does not establish equivalence. Since the trainable variant starts from the frozen bank, reporting how much the operators actually change and testing sensitivity to the learning rate would make this comparison more informative.

    The additional chaotic and physical-system experiments do not resolve this uncertainty or even help much with the main thesis. Their connection to the central question needs to be clearer, particularly given ConvLSTM's consistent advantage across Table 19. Also FlowMixer's weak results, including the failed cylinder runs, warrant clarification. The original study reports better performance than ConvLSTM on fluid forecasting. Although the experimental settings differ, the discrepancy raises questions about the spatial representation, model capacity, and training procedure used here.

In essence, a more focused study of long-horizon forecasting, with stronger controlled comparisons of frozen and learned dynamics, would give the main idea more convincing support. The Koopman interpretation also deserves clarification, since forecast-only training does not explicitly enforce consistency between encoded observations and the latent rollout.

  1. The baseline protocol and released code leave important questions about fairness and reproducibility. EchoKoop's strongest gains are concentrated on PEMS, evaluated with only 12 input steps (L=12). Elsewhere, the input length is fixed at L=96. These short windows make the baseline adaptations unclear. In particular, how were FlowMixer and Koopa configured for horizons of H =192 to 720? Were inputs padded, were forecasts generated autoregressively in blocks of 96, or were the methods adapted in some other way? Padding changes the input presented to the model, while blockwise forecasting can compound prediction errors. The exact procedure is therefore necessary to interpret the reported differences. For further clarity: FlowMixer needs an input at least of the same length as the horizon or larger (L>=H), while Koopa needs twice the input to be twice the horizon (L=2H).

    Overall the comparison would benefit from explicit adaptation details, configuration files, and comparable tuning budgets. It remains also unclear whether the baselines were tuned for these short windows or retained their default settings. Comparisons using longer contexts (typically L=512, or 1440 like Koopa) would help establish whether the rankings are robust. The supplied code appears to cover EchoKoop mainly. Providing these materials and reconciling differences between the released configurations and the paper would make the results reproducible and strengthen confidence in the conclusions.

Questions for authors
  1. Baseline implementation: How were FlowMixer and Koopa configured when the prediction horizon exceeded the input length (H>L)? Did you use padding, autoregressive forecasting in shorter blocks, or architectural changes?

  2. Frozen versus learned transitions: How much did the trainable operators change from their frozen initialization? How sensitive was this comparison to the learning rate, and how do the differences in Table 13 compare with variability across runs?

  3. Koopman interpretation: With forecasting loss alone, what evidence shows that applying the latent transition produces states consistent with encoding the corresponding future observations?

  4. Chaotic and physical systems: Which results specifically demonstrate the benefit of freezing the transition operator? Could you also plot predicted and reference trajectories for the Lorenz attractor and report EchoKoop's valid prediction horizon in Lyapunov times?

7 comments
AuthorsOpenReview ↗

Response to Reviewer qn2y [1/4]

We sincerely thank Reviewer qn2y for the detailed questions and constructive suggestions. Below, we address the comparison between frozen and trainable transitions, the adaptation and reproducibility of baselines under short-history settings, and what the chaotic-system and physical-field experiments establish.

Q1: … , but leaves uncertainty about when learning the operator is beneficial. A nonsignificant difference also does not establish equivalence. … , reporting how much the operators actually change and testing sensitivity to the learning rate would make this comparison more informative.

A1: We agree that Table 13 shows the frozen bank is competitive under the original training protocol, and that the nonsignificant result does not establish equivalence. The average gain from learning the transition was small relative to seed-to-seed variation, although all three paired seeds improved on PEMS08 at H = 96. We report the operator diagnostics and learning-rate sensitivity in our response to Frozen versus learned transitions below.

Q2: The additional chaotic and physical-system experiments do not resolve this uncertainty or even help much with the main thesis. … Also FlowMixer's weak results, including the failed cylinder runs, warrant clarification. … Although the experimental settings differ, the discrepancy raises questions about the spatial representation, model capacity, and training procedure used here.

A2: The purpose of these experiments is to evaluate frozen structured flows beyond standard forecasting benchmarks by testing finite-horizon accuracy and preservation of selected long-run statistics in low-dimensional dynamical systems.

Due to compatibility issues between TensorFlow and our experimental environment, we ported the core operations from the official TensorFlow/CuPy Cylinder notebook, using the official PyTorch demo as a reference.The adapted implementation retains mixing along the time and feature dimensions, reversible instance normalization, and spatial downsampling with a stride of 3.

The official Cylinder notebook uses 64 input frames and 64 output frames. In our evaluation, the model uses 16 input frames, is trained on 20-step free rollouts, and is evaluated at forecast steps 5, 20, and 50. The training loss is the field MSE plus 0.05 times the vorticity MSE. The three runs stopped at epochs 10, 6, and 2, respectively, after encountering non-finite gradient norms. Table 24 marks these results as missing. We documented the differences between the two protocols in Appendix J.2.

Q3: Comparisons using longer contexts (typically L=512, or 1440 like Koopa) would help establish whether the rankings are robust. ... Providing these materials and reconciling differences between the released configurations and the paper would make the results reproducible and strengthen confidence in the conclusions.

A3: We repeated the comparison with all three methods using the same longer input history. We kept the original forecast horizons and ran each configuration with three random seeds. Koopa used seg_len = H. FlowMixer generated the forecast in a single pass without blockwise recursion and used its original supervision over the full shifted window.

With the longer shared input history, FlowMixer achieves the lowest average MSE and MAE on ETTm1 and Weather. EchoKoop still outperforms both baselines on PEMS04 and PEMS08.

All three methods use the same input history length (L = 192). Each cell reports MSE / MAE. “Average” denotes the mean across the four forecast horizons. Bold indicates the lowest value for each metric.

DatasetForecast horizon (H)EchoKoopKoopaFlowMixer
PEMS04120.0945 / 0.20080.0792 / 0.19110.1158 / 0.2292
PEMS04240.1108 / 0.21800.0916 / 0.20570.1751 / 0.2852
PEMS04480.1312 / 0.23890.1184 / 0.23490.2433 / 0.3511
PEMS04960.1525 / 0.26150.3446 / 0.42100.2622 / 0.3658
PEMS04Average0.1222 / 0.22980.1584 / 0.26310.1991 / 0.3078
PEMS08120.0916 / 0.19570.0910 / 0.19710.1131 / 0.2242
PEMS08240.1157 / 0.22030.1205 / 0.22550.1897 / 0.2930
PEMS08480.1590 / 0.25680.1670 / 0.26210.2950 / 0.3720
PEMS08960.2303 / 0.30510.2456 / 0.30150.3759 / 0.4101
PEMS08Average0.1492 / 0.24450.1560 / 0.24650.2434 / 0.3248
AuthorsOpenReview ↗

Response to Reviewer qn2y [2/4]

Q4: Baseline implementation: How were FlowMixer and Koopa configured when the prediction horizon exceeded the input length (H>L)? Did you use padding, autoregressive forecasting in shorter blocks, or architectural changes?

A4: We retained the core architectures of Koopa and FlowMixer.

Koopa. The relation L = 2H is an experimental setting used in Table 1 of the original paper. Koopa’s original code allows the segment length S to be set independently of the forecast horizon H. In the Time-Series-Library implementation we used, S defaults to H. We therefore restored S as an independent parameter, using L = 96 and S = 24 for the standard benchmarks and L = 12 and S = 4 for PEMS. The ratio S = L/4 used on the standard benchmarks was also evaluated in the original paper (Figure 8, p. 16).

FlowMixer. To give all methods the same length of observed history, we retained FlowMixer’s core temporal and feature mixing operations and used a blockwise rollout with shared parameters when H > L. We compute the normalization statistics using only the observed history and keep them fixed throughout the forecast. We repeatedly apply the same mixing block in normalized space to generate the next L steps, then concatenate the predicted blocks and retain the first H steps. For example, with L = 96 and H = 720, eight applications yield seven full 96-step blocks and the first 48 steps of the final block.

The anonymous baseline supplement includes the adapted Koopa and FlowMixer implementations, configuration files, and available run commands.

Q5: Frozen versus learned transitions: How much did the trainable operators change from their frozen initialization? How sensitive was this comparison to the learning rate, and how do the differences in Table 13 compare with variability across runs?

A5: In the extended experiments, both trainable variants moved away from their frozen initialization. The dense variant (which learns every entry of each transition matrix) became non-normal, while the structured variant preserved normality and contractivity as it learned the frequencies and half-lives. In the original Table 13 ablation, the mean paired test MSE difference across the 24 settings was −0.00029, small relative to the across-seed standard deviation of 0.00190, with only one of three seeds showing an overall improvement. The original ablation did not retain the trained transition matrices, so we report operator diagnostics from the extended experiments, which started from the same frozen modal bank. At the learning rates selected by validation MSE, the mean relative Frobenius distance from initialization ranged from 0.0969 to 0.8912 for the dense variant and from 0.0018 to 0.0950 for the structured variant. Averaging the maximum sampled power gain in each of the six settings, including step zero, gives 10.63 and 1.00, respectively. The corresponding mean normalized Henrici departures from normality were 0.1469 and 0.0000 (Table 18).

Dataset (H)Dense 5 × 10⁻⁵Dense 2 × 10⁻⁴Dense 10⁻³Structured 10⁻⁴Structured 10⁻³
ETTm1 (96)0.1192 / 0.0740†0.2845 / 0.12860.7887 / 0.30440.0032 / 0.0000†0.0140 / 0.0000
ETTm1 (720)0.0969 / 0.0478†0.3822 / 0.09030/3 completed0.0018 / 0.0000†0.0049 / 0.0000
Weather (96)0.3849 / 0.10640.8912 / 0.2329†0.9595 / 0.43630.0099 / 0.00000.0216 / 0.0000†
Weather (720)0.1373 / 0.0531†0.3912 / 0.10620/3 completed0.0042 / 0.00000.0086 / 0.0000†
PEMS08 (12)0.5679 / 0.21200.7992 / 0.4326†0.9516 / 0.63010.0443 / 0.00000.0950 / 0.0000†
PEMS08 (96)0.3935 / 0.1677†0.6434 / 0.29570.9483 / 0.5440 (1/3 completed)0.0201 / 0.00000.0378 / 0.0000†

Each cell reports the mean relative Frobenius distance from initialization / mean normalized Henrici departure from normality. Both metrics are averaged over operators and completed seeds at their best validation checkpoints. † marks the learning rate selected by validation MSE.

Dataset (H)FrozenDense 5 × 10⁻⁵Dense 2 × 10⁻⁴Dense 10⁻³Structured 10⁻⁴Structured 10⁻³
ETTm1 (96)0.3240.321†0.3260.3310.323†0.324
ETTm1 (720)0.4310.430†0.4370/3 completed0.431†0.432
Weather (96)0.1800.1750.177†0.1780.1800.180†
Weather (720)0.3730.373†0.3610/3 completed0.3820.381†
PEMS08 (12)0.0780.0770.077†0.0780.0780.078†
PEMS08 (96)0.3870.369†0.3761/3 completed0.3780.372†

Numerical entries report mean test MSE over three paired seeds. † marks the learning rate selected by validation MSE for each dataset, horizon, and trainable variant.

AuthorsOpenReview ↗

Response to Reviewer qn2y [3/4]

The overall gain in the original ablation was small relative to seed-to-seed variation. In Table 13 of the original manuscript (Table 15 in the revised manuscript), the dense trainable variant achieved lower mean test MSE in 14 of 24 settings and reduced macro-averaged test MSE by 0.11%. We paired 72 runs by dataset, forecast horizon, and seed. For the overall comparison, we first averaged test MSE over the 24 settings within each seed, then computed the difference between the dense and frozen variants.

Training budgetSettingMean paired MSE differenceSample SD of paired differencesSeeds with lower MSE
Original, up to 25 epochsMacro-average over 24 settings−0.000290.001901/3
Original, up to 25 epochsPEMS08 (H = 96)−0.024140.006053/3
Extended, up to 50 epochsETTm1 (H = 720)−0.001400.006841/3
Extended, up to 50 epochsWeather (H = 720)−0.000260.021561/3
Extended, up to 50 epochsPEMS08 (H = 96)−0.017740.006543/3

Each difference is the dense trainable variant’s test MSE minus the frozen variant’s test MSE for the same seed. Negative values indicate improvement. The extended experiments use the learning rate selected by validation MSE for each setting.

Consistent gains depended on the learning rate. A larger training budget did not increase the average gain on the six shared settings. We raised the training cap from 25 to 50 epochs and increased early-stopping patience from 3 to 10. At 5 × 10⁻⁵, the dense variant improved all six shared settings under both budgets. Its reduction in macro-averaged MSE relative to the corresponding frozen variant went from 1.84% to 1.60%. Under the extended budget, 8 of 18 runs encountered non-finite values when the dense transition learning rate was raised to 10⁻³. The structured variant completed all runs at both learning rates.

Transition typeTransition learning rateSettings with three completed seedsSettings with lower MSEMacro-averaged MSE change vs. frozen
Dense5 × 10⁻⁵6/66/6−1.60%
Dense2 × 10⁻⁴6/64/6−1.07%
Dense10⁻³3/61/3+0.81%
Structured10⁻⁴6/65/6−0.07%
Structured10⁻³6/62/6−0.35%

For each learning rate, we compute the macro-averaged MSE change against the frozen variant on the settings for which all three seeds completed. The full learning-rate sweep and operator diagnostics are reported in Appendix G.3, Figure 9, and Tables 17–18.

Reviewer_xYFyOpenReview ↗
Rating
4(+2)
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

I appreciate the authors' effort on this work. I have read the paper carefully, and below is my summary of the paper and its contributions: The paper introduces EchoKoop, a Koopman-style forecaster built on the principle ``learn the lift, freeze the flow''. Instead of learning the latent transition operator, as neural Koopman forecasters usually do, EchoKoop fixes it to a bank of damped rotations. These operators are normal and contractive so transient growth through ∥Kh∥2\|K^h\|_2 and the non-identifiability of (E,K,D)(E, K, D) are avoided by construction. Only the multiscale encoder, the softmax selector over half-lives, the decoders, and a cross-scale correction module are trained. Empirically, the authors conducted experiments widely on 10 TS benchmarks, dynamical systems, physical fields. They report that in the studied settings learning the latent transition gives no consistent gain, while multiscale rollout and phase-consistent initialization have larger effects. Overall, I think this is a careful and interesting study to me, and I am currently borderline (a little bit towards reject) on it, mainly because of the concerns below.

Critical strengths

I really appreciate the authors' effort on the paper. Here are some bullet points that I believe are the strengths of the paper:

(S1) Well-motivated idea: Fixing the transition to normal, contractive operators addresses concrete problems of learned Koopman transitions (non-identifiability, non-normal transient growth, and O(d2)O(d^2) parameters). The principle is simple, well specified, and directly testable.

(S2) The math derived in the paper is correct, and the claims are scoped carefully. I don't see much of overclaim inside the paper (if yes, then little)

(S3) Broad experimental effort: The evaluation covers ten forecasting benchmarks, dynamical systems, physical fields across architectures and seeds. The reporting, appendices and statistical analysis are detailed and explicit.

Critical weaknesses

Here are some bullet points on the shortcomings of the paper:

Theoretically:

(W1) The theory does not cover the long-horizon regime. Theorem 1 requires at least H distinct modes per native grid, and the paper acknowledges that the configured bank doesn't need to meet this at every horizon. By my count from Appendix C.2 and Table 6, the condition fails at all three scales at H=720 on ETT/Weather (e.g., about 570/282/138 distinct modes vs. 720/360/180 needed) and at H=96 on PEMS (about 66/30/6 vs. 96/48/24), the setting with the largest reported margins. The theorem is also generic, since any H distinct modes span, so it does not distinguish frozen from trainable transitions, and the error bound covers the latent rollout, not the learned decoder. The theory motivates the design but does not explain the long-horizon behavior.

Empirically:

(W2) The benefit for standard long-term forecasting is limited. On the six L=96 benchmarks, EchoKoop has the best dataset-averaged MSE on only two (ETTh1, ETTm1) and the best MAE on one (ETTm1). No EchoKoop-baseline comparison survives Holm correction, and the "six of ten" headline rests on four PEMS datasets with L=12.

(W3) I think the central ablation is too weak to carry the main claim. The key finding of the paper is that training the transition gives no consistent gain, but I only see one training protocol for this test: the same learning rate of 5e-5, at most 25 epochs, and early stopping with patience 3. I could not find any drift or spectral statistics for EchoKoop's own trained matrices, so I do not know if they moved at all from their starting point. If they barely moved, the result says very little about whether learning the transition helps. The paper is missing two obvious controls: a structured trainable bank that learns only the frequencies and half-lives, and a no-dynamics baseline such as a direct multiscale linear readout.

(W4) The Koopman framing is largely nominal. The model is trained on forecast loss only, with no latent-linearity term, and the authors state the modes are a forecasting basis, not Koopman spectrum estimates. It is close to a fixed-eigenvalue complex-diagonal linear recurrence with a nonlinear encoder and decoder. SSMs and linear RNNs are discussed, but no S4D/LRU-style baseline with fixed or structured-trainable eigenvalues is compared.

(W5) The general-purpose baselines are also older (PatchTST, iTransformer, TimesNet, DLinear), and recent strong models such as TimeMixer, TimeMixer++ and OLinear are not compared, so I cannot tell whether "competitive" holds against the current state-of-the-art. (Citations are added below).

References:

Wang, S., Wu, H., Shi, X., Hu, T., Luo, H., Ma, L., Zhang, J. Y., Zhou, J. TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting. ICLR 2024. arXiv:2405.14616.

Wang, S., Li, J., Shi, X., Ye, Z., Mo, B., Lin, W., Ju, S., Chu, Z., Jin, M. TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis. ICLR 2025. arXiv:2410.16032.

Yue, W., Liu, Y., Wang, H., Li, H., Ying, X., Guo, R., Xing, B., Shi, J. OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain. NeurIPS 2025. arXiv:2505.08550.

Questions for authors

(Q1) Theorem 1 needs at least H distinct modes per grid, but by my count the default banks fall short at H=720 (ETT/Weather) and H=96 (PEMS). How do you explain the long-horizon gains when the condition fails?

(Q2) How far did EchoKoop's own trainable matrices move from initialization in the ablation, and does "no consistent gain" hold with a larger learning rate or longer training?

(Q3) Can you add a structured trainable bank (learning only frequencies and half-lives) and a no-dynamics multiscale linear readout as controls?

(Q4) Can you compare empirically against an S4D/LRU-style model with frozen or structured-trainable eigenvalues?

(Q5) Can you include recent strong general forecasters (e.g., TimeMixer, OLinear) on the L=96 benchmarks?

4 comments
AuthorsOpenReview ↗

Response to Reviewer xYFy [1/4]

We sincerely thank Reviewer xYFy for the careful review and insightful questions, particularly for checking the modal counts in the long-horizon settings and the training protocol for the transition ablation. Below, we clarify the scope of our theoretical guarantees and report results from the extended ablations and new baseline comparisons.

Q1: Theorem 1 needs at least H distinct modes per grid, but by my count the default banks fall short at H=720 (ETT/Weather) and H=96 (PEMS). How do you explain the long-horizon gains when the condition fails?

A1: The two long-horizon settings you identified indeed fall short of the full-row-rank condition in Theorem 1. The theorem gives the condition for exact representation of arbitrary trajectories on a single native time grid with a linear readout and freely chosen modal coefficients. It applies to both fixed and trainable dictionaries that meet the same rank condition. The new projection result in Appendix B.2 shows that, for a dictionary of any rank, the minimum approximation error in this setting equals the distance from the target trajectory to the span of the modes. Thus, even with fewer modes than forecast steps, the approximation error can be small if the target trajectory lies close to that span.

Specifically, let VV be the modal matrix on one native grid, yy the target trajectory, and c⋆=V†yc^\star=V^\dagger y the minimum-norm least-squares coefficients. For any coefficient estimate c^\hat c, Proposition 1 gives

∥Vc^(X)−y∥22=∥(I−VV†)y∥22⏟error due to the fixed modal span+∥V(c^(X)−c⋆)∥22⏟trajectory error due to coefficient estimation.\lVert V\hat{c}(X)-y\rVert_2^2 = \underbrace{\lVert (I-VV^\dagger)y\rVert_2^2}_{\text{error due to the fixed modal span}} + \underbrace{\lVert V(\hat{c}(X)-c^\star)\rVert_2^2}_{\text{trajectory error due to coefficient estimation}}.

The first term measures how well the dictionary can approximate the target. The second measures the additional error from the choice of coefficients.

Fixed normal contraction operators control error amplification during latent rollouts. Appendix B.4 uses ∥Kn∥2=ρ(K)n≤1\lVert K^n\rVert_2=\rho(K)^n\leq 1 to bound the propagation of initial-state errors and stepwise dynamics residuals. This property follows from normality and the spectral-radius constraint and holds for modal dictionaries of any rank. Trainable operators that satisfy the same structural constraints have the same property. Appendix B.5 extends the analysis to the full multiscale readout. When the readout is Lipschitz continuous, the bound combines the readout error on a reference latent trajectory with the latent trajectory error scaled by the readout’s Lipschitz constant.

The long-horizon gains are measured empirically on the full EchoKoop model. In the horizon-by-horizon ablation on PEMS04, retaining only the finest scale increases forecast error, providing evidence that the multiscale design contributes to performance on this task.

Q2: The benefit for standard long-term forecasting is limited. On the six L=96 benchmarks, EchoKoop has the best dataset-averaged MSE on only two (ETTh1, ETTm1) and the best MAE on one (ETTm1). No EchoKoop-baseline comparison survives Holm correction, and the "six of ten" headline rests on four PEMS datasets with L=12.

A2: On the original ten-dataset benchmark, the omnibus test detected differences in method rankings, but none of the pairwise comparisons between EchoKoop and the baselines remained significant after correction for multiple comparisons. Given the limited number of dataset-level pairs, we report the specific settings in which EchoKoop performs better.

We report the six datasets with L = 96 separately from the short-history PEMS benchmarks. After adding TimeMixer and OLinear, ETTm1 is the only L = 96 dataset on which EchoKoop achieves the lowest MSE averaged across forecast horizons. The gap is clearest at H = 720, where EchoKoop, TimeMixer, and OLinear achieve MSEs of 0.431, 0.453, and 0.461, respectively. OLinear achieves lower MAE than EchoKoop at all four horizons on ETTm1 and lower horizon-averaged MAE on each of the six datasets.

AuthorsOpenReview ↗

Response to Reviewer xYFy [2/4]

Q3: How far did EchoKoop's own trainable matrices move from initialization in the ablation, and does "no consistent gain" hold with a larger learning rate or longer training?

A3: Thank you for pointing out that the original ablation used a single training protocol. The dense variant (which learns every entry of each transition matrix) optimizes every entry of the transition matrices at all three scales. Its forecasting gains vary with the learning rate, dataset, and forecast horizon.

A larger training budget did not increase the average gain on the six shared settings. In the original ablation, the frozen and dense trainable variants started from the same modal bank, used an initial learning rate of 5 × 10⁻⁵, and were trained for up to 25 epochs with an early-stopping patience of 3. The dense variant achieved lower mean test MSE in 14 of the 24 settings, and the frozen variant led in the other 10. Across all 24 settings, macro-averaged test MSE decreased from 0.253532 to 0.253243, a reduction of 0.11%. On the six settings shared with the extended experiment, the dense variant improved mean test MSE in every setting under the original budget. With up to 50 epochs, an early-stopping patience of 10, and a transition learning rate of 5 × 10⁻⁵, all six settings again improved. With the larger training budget, the macro-averaged MSE reduction relative to the frozen variant went from 1.84% to 1.60%, so the overall gain did not increase.

Both trainable variants moved from initialization, and dense training introduced non-normality. In the extended experiment, we swept three transition learning rates for the dense variant and two for the structured variant. For each setting, we selected the rate with the lowest mean best validation MSE among those with three completed seeds, then evaluated each seed at its best validation checkpoint. At the selected learning rates, the mean relative Frobenius distance from initialization ranged from 0.0969 to 0.8912 for the dense operators and from 0.0018 to 0.0950 for the structured operators. The structured operators remained normal and contractive. Across the six settings, at the validation-selected learning rates, the dense and structured variants had average maximum sampled power gains of 10.63 and 1.00, respectively, including step zero. Their mean normalized Henrici departures from normality were 0.1469 and 0.0000, respectively (Table 18).

Dataset (H)Dense 5 × 10⁻⁵Dense 2 × 10⁻⁴Dense 10⁻³Structured 10⁻⁴Structured 10⁻³
ETTm1 (96)0.1192 / 0.0740†0.2845 / 0.12860.7887 / 0.30440.0032 / 0.0000†0.0140 / 0.0000
ETTm1 (720)0.0969 / 0.0478†0.3822 / 0.09030/3 completed0.0018 / 0.0000†0.0049 / 0.0000
Weather (96)0.3849 / 0.10640.8912 / 0.2329†0.9595 / 0.43630.0099 / 0.00000.0216 / 0.0000†
Weather (720)0.1373 / 0.0531†0.3912 / 0.10620/3 completed0.0042 / 0.00000.0086 / 0.0000†
PEMS08 (12)0.5679 / 0.21200.7992 / 0.4326†0.9516 / 0.63010.0443 / 0.00000.0950 / 0.0000†
PEMS08 (96)0.3935 / 0.1677†0.6434 / 0.29570.9483 / 0.5440 (1/3 completed)0.0201 / 0.00000.0378 / 0.0000†

Each cell reports the mean relative Frobenius distance from initialization / mean normalized Henrici departure from normality. Both metrics are averaged over operators and completed seeds at their best validation checkpoints. † marks the learning rate selected by validation MSE.

Dataset (H)FrozenDense 5 × 10⁻⁵Dense 2 × 10⁻⁴Dense 10⁻³Structured 10⁻⁴Structured 10⁻³
ETTm1 (96)0.3240.321†0.3260.3310.323†0.324
ETTm1 (720)0.4310.430†0.4370/3 completed0.431†0.432
Weather (96)0.1800.1750.177†0.1780.1800.180†
Weather (720)0.3730.373†0.3610/3 completed0.3820.381†
PEMS08 (12)0.0780.0770.077†0.0780.0780.078†
PEMS08 (96)0.3870.369†0.3761/3 completed0.3780.372†

Numerical entries report mean test MSE over three paired seeds. † marks the learning rate selected by validation MSE for each dataset, horizon, and trainable variant.

AuthorsOpenReview ↗

Response to Reviewer xYFy [3/4]

Consistent gains depended on the transition learning rate. Under the extended budget, the dense variant improved all six settings at 5 × 10⁻⁵ and four of six at 2 × 10⁻⁴. At 10⁻³, 8 of 18 runs encountered non-finite values. Among the three settings that completed all three seeds, macro-averaged MSE increased by 0.81% relative to the frozen variant on those same settings. The structured variant completed all runs at both learning rates.

Transition typeTransition learning rateSettings with three completed seedsSettings with lower MSEMacro-averaged MSE change
Dense5 × 10⁻⁵6/66/6−1.60%
Dense2 × 10⁻⁴6/64/6−1.07%
Dense10⁻³3/61/3+0.81%
Structured10⁻⁴6/65/6−0.07%
Structured10⁻³6/62/6−0.35%

For each learning rate, the macro-averaged change compares the trainable variant with the frozen variant on the settings that completed all three seeds.

We updated Section 4.2 accordingly. The full learning-rate sweep and operator diagnostics are reported in Appendix G.3 and Tables 17–18.

AuthorsOpenReview ↗

Response to Reviewer xYFy [4/4]

Q4: Can you add a structured trainable bank (learning only frequencies and half-lives) and a no-dynamics multiscale linear readout as controls?

A4: We added two controls. The structured trainable variant starts from the same modal bank as the frozen variant and learns only the frequencies and half-lives of the damped rotation blocks, preserving the contractivity of the transition operator. The direct-readout variant retains the multiscale history encoder and initial-state construction, and uses a linear head to predict the residuals for the next H steps directly from the initial state at each scale. The table compares the frozen, dense trainable, structured trainable, and parameter-matched direct-readout variants. The trainable parameter count of the parameter-matched variant differs from that of the frozen variant by less than 0.7%. Results for the standard linear head are also reported in Appendix G.3, Table 17.

Compared with the parameter-matched direct readout, the frozen variant achieves lower MSE and MAE for all three paired seeds on ETTm1 (H = 720) and at both forecast horizons on PEMS08. On PEMS08 (H = 96), the structured trainable variant further reduces mean MSE from 0.387 to 0.372 while preserving contractivity. Because the direct readout replaces both latent propagation and the original decoding path, this control compares two complete forecasting paths with nearly equal trainable parameter counts.

Dataset (H)FrozenDense trainable†Structured trainable†Parameter-matched direct readout
ETTm1 (96)0.3240.3210.3230.320
ETTm1 (720)0.4310.4300.4310.449
PEMS08 (12)0.0780.0770.0780.082
PEMS08 (96)0.3870.3690.3720.404

† Learning rates were selected by validation MSE.

Q5: Can you compare empirically against an S4D/LRU-style model with frozen or structured-trainable eigenvalues?

A5: We compared an LRU-style forecaster with a fixed spectrum against one that learns only the frequencies and decay rates. Both variants used the same data splits, forecast horizons, and three seeds, with up to 50 training epochs and an early-stopping patience of 10 (Appendix G.4, Table 20).

Because the two architectures differ in their history encoders, initial-state construction, readouts, and parameter counts, we use comparisons within each architecture to assess the benefit of learning spectral parameters, and comparisons across architectures to assess the complete forecasters.

Dataset (H)Frozen EchoKoopFixed-spectrum LRUTrainable-spectrum LRU†
ETTm1 (96)0.3239 / 0.35910.3568 / 0.38710.3537 / 0.3853
ETTm1 (720)0.4312 / 0.43020.5240 / 0.49540.4812 / 0.4708
Weather (96)0.1803 / 0.22360.1748 / 0.21030.1737 / 0.2105
Weather (720)0.3731 / 0.37230.4010 / 0.37610.3881 / 0.3664
PEMS08 (12)0.0776 / 0.18120.0862 / 0.19240.0843 / 0.1899
PEMS08 (96)0.3872 / 0.39890.4167 / 0.42410.4114 / 0.4192

Each cell reports mean test MSE / MAE over three paired seeds. Bold indicates the lowest value for each metric in that setting. † Trainable-spectrum LRU learns only the frequencies and decay rates, and its transition learning rate is selected by validation MSE.

Q6: Can you include recent strong general forecasters (e.g., TimeMixer, OLinear) on the L=96 benchmarks?

A6: We evaluated TimeMixer and OLinear on the six L = 96 datasets using the same data splits and four forecast horizons as in the original benchmark. Each configuration was run with three seeds, with up to 25 training epochs and an early-stopping patience of 3. In total, we completed 144 runs. Table 10 reports the results for each forecast horizon. The updated main comparison and statistical analysis are in Table 1 and Appendix H, respectively.

Among the three methods shown, EchoKoop achieves the lowest horizon-averaged MSE on ETTm1, with its largest lead at H = 720, while OLinear achieves the lowest horizon-averaged MAE on all six datasets.

ETTm1 forecast horizonEchoKoop MSE / MAETimeMixer MSE / MAEOLinear MSE / MAE
960.324 / 0.3590.3235 / 0.36170.3047 / 0.3369
1920.358 / 0.3830.3668 / 0.38570.3585 / 0.3652
3360.386 / 0.4020.3924 / 0.40510.3924 / 0.3884
7200.431 / 0.4300.4526 / 0.44130.4607 / 0.4272
Average0.375 / 0.3940.3838 / 0.39850.3791 / 0.3794

Each cell reports mean test MSE / MAE over three seeds. Bold indicates the lowest value for each metric among the three methods.

Reviewer_uG4GOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EchoKoop, a time series forecasting model inspired by Koopman Theory. Unlike conventional methods that jointly learn nonlinear representations and transition matrices, EchoKoop employs fixed normal contraction operators characterized by different frequencies and decay half-lives. Forecasting is achieved through a learnable encoder, phase-consistent historical initialization, a mode selector, and a decoder. This design reduces the number of learnable dynamical parameters and mitigates transient error amplification during latent-space recurrence. Across ten long-term forecasting benchmarks, EchoKoop achieves the lowest average MSE across prediction horizons on six datasets. Ablation studies show that the multiscale structure and phase-aligned initialization contribute more substantially to performance, while learning the transition matrix does not yield consistent improvements. Furthermore, incorporating fixed dynamics into other Koopman architectures also leads to varying degrees of improvement.

Critical strengths
  1. The paper provides a theoretical justification for using fixed dynamical operators by establishing a finite-horizon representation result and an error-propagation bound. This offers a useful perspective on the trade-off between the expressiveness of learned dynamics and the stability of latent rollouts.
  2. Ablation study is solid. The proposed framework integrates frequency diversity, decay timescales, and phase-aware initialization into a unified forecasting architecture. The ablation studies provide empirical evidence that these structural choices contribute more consistently to forecasting performance than making the transition operators trainable.
  3. The paper evaluates the proposed approach across a diverse range of tasks, including standard time-series forecasting benchmarks, low-dimensional dynamical systems, and high-dimensional physical fields. This broad evaluation provides a more comprehensive assessment of the method's applicability beyond conventional forecasting settings.
Critical weaknesses
  1. The choice of fixed frequencies and decay half-lives relies on predefined temporal scales, potentially limiting the model's adaptability to highly non-stationary dynamics and abrupt regime shifts.
  2. The theoretical finite-horizon representation result relies on a sufficiently rich modal dictionary satisfying a full-rank condition, which does not guarantee equivalent expressive capacity for the practical model with finite parameters and training data.
  3. The experimental results do not establish statistically significant overall superiority over existing methods, and the benefits of frozen dynamics vary across different architectures.
Questions for authors

N/A

4 comments
AuthorsOpenReview ↗

Response to Reviewer uG4G [1/2]

We sincerely thank Reviewer uG4G for the thoughtful feedback and insightful suggestions. Below, we address how fixed modes adapt to nonstationary dynamics, the scope of the representation theorem, and the statistical evidence for performance gains within and across architectures.

Q1: The choice of fixed frequencies and decay half-lives relies on predefined temporal scales, potentially limiting the model's adaptability to highly non-stationary dynamics and abrupt regime shifts.

A1: EchoKoop reconstructs the initial modal state from the most recent observation window, allowing it to respond to changes already reflected in that window by adjusting the amplitudes and phases of its modes. The projection result in Appendix B.2 shows that, with a single-scale linear readout, a combination of the fixed modes can approximate the post-change target trajectory over a finite horizon when that trajectory lies close to their span. Appendix B.3 further describes how the approximation coefficients depend on the observed history. When the post-change trajectory lies far from the modal span, adjusting the initial state alone offers limited approximation accuracy.

We tested frequency alignment by comparing a grid aligned to ETTm1’s daily cycle with an equally sized half-bin-shifted grid. Both grids used the same half-lives and had the same spectral radius and relative power gain (Appendix F.4).

HostH = 96: MSE changeSeeds improvedH = 720: MSE changeSeeds improved
Koopa+0.21% ± 0.24%1/3−0.24% ± 0.28%3/3
SKOLR+0.34% ± 1.78%1/3−0.90% ± 0.64%3/3
K²VAE+0.36% ± 4.18%2/3−2.30% ± 2.91%2/3
Overall+0.30% ± 2.28%4/9−1.15% ± 1.75%8/9

Values are the mean ± sample standard deviation of paired relative test MSE changes. Negative values favor the daily-cycle-aligned grid.

At H = 720, alignment yielded a negative mean relative MSE change in all three hosts and improved eight of nine host–seed pairs. At H = 96, the mean relative change was slightly positive in all three hosts. This experiment tests sensitivity to frequency alignment on ETTm1. Adaptation to abrupt regime shifts remains an open empirical question.

Q2: The theoretical finite-horizon representation result relies on a sufficiently rich modal dictionary satisfying a full-rank condition, which does not guarantee equivalent expressive capacity for the practical model with finite parameters and training data.

A2: We agree that Theorem 1 does not guarantee the expressive capacity of the practical EchoKoop model. The key distinction is between what the fixed modes can represent and what a finite encoder can infer from the observed history.

Proposition 1 makes this distinction explicit for a linear readout on a single native time grid. Let VV be the modal matrix, yy the target trajectory, c⋆=V†yc^\star = V^\dagger y the least-squares coefficients, and c^(X)\hat c(X) the coefficients predicted from the observed history XX. Then

∥Vc^(X)−y∥22=∥(I−VV†)y∥22⏟error due to the fixed modal span+∥V(c^(X)−c⋆)∥22⏟trajectory error due to coefficient estimation.\lVert V\hat{c}(X)-y\rVert_2^2 = \underbrace{\lVert (I-VV^\dagger)y\rVert_2^2}_{\text{error due to the fixed modal span}} + \underbrace{\lVert V(\hat{c}(X)-c^\star)\rVert_2^2}_{\text{trajectory error due to coefficient estimation}}.

Full row rank makes the first term zero for every target trajectory. The second still depends on how well the encoder estimates the coefficients from finite training data. Conversely, even when the default dictionary falls short of full row rank at long horizons, the first term can remain small for trajectories close to the modal span. This explains when a fixed dictionary can support accurate forecasts in the single-grid setting. We clarified this scope in Appendices B.2–B.3 and assess the full multiscale model through held-out forecasting results and structural ablations.

AuthorsOpenReview ↗

Response to Reviewer uG4G [2/2]

Q3: The experimental results do not establish statistically significant overall superiority over existing methods, …

A3: On the original ten-dataset benchmark, which included the four PEMS datasets, EchoKoop ranked second by both MSE and MAE among the eight methods with complete results. The omnibus test detected significant differences in method rankings**, and none of the post-hoc comparisons between EchoKoop and individual baselines remained significant after Holm correction.

The updated statistical analysis covers only the six non-PEMS datasets with L = 96 and includes TimeMixer and OLinear. Each dataset contributes one error value per method and metric, averaged over forecast horizons and seeds. The small number of dataset-level pairs limits the ability of the paired tests to detect differences after multiple-comparison correction. The two analyses also differ in both the datasets and the methods included, so their rankings describe the respective benchmark settings.

We report the PEMS results separately. EchoKoop achieved the lowest horizon-averaged MSE on all four PEMS datasets under the original L = 12 protocol (Table 1). With a shared input history of L = 192, EchoKoop still achieved lower horizon-averaged MSE and MAE than Koopa and FlowMixer on PEMS04 and PEMS08 (Table 11).

DatasetForecast horizon (H)EchoKoopKoopaFlowMixer
PEMS04120.0945 / 0.20080.0792 / 0.19110.1158 / 0.2292
PEMS04240.1108 / 0.21800.0916 / 0.20570.1751 / 0.2852
PEMS04480.1312 / 0.23890.1184 / 0.23490.2433 / 0.3511
PEMS04960.1525 / 0.26150.3446 / 0.42100.2622 / 0.3658
PEMS04Average0.1222 / 0.22980.1584 / 0.26310.1991 / 0.3078
PEMS08120.0916 / 0.19570.0910 / 0.19710.1131 / 0.2242
PEMS08240.1157 / 0.22030.1205 / 0.22550.1897 / 0.2930
PEMS08480.1590 / 0.25680.1670 / 0.26210.2950 / 0.3720
PEMS08960.2303 / 0.30510.2456 / 0.30150.3759 / 0.4101
PEMS08Average0.1492 / 0.24450.1560 / 0.24650.2434 / 0.3248

Q4: …,and the benefits of frozen dynamics vary across different architectures.

A4: The benefits of frozen dynamics depend on the host architecture. **Compared with fully trainable transition matrices, freezing eliminates their trainable parameters, while the normality and contractivity of the prescribed operators bound amplification during latent rollouts.

The cross-architecture experiments extend Koopman-inspired frozen dynamics, zt+1=Kfixedztz_{t+1}=K_{\mathrm{fixed}}z_t, to different host models. Fixed propagation plays a different role in each host's state updates, which may help explain why the gains vary across architectures and delineate the scope of applicability for frozen dynamics.

If the entire forecasting-stage latent update were replaced by zt+1=Kfixedztz_{t+1}=K_{\mathrm{fixed}}z_t, then, given the same initial state, the subsequent latent trajectory would be fully determined by zt+h=Kfixedhztz_{t+h}=K_{\mathrm{fixed}}^h z_t. The host’s original step-by-step updates would no longer affect propagation at this stage. If retained, the host’s history encoder would still determine the initial state, and its readout would still map the latent trajectory to forecasts.

In Koopa, the fixed operator directly propagates the latent states in all three time-varying blocks. In SKOLR, the observed inputs help construct the initial forecast state, which the fixed operator then propagates autonomously. In K²VAE, the fixed operator changes the pseudo-observations generated by the Koopman branch, while auxiliary control inputs and Kalman corrections still shape the forecast state. The same operator replacement thus affects the forecast through different update paths.

HostRole of the fixed operatorShort-horizon MSE change (Q = 1 / Q = 6)Long-horizon MSE change (Q = 1 / Q = 6)
KoopaDirectly propagates the latent states in all three time-varying blocks−2.87% / −2.90%−6.27% / −6.32%
SKOLRObserved inputs help construct the initial state, which the fixed operator then propagates autonomously−0.18% / +0.38%−0.35% / −0.59%
K²VAEChanges the Koopman pseudo-observations, while forecast state updates still include control inputs and Kalman corrections−4.11% / −2.18%+3.10% / +2.03%

Values show the mean MSE change relative to each original host across six datasets. Negative values indicate lower MSE (Table 2 and Appendix F).

Thanks for yoru feedback, which has addressed most of concerns. I have raised my score.

Thank you for taking the time to consider our responses and reassess the paper. We appreciate your thoughtful feedback, which helped us clarify the scope of our theoretical results and the interpretation of our empirical findings. Thank you again for your time and consideration.

All the best.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We sincerely thank all the reviewers for their insightful reviews and positive perspectives, which are a great encouragement to us. Specifically, the reviewers commented:

  • EchoKoop addresses a clear and worthwhile research question (Reviewer qn2y), and its fixed-transition principle is well motivated, simple, well specified, and directly testable (Reviewer xYFy). Its phase-aligned initializer is a particularly appealing component that connects the observed history to the prescribed dynamics (Reviewer qn2y).
  • The finite-horizon representation result and error-propagation bound offer a useful perspective on the trade-off between the expressiveness of learned dynamics and the stability of latent rollouts (Reviewer uG4G), and the theoretical claims are scoped carefully (Reviewer xYFy).
  • The paper is a careful and interesting study that reflects a broad experimental effort (Reviewer xYFy), with an evaluation spanning a diverse range of tasks (Reviewer uG4G). The supporting controls are thorough, covering operator diagnostics, a learning-rate sweep, structured trainable transitions, a parameter-matched direct readout, an LRU comparison, and cross-host experiments (Reviewer qn2y).

The reviewers also raised constructive concerns, which we addressed through additional analyses, experiments, and clarifications.

1. Scope of the theoretical guarantees [Reviewers: uG4G, xYFy, qn2y]

We clarified the scope of Theorem 1 (in response to concerns raised by Reviewers uG4G and xYFy) and the interpretation of Koopman closure. We derived a projection result for modal dictionaries of arbitrary rank and a conditional error bound for the multiscale readout. For a linear readout on a single native time grid, even a rank-deficient dictionary can approximate target trajectories that lie close to its modal span. In the full model, forecast accuracy also depends on how well the encoder estimates the modal coefficients and how sensitive the multiscale readout is to latent errors (Appendices A.1 and B.2–B.5).

2. Extended Transition Ablations and Diagnostics [Reviewers: xYFy, qn2y]

We allowed for longer training and ran a transition learning-rate sweep with measurements of operator drift from initialization (suggested by Reviewers xYFy and qn2y). We also measured departure from normality and sampled power gain. The gains from learning the transition varied with the learning rate, dataset, and forecast horizon. At the matched learning rate of 5 × 10⁻⁵, the larger training budget yielded a slightly smaller average MSE reduction relative to the frozen variant across the six settings shared by both protocols (Section 4.2, Appendix G.3).

3. Additional Controls and Evaluations [Reviewers: xYFy, uG4G, qn2y]

We introduced three controls (suggested by Reviewer xYFy): (i) structured trainable transitions, (ii) a nearly parameter-matched direct readout, and (iii) fixed- and trainable-spectrum LRU variants (Appendices G.3–G.4). We also clarified which operator was replaced in each host architecture (Appendix F).

Moreover, we benchmarked TimeMixer and OLinear, updated the statistical analysis, and repeated baseline comparisons with shared longer histories (suggested by Reviewer qn2y). Of the six datasets evaluated at L = 96, ETTm1 was the only one on which EchoKoop achieved the lowest horizon-averaged MSE among the compared methods. With a shared input history of L = 192, it also retained lower horizon-averaged MSE and MAE than Koopa and FlowMixer on PEMS04 and PEMS08 (Tables 1 and 10–11, Appendix H).

Finally, we documented the baseline adaptations and Cylinder training failures (Appendices D.3 and J.2). As suggested by Reviewer qn2y, we presented Lorenz–63 trajectory comparisons and valid prediction times in Lyapunov units (Figure 10 and Appendix I).

4. Planned Follow-up Revisions [Reviewer: qn2y]

In the next revision, we will address Reviewer qn2y's follow-up suggestions by refining the paper's positioning, qualifying the empirical claims, and providing the requested experimental details and reference comparisons.


Following the revisions and discussion, Reviewer uG4G indicated that our responses addressed most of their concerns and raised their rating to 3 (Weak acceptance). Reviewer xYFy updated their rating to 4 (Clear acceptance), and we sincerely thank them for reconsidering our work in light of the revisions. Reviewer qn2y also raised their score to 3 (Weak acceptance) and shared the following assessment.

In short, there is an idea here worth sharing with the ML community: in this architecture, a fixed, well-structured transition bank performs nearly as well as learned transitions. I think the paper is acceptable as it is.

We were especially encouraged by this assessment and sincerely thank all three reviewers for their reassessments and constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.