- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes SVD, a two-stage Koopman-based point forecaster.
-
Stage 1. The multivariate series is split into non-overlapping patches, treated as Markov states. Two encoders and are trained with the Hilbert–Schmidt low-rank ("LoRA") objective of Jeong et al. (2025) to span the top- singular subspace of the Koopman operator ( for most datasets). A Koopman matrix is then fit by least squares, and a linear regression decoder maps latent states back to patches.
-
Stage 2. With the encoders frozen, the linear latent rollout serves as a pseudo-observation for a Kalman-filter-style recursion. Its matrices ( and the Koopman matrix) are trained end-to-end with an MSE + KL loss.
Critical strengths
-
Learning the top singular subspace of the Koopman operator with a Hilbert–Schmidt objective gives the latent space a clear meaning: the best rank- approximation of the transition operator. This differs from Koopa/KNF-style latent spaces, which are shaped only by reconstruction or forecast losses. The learned spectra (Fig. 3) also give a simple diagnostic of intrinsic rank (about 10–30 for most datasets). This matters because it is the paper's main conceptual difference from prior Koopman forecasters.
-
With a 16-dimensional latent space, SVD reports the best NRMSE on Electricity-L, Ordered-MNIST and Exchange-L (Table 1) and the lowest memory on all high-dimensional datasets (Table 2). If the measurements hold up (see Weakness), this is practically useful.
Critical weaknesses
- The Kalman step is not real filtering, and Table 4 does not show that it helps. During the forecast there are no real observations: the "observation" given to the filter is just the plain linear forecast from the same starting state. Since everything the filter uses comes from that one state and its matrices are fixed after training, the whole module is just a learned linear map of that state. With the paper's initialization, it even returns exactly the plain linear forecast. So the large gain in Table 4 (Electricity-L: 3.09 to 0.85) most likely comes from training these matrices end-to-end on the forecast loss, not from Kalman filtering itself.
- The paper's core idea, the Koopman SVD representation, is not shown to be what makes the method accurate. On its own with the plain linear forecast (the Linear row in Table 4), it is worse than every baseline on Electricity-L and worse than most on Weather-L. This suggests the learned features do not really follow the linear dynamics the method assumes. There is also no test that replaces the representation with a simple basis such as PCA inside the full model, so its real contribution is unknown.
- The timing results do not seem to measure the models themselves. The paper states the model needs under 1 GFLOP on a GPU with over 100 TFLOPS, which should take well under a millisecond, yet it reports about 10 s per batch. FITS, a tiny model with about 10k parameters, takes 31 s, and going from rank 16 to 64 does not change the time at all. So the timings are probably dominated by other overhead (e.g., data loading, sampling in the probabilistic baselines, or different code), and they cannot support the claim that the speed comes from the low-rank design.
- The memory numbers are hard to interpret. FITS (about 10k parameters) and iTransformer use almost the same memory (about 1.6 GB) on Electricity-L, which points to shared overhead rather than model size.
- The same model gets different scores in different tables: MNIST 1.7844 vs. 1.8125 and Weather-L 0.3054 vs. 0.3124 (Tables 1 and 4), and Exchange-L 0.0419 vs. 0.0441 (Tables 1 and 3). It is unclear which numbers are the final ones.
- Strong baselines look unusually weak on Electricity-L (NRMSE 1.30 for iTransformer and 1.34 for FITS), which makes me unsure whether they were set up and tuned properly.
Questions for authors
- How exactly were the speed and memory numbers measured? For example, does the reported time include data loading, or drawing many samples in the probabilistic baselines (TSDiff, GRU NVP, VAE)? Were all methods run in the same code base? Is memory measured during training (as the Fig. 1 caption says) or during inference? Please also report the model-only forward time and the parameter count of every method.
- The target sequence defined in Sec. 3 and Sec. 4.1.1 includes the context window. Is NRMSE computed only on the forecast horizon, in the same way as for the baselines?
- Why does the same model get different scores in Tables 1, 3 and 4? Which numbers are final, and how many random seeds were used?
- How were the baselines set up and tuned (normalization, hyperparameter search, context length)? Adding simple references such as seasonal naive and DLinear, plus PatchTST (which also uses patches), would help check whether the high NRMSE values on Electricity-L are reasonable.
- Eq. (11) suggests the forecast uses only the last patch of the context. Is that right? If so, how is the patch size chosen relative to seasonal cycles (e.g., daily and weekly patterns in Electricity), and how sensitive are the results to it? This also affects the speed comparison, since the baselines read the whole context.
- Please add a version without the Kalman step in which the Koopman matrix is still trained end-to-end on the forecast loss. This would show whether the gain in Table 4 comes from the Kalman step or simply from the training. It would also help to report how much the trained matrices differ from the least-squares estimate (e.g., their eigenvalues).
- Please add a version that keeps everything else the same but replaces the Koopman SVD features with a simple basis (e.g., PCA, a frozen random encoder, or an autoencoder). This would show whether the paper's core representation is what makes the method accurate.
- I think the following papers are also quite important to cite:
- Xu et al. A Data-Driven Framework for Koopman Semigroup Estimation in Stochastic Dynamical Systems. Chaos, 2025.
- Ryu et al. Operator SVD with Neural Networks via Nested Low-Rank Approximation. ICML, 2024.
- Netto and Mili. A Robust Data-Driven Koopman Kalman Filter for Power Systems Dynamic State Estimation. IEEE Transactions on Power Systems, 2018.
- Klus et al. Data-Driven Approximation of the Koopman Generator: Model Reduction, System Identification, and Control. Physica D, 2020.
- Kostic et al. Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces. NeurIPS, 2022.
- Kostic et al. Sharp Spectral Rates for Koopman Operator Learning. NeurIPS, 2023.
- Kostic et al. Consistent Long-Term Forecasting of Ergodic Dynamical Systems. ICML, 2024.
- Kostic et al. Neural Conditional Probability for Uncertainty Quantification. NeurIPS, 2024.
9 comments
2. Contribution of the Koopman SVD representation (Weakness 2; Q7).
We want to clarify the effect of Koopman SVD representation and the relation of plain linear with trainable linear matrix. For a finite-dimensional approximation, subspace closure error and bias within the space both contribute to long horizon rollout error. Stage 1 learns a low-rank representation by minimizing the Hilbert–Schmidt objective, which is proven to upper bound the subspace error and returns a close-to-optimal representation space, determined by and . Stage 2 freezes the encoder functions in the representation space. Optimizing the transition coefficients changes the dynamics within the fixed learned basis, not the basis itself. So the shown bad plain linear forecast should be reasoned as a bad approximation of coefficients rather than representation. We demonstrate this through the ablations below.
(1) Forecast-trained encoders versus proposed two-stage Koopman representation.
- Koopman SVD: Train using the LoRA objective in Stage 1, then freeze them during Stage 2.
- End-to-end encoders: Omit Stage 1 and jointly train and the forecasting components using the Stage-2 objective.
Context/prediction lengths are 96/96 for the time-series datasets and 48/48 for MNIST. Entries report NRMSE with uncertainty terms; lower is better. Although the long-term prediction experiment appears in Appendix D.4.2, we also report a short-term prediction comparison as a supplement.
| Representation / training scheme | Electricity-L | MNIST | Weather-L | Exchange-L | ETTh1 | ETTm1 |
|---|---|---|---|---|---|---|
| End-to-end; trainable on forecasting loss | 0.9175 ± 0.0345 | 1.7932 ± 0.0014 | 0.3085 ± 0.0067 | 0.0422 ± 0.0019 | 0.6637 ± 0.0118 | 0.5764 ± 0.0092 |
| Koopman SVD; LoRA-trained frozen in Stage 2 | 0.8502 ± 0.0014 | 1.7844 ± 0.0007 | 0.3054 ± 0.0062 | 0.0419 ± 0.0004 | 0.6124 ± 0.0091 | 0.5555 ± 0.0010 |
(2) Replacing Koopman SVD with PCA. As you suggested, we replace the learned Koopman representation with a frozen PCA representation while retaining the Kalman-style correction and trainable forecasting components, including the decoder. PCA is fitted using training-context data only, and the LoRA stage is omitted.
| Representation | Electricity-L | MNIST | Weather-L | Exchange-L | ETTh1 | ETTm1 |
|---|---|---|---|---|---|---|
| PCA + Kalman | 0.9300 ± 0.0167 | 1.8584 ± 0.0004 | 0.3218 ± 0.0012 | 0.0371 ± 0.0001 | 0.6350 ± 0.0020 | 0.5602 ± 0.0026 |
| Koopman SVD | 0.8502 ± 0.0014 | 1.7844 ± 0.0007 | 0.3054 ± 0.0062 | 0.0419 ± 0.0004 | 0.6124 ± 0.0091 | 0.5555 ± 0.0010 |
The Koopman-SVD representation outperforms PCA on five of the six datasets, with particularly noticeable improvements on Electricity-L, MNIST, and Weather-L, while PCA performs better on Exchange-L. Together with the end-to-end encoder ablation, these results provide empirical evidence that the operator-based representation contributes to forecasting accuracy beyond a generic PCA representation or features learned solely through forecasting loss. With the above ablation, we show the necessity of both modules in the proposed method.
3. Model-only latency, sampling, memory (Weaknesses 3–4; Q1).
We confirm that, in the original draft, evaluation-pipeline timing was measured over the entire end-to-end pipeline and did not isolate the latency of a single forward pass.
In the original benchmark, we measured wall-clock time per evaluation batch, with a configured batch size of 32. In the same codebase, we added the same 1-layer MLP variance decoder head for all point-wise models, including ours, for a fair comparison. The interval included normalization, forecasting, output sampling, inverse scaling, and metric calculation, which constitute a full forward pass. Data fetching before the batch-start callback and epoch-end aggregation was excluded. Validation timings are collected during training with the model in evaluation mode. The analysis averaged the last ten logged validation batches per run, then averaged across runs to reduce variability.
Although our original setting reflects the complete inference process in practical deployment, we appreciate your suggestion and have added a dedicated benchmark using a common timing framework for all model implementations within the same codebase:
- Hardware: CPU: AMD EPYC 9354 32-Core Processor; GPU: NVIDIA RTX PRO 6000 Blackwell. All models are measured on the same machine.
- Batch and lengths: batch size 32; context/prediction lengths 96/96 for Electricity-L, Weather-L, and Exchange-L, and 48/48 for MNIST.
- Execution: freshly instantiated models without training, in evaluation mode with
torch.inference_mode(), using the same resident batch for repeated passes. - Warm-up and repetition: five warm-up passes followed by twenty measured passes; report median latency. These repetitions are timing measurements, not independent training seeds.
- Timing: wall-clock measurement with CUDA synchronization immediately before and after each GPU prediction call.
- Included: the complete configured prediction path, including active encoders, forecasting modules, and decoder heads.
- Excluded: model construction, data loading, device transfer, external normalization, inverse scaling, and metric evaluation.
- Sampling: no terminal ensemble sampling for models directly producing predictions or distribution parameters. For models requiring iterative generation, one complete generated trajectory is included; native latent stochastic computation is retained.
- Memory: peak CUDA allocated memory during inference, resetting peak statistics before each measured pass. We also record its increase over the pre-pass allocation. This is not training memory, reserved allocator memory, or total GPU process memory.
The corresponding CPU/GPU median latencies, in milliseconds, and GPU peak usage are:
| Model | Exchange-L | Electricity-L | Weather-L | MNIST | MNIST without CNN |
|---|---|---|---|---|---|
| K²VAE | 102.064 / 53.855 (60.138) | 74.829 / 53.631 (139.687) | 100.733 / 54.169 (62.378) | 75.242 / 38.038 (187.434) | 45.546 / 37.425 (138.184) |
| FITS | 5.444 / 6.919 (10.350) | 5.343 / 11.211 (115.262) | 4.218 / 9.218 (13.671) | 24.525 / 11.741 (138.958) | 12.900 / 7.005 (134.094) |
| iTransformer | 5.302 / 4.855 (15.417) | 11.634 / 4.858 (158.850) | 7.794 / 4.896 (20.348) | 24.613 / 9.501 (139.090) | 6.851 / 4.643 (110.932) |
| Koopa | 36.186 / 30.329 (20.299) | 50.144 / 30.667 (135.010) | 47.483 / 30.380 (22.944) | 47.179 / 31.938 (145.408) | 45.671 / 27.959 (187.270) |
| TSDiff | 3192.486/ 2130.142 (36.452) | 3759.265/ 2143.977 (169.344) | 2714.395/ 2130.740 (40.074) | 3816.160/ 2151.952 (186.107) | - |
| K²SVD | 1.943 / 12.099 (11.280) | 4.715 / 12.094 (94.184) | 2.357 / 7.397 (13.653) | 25.794 / 7.318 (139.122) | 4.394 / 1.022 (79.418) |
GRU_NVP is not competitive in efficiency and TimeBase needs additional FFT to determine the period of data otherwise the accuracy collapse so not reported in the single-pass results. The revised results support a hardware- and dataset-dependent efficiency claim. Among the five competitive models in the supplied timing comparison, K²SVD has the lowest CPU median latency on Electricity-L, Weather-L, and Exchange-L. On the MNIST dataset, a common encoder makes the CPU inference times similar, but K²SVD achieves the lowest GPU median latency. After isolating the CNN from the rest of the forward pass, we achieve the best results in all inference time and usage metrics. It has the lowest peak GPU allocation on Electricity-L and Weather-L. We therefore supplement the efficiency claim with these explicit measurements and comparisons.
We thank reviewer BJbq for the detailed review and for recognizing the potential value of the low-rank representation. Your comments identify three issues that need separate evidence: the contribution of the representation, the contribution of the correction module, and the interpretation of efficiency measurements. We group the related questions below.
1. Kalman interpretation and the trainable-linear control (Weakness 1; Q6).
We agree that the forecast-time pseudo-observations contain no newly observed future information. They are generated from the available context. We therefore describe the component as a Kalman-style latent correction, rather than claiming classical filtering.
We added a trainable-linear ablation in which the Koopman encoders are trained in Stage 1 and frozen in Stage 2, while the transition matrix and linear decoder are jointly optimized using forecasting MSE. The decoder uses the same parameterization and initialization scheme as the full model. This ablation removes the Kalman-style correction and covariance recursion while retaining downstream transition training, allowing us to isolate the additional benefit of the correction module over a trainable linear rollout.
The results show that a trainable linear rollout alone does not recover the performance of the full model. K²SVD achieves lower mean NRMSE across all four datasets, with relative reductions of 8.7%, 0.5%, 5.7%, and 2.6%, respectively. The largest gain occurs on Electricity-L, while the smaller gain on MNIST is consistent with its more regular dynamics, which can already be modeled effectively by linear Koopman evolution. These results support the additional empirical value of the Kalman correction beyond a trainable linear rollout.