acceptodds
Under review as a conference paper at ICLR 2027

Principled Koopman Representations with Kalman Inference for Efficient Time-Series Prediction

Abstract

The Koopman operator has been widely used for time-series prediction in dynamical systems. However, prior work that learns latent “Koopman spaces” using neural networks often did not construct a valid Koopman space for forecasting, as these representations may be mathematically inconsistent with the operator-theoretic formulation and fail to capture the intrinsic low-rank structure of system dynamics. To address this issue, we introduce KSVD, a method that explicitly learns the leading singular functions of the Koopman operator by optimizing a Hilbert-Schmidt objective. This yields a well-defined low-rank approximation of the Koopman operator with an interpretable linear combination, featuring a compact latent space with less than of the dimensions used in previous work. In the learned Koopman space, KSVD further captures temporal evolution with a linear Gaussian state-space model and performs inference via Kalman filtering, mitigating noise accumulation during multi-step prediction. Empirical results show that KSVD outperforms state-of-the-art methods across multiple datasets, with significantly faster prediction speeds and lower computational cost than previous efficiency-focused models. This highlights the benefits of principled low-rank Koopman representations and opens up broader potential for applications.

Primary area:
learning on time series and dynamical systems
Keywords:
Koopman theory, Time series prediction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BJbqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes K2\mathrm{K}^2SVD, a two-stage Koopman-based point forecaster.

  • Stage 1. The multivariate series is split into non-overlapping patches, treated as Markov states. Two encoders f\mathbf{f} and g\mathbf{g} are trained with the Hilbert–Schmidt low-rank ("LoRA") objective of Jeong et al. (2025) to span the top-dd singular subspace of the Koopman operator (d=16d=16 for most datasets). A d×dd \times d Koopman matrix is then fit by least squares, and a linear regression decoder maps latent states back to patches.

  • Stage 2. With the encoders frozen, the linear latent rollout serves as a pseudo-observation for a Kalman-filter-style recursion. Its matrices (A,H,Q,RA, H, Q, R and the Koopman matrix) are trained end-to-end with an MSE + KL loss.

Critical strengths
  • Learning the top singular subspace of the Koopman operator with a Hilbert–Schmidt objective gives the latent space a clear meaning: the best rank-dd approximation of the transition operator. This differs from Koopa/KNF-style latent spaces, which are shaped only by reconstruction or forecast losses. The learned spectra (Fig. 3) also give a simple diagnostic of intrinsic rank (about 10–30 for most datasets). This matters because it is the paper's main conceptual difference from prior Koopman forecasters.

  • With a 16-dimensional latent space, K2\mathrm{K}^2SVD reports the best NRMSE on Electricity-L, Ordered-MNIST and Exchange-L (Table 1) and the lowest memory on all high-dimensional datasets (Table 2). If the measurements hold up (see Weakness), this is practically useful.

Critical weaknesses
  • The Kalman step is not real filtering, and Table 4 does not show that it helps. During the forecast there are no real observations: the "observation" given to the filter is just the plain linear forecast from the same starting state. Since everything the filter uses comes from that one state and its matrices are fixed after training, the whole module is just a learned linear map of that state. With the paper's initialization, it even returns exactly the plain linear forecast. So the large gain in Table 4 (Electricity-L: 3.09 to 0.85) most likely comes from training these matrices end-to-end on the forecast loss, not from Kalman filtering itself.
  • The paper's core idea, the Koopman SVD representation, is not shown to be what makes the method accurate. On its own with the plain linear forecast (the Linear row in Table 4), it is worse than every baseline on Electricity-L and worse than most on Weather-L. This suggests the learned features do not really follow the linear dynamics the method assumes. There is also no test that replaces the representation with a simple basis such as PCA inside the full model, so its real contribution is unknown.
  • The timing results do not seem to measure the models themselves. The paper states the model needs under 1 GFLOP on a GPU with over 100 TFLOPS, which should take well under a millisecond, yet it reports about 10 s per batch. FITS, a tiny model with about 10k parameters, takes 31 s, and going from rank 16 to 64 does not change the time at all. So the timings are probably dominated by other overhead (e.g., data loading, sampling in the probabilistic baselines, or different code), and they cannot support the claim that the speed comes from the low-rank design.
  • The memory numbers are hard to interpret. FITS (about 10k parameters) and iTransformer use almost the same memory (about 1.6 GB) on Electricity-L, which points to shared overhead rather than model size.
  • The same model gets different scores in different tables: MNIST 1.7844 vs. 1.8125 and Weather-L 0.3054 vs. 0.3124 (Tables 1 and 4), and Exchange-L 0.0419 vs. 0.0441 (Tables 1 and 3). It is unclear which numbers are the final ones.
  • Strong baselines look unusually weak on Electricity-L (NRMSE 1.30 for iTransformer and 1.34 for FITS), which makes me unsure whether they were set up and tuned properly.
Questions for authors
  • How exactly were the speed and memory numbers measured? For example, does the reported time include data loading, or drawing many samples in the probabilistic baselines (TSDiff, GRU NVP, K2\mathrm{K}^2VAE)? Were all methods run in the same code base? Is memory measured during training (as the Fig. 1 caption says) or during inference? Please also report the model-only forward time and the parameter count of every method.
  • The target sequence defined in Sec. 3 and Sec. 4.1.1 includes the context window. Is NRMSE computed only on the forecast horizon, in the same way as for the baselines?
  • Why does the same model get different scores in Tables 1, 3 and 4? Which numbers are final, and how many random seeds were used?
  • How were the baselines set up and tuned (normalization, hyperparameter search, context length)? Adding simple references such as seasonal naive and DLinear, plus PatchTST (which also uses patches), would help check whether the high NRMSE values on Electricity-L are reasonable.
  • Eq. (11) suggests the forecast uses only the last patch of the context. Is that right? If so, how is the patch size chosen relative to seasonal cycles (e.g., daily and weekly patterns in Electricity), and how sensitive are the results to it? This also affects the speed comparison, since the baselines read the whole context.
  • Please add a version without the Kalman step in which the Koopman matrix is still trained end-to-end on the forecast loss. This would show whether the gain in Table 4 comes from the Kalman step or simply from the training. It would also help to report how much the trained matrices differ from the least-squares estimate (e.g., their eigenvalues).
  • Please add a version that keeps everything else the same but replaces the Koopman SVD features with a simple basis (e.g., PCA, a frozen random encoder, or an autoencoder). This would show whether the paper's core representation is what makes the method accurate.
  • I think the following papers are also quite important to cite:
    • Xu et al. A Data-Driven Framework for Koopman Semigroup Estimation in Stochastic Dynamical Systems. Chaos, 2025.
    • Ryu et al. Operator SVD with Neural Networks via Nested Low-Rank Approximation. ICML, 2024.
    • Netto and Mili. A Robust Data-Driven Koopman Kalman Filter for Power Systems Dynamic State Estimation. IEEE Transactions on Power Systems, 2018.
    • Klus et al. Data-Driven Approximation of the Koopman Generator: Model Reduction, System Identification, and Control. Physica D, 2020.
    • Kostic et al. Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces. NeurIPS, 2022.
    • Kostic et al. Sharp Spectral Rates for Koopman Operator Learning. NeurIPS, 2023.
    • Kostic et al. Consistent Long-Term Forecasting of Ergodic Dynamical Systems. ICML, 2024.
    • Kostic et al. Neural Conditional Probability for Uncertainty Quantification. NeurIPS, 2024.
9 comments

We thank reviewer BJbq for the detailed review and for recognizing the potential value of the low-rank representation. Your comments identify three issues that need separate evidence: the contribution of the representation, the contribution of the correction module, and the interpretation of efficiency measurements. We group the related questions below.

1. Kalman interpretation and the trainable-linear control (Weakness 1; Q6).

We agree that the forecast-time pseudo-observations contain no newly observed future information. They are generated from the available context. We therefore describe the component as a Kalman-style latent correction, rather than claiming classical filtering.

We added a trainable-linear ablation in which the Koopman encoders are trained in Stage 1 and frozen in Stage 2, while the transition matrix and linear decoder are jointly optimized using forecasting MSE. The decoder uses the same parameterization and initialization scheme as the full model. This ablation removes the Kalman-style correction and covariance recursion while retaining downstream transition training, allowing us to isolate the additional benefit of the correction module over a trainable linear rollout.

Forecasting moduleElectricity-LMNISTWeather-LExchange-L
Trainable linear rollout + trainable decoder0.9310 ± 0.01631.7933 ± 0.00030.3239 ± 0.00090.0430 ± 0.0001
Full Kalman-style correction + trainable decoder0.8502 ± 0.00141.7844 ± 0.00070.3054 ± 0.00620.0419 ± 0.0004

The results show that a trainable linear rollout alone does not recover the performance of the full model. K²SVD achieves lower mean NRMSE across all four datasets, with relative reductions of 8.7%, 0.5%, 5.7%, and 2.6%, respectively. The largest gain occurs on Electricity-L, while the smaller gain on MNIST is consistent with its more regular dynamics, which can already be modeled effectively by linear Koopman evolution. These results support the additional empirical value of the Kalman correction beyond a trainable linear rollout.

2. Contribution of the Koopman SVD representation (Weakness 2; Q7).

We want to clarify the effect of Koopman SVD representation and the relation of plain linear with trainable linear matrix. For a finite-dimensional approximation, subspace closure error and bias within the space both contribute to long horizon rollout error. Stage 1 learns a low-rank representation by minimizing the Hilbert–Schmidt objective, which is proven to upper bound the subspace error and returns a close-to-optimal representation space, determined by ff and gg. Stage 2 freezes the encoder functions in the representation space. Optimizing the transition coefficients changes the dynamics within the fixed learned basis, not the basis itself. So the shown bad plain linear forecast should be reasoned as a bad approximation of coefficients rather than representation. We demonstrate this through the ablations below.

(1) Forecast-trained encoders versus proposed two-stage Koopman representation.

  • Koopman SVD: Train f,gf,g using the LoRA objective in Stage 1, then freeze them during Stage 2.
  • End-to-end encoders: Omit Stage 1 and jointly train f,gf,g and the forecasting components using the Stage-2 objective.

Context/prediction lengths are 96/96 for the time-series datasets and 48/48 for MNIST. Entries report NRMSE with uncertainty terms; lower is better. Although the long-term prediction experiment appears in Appendix D.4.2, we also report a short-term prediction comparison as a supplement.

Representation / training schemeElectricity-LMNISTWeather-LExchange-LETTh1ETTm1
End-to-end; f,gf,g trainable on forecasting loss0.9175 ± 0.03451.7932 ± 0.00140.3085 ± 0.00670.0422 ± 0.00190.6637 ± 0.01180.5764 ± 0.0092
Koopman SVD; LoRA-trained f,gf,g frozen in Stage 20.8502 ± 0.00141.7844 ± 0.00070.3054 ± 0.00620.0419 ± 0.00040.6124 ± 0.00910.5555 ± 0.0010

(2) Replacing Koopman SVD with PCA. As you suggested, we replace the learned Koopman representation with a frozen PCA representation while retaining the Kalman-style correction and trainable forecasting components, including the decoder. PCA is fitted using training-context data only, and the LoRA stage is omitted.

RepresentationElectricity-LMNISTWeather-LExchange-LETTh1ETTm1
PCA + Kalman0.9300 ± 0.01671.8584 ± 0.00040.3218 ± 0.00120.0371 ± 0.00010.6350 ± 0.00200.5602 ± 0.0026
Koopman SVD0.8502 ± 0.00141.7844 ± 0.00070.3054 ± 0.00620.0419 ± 0.00040.6124 ± 0.00910.5555 ± 0.0010

The Koopman-SVD representation outperforms PCA on five of the six datasets, with particularly noticeable improvements on Electricity-L, MNIST, and Weather-L, while PCA performs better on Exchange-L. Together with the end-to-end encoder ablation, these results provide empirical evidence that the operator-based representation contributes to forecasting accuracy beyond a generic PCA representation or features learned solely through forecasting loss. With the above ablation, we show the necessity of both modules in the proposed method.

3. Model-only latency, sampling, memory (Weaknesses 3–4; Q1).

We confirm that, in the original draft, evaluation-pipeline timing was measured over the entire end-to-end pipeline and did not isolate the latency of a single forward pass.

In the original benchmark, we measured wall-clock time per evaluation batch, with a configured batch size of 32. In the same codebase, we added the same 1-layer MLP variance decoder head for all point-wise models, including ours, for a fair comparison. The interval included normalization, forecasting, output sampling, inverse scaling, and metric calculation, which constitute a full forward pass. Data fetching before the batch-start callback and epoch-end aggregation was excluded. Validation timings are collected during training with the model in evaluation mode. The analysis averaged the last ten logged validation batches per run, then averaged across runs to reduce variability.

Although our original setting reflects the complete inference process in practical deployment, we appreciate your suggestion and have added a dedicated benchmark using a common timing framework for all model implementations within the same codebase:

  • Hardware: CPU: AMD EPYC 9354 32-Core Processor; GPU: NVIDIA RTX PRO 6000 Blackwell. All models are measured on the same machine.
  • Batch and lengths: batch size 32; context/prediction lengths 96/96 for Electricity-L, Weather-L, and Exchange-L, and 48/48 for MNIST.
  • Execution: freshly instantiated models without training, in evaluation mode with torch.inference_mode(), using the same resident batch for repeated passes.
  • Warm-up and repetition: five warm-up passes followed by twenty measured passes; report median latency. These repetitions are timing measurements, not independent training seeds.
  • Timing: wall-clock measurement with CUDA synchronization immediately before and after each GPU prediction call.
  • Included: the complete configured prediction path, including active encoders, forecasting modules, and decoder heads.
  • Excluded: model construction, data loading, device transfer, external normalization, inverse scaling, and metric evaluation.
  • Sampling: no terminal ensemble sampling for models directly producing predictions or distribution parameters. For models requiring iterative generation, one complete generated trajectory is included; native latent stochastic computation is retained.
  • Memory: peak CUDA allocated memory during inference, resetting peak statistics before each measured pass. We also record its increase over the pre-pass allocation. This is not training memory, reserved allocator memory, or total GPU process memory.

The corresponding CPU/GPU median latencies, in milliseconds, and GPU peak usage are:

ModelExchange-LElectricity-LWeather-LMNISTMNIST without CNN
K²VAE102.064 / 53.855 (60.138)74.829 / 53.631 (139.687)100.733 / 54.169 (62.378)75.242 / 38.038 (187.434)45.546 / 37.425 (138.184)
FITS5.444 / 6.919 (10.350)5.343 / 11.211 (115.262)4.218 / 9.218 (13.671)24.525 / 11.741 (138.958)12.900 / 7.005 (134.094)
iTransformer5.302 / 4.855 (15.417)11.634 / 4.858 (158.850)7.794 / 4.896 (20.348)24.613 / 9.501 (139.090)6.851 / 4.643 (110.932)
Koopa36.186 / 30.329 (20.299)50.144 / 30.667 (135.010)47.483 / 30.380 (22.944)47.179 / 31.938 (145.408)45.671 / 27.959 (187.270)
TSDiff3192.486/ 2130.142 (36.452)3759.265/ 2143.977 (169.344)2714.395/ 2130.740 (40.074)3816.160/ 2151.952 (186.107)-
K²SVD1.943 / 12.099 (11.280)4.715 / 12.094 (94.184)2.357 / 7.397 (13.653)25.794 / 7.318 (139.122)4.394 / 1.022 (79.418)

GRU_NVP is not competitive in efficiency and TimeBase needs additional FFT to determine the period of data otherwise the accuracy collapse so not reported in the single-pass results. The revised results support a hardware- and dataset-dependent efficiency claim. Among the five competitive models in the supplied timing comparison, K²SVD has the lowest CPU median latency on Electricity-L, Weather-L, and Exchange-L. On the MNIST dataset, a common encoder makes the CPU inference times similar, but K²SVD achieves the lowest GPU median latency. After isolating the CNN from the rest of the forward pass, we achieve the best results in all inference time and usage metrics. It has the lowest peak GPU allocation on Electricity-L and Weather-L. We therefore supplement the efficiency claim with these explicit measurements and comparisons.

Reviewer_ZLEDOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes K$^2$SVD, a time-series forecaster whose encoders are trained so that their span approximates the leading singular functions of the Koopman operator. It forecasts by rolling the last context patch forward with a Koopman matrix and correcting the rollout with a latent Kalman filter, whose matrices are trained on the forecasting loss. It reports competitive NRMSE on ProbTS datasets and Ordered-MNIST with less inference time and memory than the baselines.

Critical strengths

The paper claims that an operator-grounded latent space with Kalman inference forecasts accurately at lower inference cost, a claim that would matter if it held. For the reasons in Critical Weaknesses, I cannot count it as a separable contribution.

Critical weaknesses

The reported gains cannot be attributed to either component named in the title: the forecast is not produced by the operator the encoders learn, and the Kalman filter receives no data beyond the last context patch.

1. The operator the encoders learn does not produce the forecast. The encoders are trained with the Hilbert-Schmidt objective, but the forecast uses a trainable copy of the Koopman matrix, trained with the filter end to end on the forecasting loss. In the paper's own ablation, keeping that matrix at its regression estimate, the matrix the theory identifies with the Koopman operator, makes the forecasts worse on every dataset tested. The accuracy therefore comes from training the matrix away from the operator on forecast error, not from the operator the theory describes.

2. The Kalman filter is used without a measurement. A Kalman filter corrects a prediction with a measurement of the system, but here the only observation at each forecast step is the model's own rollout of the last context patch, because "we have no access to the actual future states". With all parameters fixed at inference, the filter's output is therefore a fixed function of that patch, a learned correction rather than inference from data. The only ablation compares the trained filter with the regression rollout, so it cannot separate the filter from the training of its matrices.

Questions for authors
  1. What does the filter add over a linear rollout whose step matrices are trained on the same loss? Without that comparison, the gains of the filtered model cannot be attributed to the Kalman structure.
  2. Which of the conflicting values in Tables 1, 4, 9 and 12 are correct, and how is the upper end of the stated 5.5 times speed-up obtained? The answer determines K$^2$SVD's rank on Weather-L and the size of its efficiency gain.
5 comments

We thank reviewer ZLED for the careful review. If we understand correctly, the central concern is whether the Koopman representation and Kalman correction module each contribute beyond end-to-end optimization. We address this with clarification in the operator approximation and ablations containing a trainable-linear baseline.

Weakness 1: Training the transition matrix diverges from the operator the encoders learn

We respectfully disagree that the forecasting performance is unrelated to the learned Koopman operator. Our two-stage framework separates two roles: Stage 1 learns a low-rank representation space from the Koopman operator, while Stage 2 adapts the transition matrix within this fixed space to optimize forecasting performance. Thus, although the transition matrix is updated during Stage 2, the forecast remains constrained to the operator-informed representation learned in Stage 1.

Let K\mathcal{K} denote the population Koopman operator, Kd\mathcal{K}_d its best approximation within the fixed encoder-defined input and output spaces, and K^θ\widehat{\mathcal{K}}_\theta the operator represented by learned coefficients in those spaces. Under the Hilbert–Schmidt formulation, orthogonal projection gives

= \underbrace{\\|\mathcal{K}-\mathcal{K}_d\\|_{\mathrm{HS}}^2}_{\text{space approximation error}} + \underbrace{\\|\mathcal{K}_d-\widehat{\mathcal{K}}_\theta\\|_{\mathrm{HS}}^2}_{\text{coefficient approximation error within the spaces}}.$$ **Stage 1 addresses the space approximation term.** The Hilbert–Schmidt low-rank objective trains $f$ and $g$ to span spaces that capture the dominant action of the Koopman operator. Minimizing the LoRA objective trains the model towards the best rank-constrained operator approximation by constraining a subspace closure loss, as established by [Kostić et al.](https://arxiv.org/abs/2307.09912) and [Jeong et al. (2025)](https://arxiv.org/html/2507.07222v1). In practice, Stage 1 training makes the representation space fit the underlying dynamics with finite rank and data. **Stage 2 adapts the coefficients while preserving those spaces.** Both encoders are frozen, so optimizing $K_\theta$ leaves their spans—and therefore the space approximation term—unchanged. The regression estimate is not the population operator itself, but is an approximation of the coefficients within the trained space in Stage 1, as we freeze $f$ and $g$. Compared to regression, it broadly fits the transition of the whole dataset with regard to the Kalman correction model rather than considering pairwise relationships. Our claim is therefore that downstream training **preserves the minimized-error Koopman representation space while adapting its transition coefficients for forecasting**. To examine the contribution of the Koopman space and the Kalman correction, we report two ablation experiments. All entries below are NRMSE; lower is better. **Ablation on the quality of the Koopman SVD representation.** - **Koopman SVD:** Train $f,g$ using the LoRA objective in Stage 1, then freeze them during Stage 2. - **End-to-end encoders:** Omit Stage 1 and jointly train $f,g$ and the forecasting components using the Stage-2 objective. All results below concern short-term prediction: context/prediction lengths are 96/96 for the time-series datasets and 48/48 for MNIST. Entries report NRMSE with uncertainty terms; lower is better. Although we have already shown the long-term prediction test in Appendix D.4.2, we further report the short-term results as a supplement. | Representation / training scheme | Electricity-L | MNIST | Weather-L | Exchange-L | ETTh1 | ETTm1 | |:--|--:|--:|--:|--:|--:|--:| | End-to-end; $f,g$ trainable on forecasting loss | 0.9175 ± 0.0345 | 1.7932 ± 0.0014 | 0.3085 ± 0.0067 | 0.0422 ± 0.0019 | 0.6637 ± 0.0118 | 0.5764 ± 0.0092 | | Koopman SVD; LoRA-trained $f,g$ frozen in Stage 2 | **0.8502 ± 0.0014** | **1.7844 ± 0.0007** | **0.3054 ± 0.0062** | **0.0419 ± 0.0004** | **0.6124 ± 0.0091** | **0.5555 ± 0.0010** | These comparisons test whether the operator-based LoRA objective provides a useful forecasting space beyond a generic variance-preserving projection or features learned solely through forecasting loss. The available results favor Koopman SVD over end-to-end on all six datasets. They provide empirical evidence for the quality of the learned SVD representation.

Weakness 2 and Q1: What does the correction add beyond a trainable linear rollout?

We agree that the forecast-time pseudo-observations contain no newly observed future information. They are generated from the available context. We therefore describe the component as a Kalman-style latent correction, rather than claiming classical filtering.

We added a trainable-linear ablation: the Koopman encoders are trained in Stage 1 and frozen in Stage 2, while a repeatedly applied transition matrix and a linear decoder are both optimized on forecasting MSE. The decoder uses the same parameterization and initialization scheme as the dynamic full model. This ablation removes the Kalman correction and covariance recursion, rather than removing downstream training.

Forecasting moduleElectricity-LMNISTWeather-LExchange-L
Trainable linear rollout + trainable decoder0.9310 ± 0.01631.7933 ± 0.00030.3239 ± 0.00090.0430 ± 0.0001
Full Kalman-style correction + trainable decoder0.8502 ± 0.00141.7844 ± 0.00070.3054 ± 0.00620.0419 ± 0.0004

The results show that a trainable linear rollout alone does not recover the performance of the full model. K²SVD achieves lower mean NRMSE across all four datasets, with relative reductions of 8.7%, 0.5%, 5.7%, and 2.6%, respectively. The largest gain occurs on Electricity-L, while the smaller gain on MNIST is consistent with its more regular dynamics, which can already be modeled effectively by linear Koopman evolution. These results support the additional empirical value of the Kalman correction beyond a trainable linear rollout.

Q2: Which numerical entries are correct, and what supports the efficiency claim?

We apologize for the inconsistent entries across the original tables. The corrected full-model values are MNIST: 1.7844 ± 0.0007, Weather-L: 0.3054 ± 0.0062, and Exchange-L: 0.0419 ± 0.0004 in Tables 4, 12, and the baseline in Table 9. We corrected the result in the revised paper.

The increased efficiency comes not only from the lower rank of the latent space in K$^2$SVD compared to the baselines, but also from the fact that its parameter count does not increase with prediction length, while computational cost grows only linearly with the number of Kalman-style correction steps. The reported 2.8–5.5 speedup is a general summary of the results across datasets. On Weather-L, our model ranks first in inference speed and third in prediction accuracy, making it a favorable choice when computational efficiency is a priority.

Reviewer_ZLEDOpenReview ↗

Regarding W1

Thank you for the response and the new ablations. My concerns are not resolved, but because time is short I follow up on Weakness 1 first.

The finite-sample explanation in the paper holds only if the learned space is closed under $K$, and the response does not claim this. The paper says that "the patched training data provide too few samples for reliable regression" and that training lets the transition "adapt to the downstream multi-step forecasting objective, mitigating finite-sample error in the regression" (page 6).

If the span $V$ of the learned basis is closed under $K$ and the patch can be decoded exactly from $b$, the best one-step transition in $V$ is also the best multi-step one, so training on the multi-step objective can only correct sampling error as stated.

However, if $V$ is not closed, the two in general differ, and the gain can come from compensating for the space instead of the coefficients. The response to Reviewer AfSs agrees that the one-step approximation does not, by itself, guarantee an invariant subspace and states that the closure error cannot be quantified.


The decomposition in the response does not by itself explain the gain from training $K_\theta$. As in the response to Reviewer AfSs, assume a common invariant-measure space. For the single basis $b$ used in the rollout, let $P$ be the orthogonal projection onto its span $V$, so that the best approximation within $V$ is $K_d = PKP$.

For any matrix $W$, the regression residual splits into a zero-mean noise term and a term that depends only on $x$: $b(x') - W^\top b(x) = [b(x') - (Kb)(x)] + [(Kb)(x) - W^\top b(x)]$.

Since $(Kb)(x) = \mathbb{E}[b(x') \mid x]$, the noise term satisfies $\mathbb{E}[(b(x') - (Kb)(x)) h(x)^\top] = 0$ for every function $h$ (tower property). The population risk is therefore the noise variance plus $\mathbb{E}\lVert (Kb)(x) - W^\top b(x) \rVert^2$, and its minimizer $W^\star$ is the projection $W^{\star\top} b = PKb$.

So, $W^\star$ is the coefficient matrix of $K_d$ on $V$, and the coefficient term $\lVert K_d - \hat{K}_\theta \rVert_{HS}$ is zero at $W^\star$. The response describes $K_b^{\mathrm{ols}}$ as an estimate of these coefficients, so by its own account the regression already targets the minimizer of the coefficient term.

Within the decomposition, the forecast gain has one of two sources. By the above, either $K_b^{\mathrm{ols}}$ is far from $W^\star$, or $V$ is not closed under $K$.

The response does not cover the second. Rolling out the regression matrix for $j$ steps gives $(PKP)^j$, while the same argument applied to $K^j$ gives $PK^jP$ as the best $j$-step predictor.

The response to Reviewer BJbq attributes the gap to the coefficients, which holds only in the first case and leaves the second open. In the second case, training compensates for the space term that Stage 1 is said to control. The forecast then comes from the transition and filter matrices trained on the forecasting loss, not from the operator the encoders approximate.

The new ablations do not separate the two cases. The end-to-end encoder ablation and the trainable-linear rollout both train the transition matrix on the forecasting loss. The trainable-linear rollout (0.9310 on Electricity-L) also shows that training the transition and the decoder accounts for most of the improvement over the regression rollout in Table 4.

Question. Could you report the error of the regression rollout, the Linear row of Table 4, on the training windows used to fit $K_b^{\mathrm{ols}}$? If it is as poor there as on the test split, sampling error does not explain it, and the gain comes from the space rather than the coefficients. If it is small there, that supports the account in the response.

Reviewer_AfSsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes K2SVD, a two-stage time-series forecaster. Stage 1 learns a low-rank latent space with a Hilbert-Schmidt low-rank approximation objective taken from Jeong et al. Stage 2 freezes the encoders and trains a low-dimensional linear transition with a Kalman-style latent correction. It is validated on NRMSE, inference time and GPU memory against seven baselines, plus ablations. The main new contribution is the forecasting system and its efficiency evaluation, not a new operator-SVD principle.

Critical strengths
  1. Efficiency with competitive accuracy. In Table 2, K2SVD takes 10.1 s on Electricity versus 29.6 s for the fastest baseline (about 2.9x), and 0.44 GB versus 1.31 GB of GPU memory. Table 1 shows the best NRMSE on Electricity, MNIST and Exchange, though not on Weather. This is the paper's most consequential result: a very small structured latent space can give a useful accuracy-efficiency trade-off for resource-constrained forecasting.
Critical weaknesses
  1. The operator interpretation goes beyond the evidence. A good low-rank one-step approximation does not by itself yield an approximately invariant subspace for repeated rollout. Eq. 10 assumes this without closure-error, long-horizon or stability analysis. The factorized objective is also invariant to f -> Sf, g -> S^{-T}g, so without further normalization the network outputs are not canonical singular functions.

  2. "Kalman inference" is statistically weaker than the name implies. The observation in Eq. 13 is o_k = b_L[k], a linear prediction from the same history rather than new information, so the module blends two model trajectories, and the optimality conditions of a Kalman filter are not verified. If the gain depends only on time and fixed parameters and the decoder is linear, the correction is still linear in the initial latent state. The gain may therefore come from a more flexible linear head, which the ablations do not rule out.

Questions for authors
  1. How much of the advantage remains if the Kalman module is replaced by a linear multi-step head of equal capacity?
  2. Can you report subspace closure error and how it relates to long-horizon error?
  3. Section 4.2.2 says the decoder coefficients are fixed, but Table 5 optimizes them in Stage 2. Algorithm 1 updates K_theta, while the reference rollout uses fixed K_b^ols. What is the actual update and gradient path?
  4. Why do MNIST and Weather differ between Table 1 (1.7844, 0.3054) and Tables 4 and 12 (1.8125, 0.3124)?
  5. What is the measurement protocol for inference time and GPU memory (warm-up, synchronization, included operations, training or inference memory)?
3 comments

We thank reviewer AfSs for the careful assessment and constructive suggestions. We appreciate the recognition of K²SVD's accuracy-efficiency trade-off, which is a central motivation of our work. In response to the concerns, we have added new ablations to isolate the contributions of representation learning and downstream correction, clarified the theoretical scope of the learned subspace, and corrected the training and evaluation descriptions. We address each point below.

W1. The learned singular space and repeated rollout

Thank you for highlighting this distinction. Our claim concerns a learned basis approximating the dominant singular subspaces, not uniquely identified, normalized singular functions. The transformation f↦Sff\mapsto Sf, g↦S−⊤gg\mapsto S^{-\top}g changes the coordinates but preserves their spans and the factorized operator approximation.

We agree that a good one-step low-rank approximation does not, by itself, guarantee an invariant subspace or stable long-horizon rollout. Our claim is instead that the Hilbert–Schmidt approximation objective provides a connection to subspace closure. Our multi-step prediction follows the standard Koopman formulation, where operator powers propagate conditional expectations, and EDMD approximates this evolution through matrix powers in a learned observable space, as developed in Kostić et al. (2024) and section 3.2.2 of Jeong et al. (2025). We acknowledge this limitation and clarify that our proposed Kalman module is designed to mitigate accumulated rollout errors through sequential state estimation and correction, rather than relying solely on repeated linear propagation.

The Hilbert–Schmidt objective also provides a connection to subspace closure. Let H\mathcal{H} be the output span of the learned low-rank operator K^d\widehat{\mathcal{K}}_d, and let PHP_{\mathcal{H}} be its orthogonal projector. When the Koopman operator acts on a common invariant-measure space and is Hilbert–Schmidt as given by Kostić et al. (2024) and Jeong et al. (2025),

=\\|(I-P_{\mathcal{H}})\mathcal{K}P_{\mathcal{H}}\\|_{\mathrm{op}} \leq \\|\mathcal{K}-\widehat{\mathcal{K}}_d\\|_{\mathrm{HS}}.$$ Thus, reducing the HS approximation error also reduces an upper bound on the subspace closure error. **Q2. Can you report subspace closure error and how it relates to long-horizon error?** For a finite-dimensional approximation, both **subspace closure error and bias within the space both contribute to long-horizon rollout error**. The subspace closure error is bounded in stage 1 training on HS loss and frozen in stage 2 to preserve the optimized subspace. We cannot directly quantify the subspace closure error because the available data are finite, making it difficult to explicitly characterize the discrepancy between the subspaces. However, we provide evidence of the quality of the subspace learned with HS loss by comparing our model with a subspace trained end-to-end using only the Stage 2 MSE loss. The following results show the short-term prediction performance, while the long-term prediction results are detailed in Appendix D.4.2, which we have tested on the original version of the paper. - **Koopman SVD:** Train $f,g$ using the LoRA objective in Stage 1, then freeze them during Stage 2. - **End-to-end encoders:** Omit Stage 1 and jointly train $f,g$ and the forecasting components using the Stage-2 objective. Context/prediction lengths are 96/96 for the time-series datasets and 48/48 for MNIST. Entries report NRMSE with uncertainty terms; lower is better. | Representation / training scheme | Electricity-L | MNIST | Weather-L | Exchange-L | ETTh1 | ETTm1 | |:--|--:|--:|--:|--:|--:|--:| | End-to-end; $f,g$ trainable on forecasting loss | 0.9175 ± 0.0345 | 1.7932 ± 0.0014 | 0.3085 ± 0.0067 | 0.0422 ± 0.0019 | 0.6637 ± 0.0118 | 0.5764 ± 0.0092 | | Koopman SVD; LoRA-trained $f,g$ frozen in Stage 2 | **0.8502 ± 0.0014** | **1.7844 ± 0.0007** | **0.3054 ± 0.0062** | **0.0419 ± 0.0004** | **0.6124 ± 0.0091** | **0.5555 ± 0.0010** | The second bias term could be caused by poor estimation of the transition matrix or a shift in dynamics that a simple linear rollout cannot handle. We address both issues in the second stage using a trainable transition matrix and a Kalman correction module. Their effects are discussed in our response to Q1.

Q1. How much of the advantage remains if the Kalman module is replaced by a linear multi-step head of equal capacity?

Thank you for suggesting this important control. We agree that comparing against a trainable rollout better separates the benefit of optimization from that of the Kalman-style correction.

We applied a trainable-linear matrix multiplied repeatedly over the prediction horizon. It retains the Koopman representation learning in stage 1 and optimizes the downstream linear predictor using forecast error.

MethodElectricity-LMNISTWeather-LExchange-L
Trainable-linear rollout (without Kalman)0.9310 ± 0.01631.7933 ± 0.00030.3239 ± 0.00090.0430 ± 0.0001
Full K²SVD0.8502 ± 0.00141.7844 ± 0.00070.3054 ± 0.00620.0419 ± 0.0004

The metric is NRMSE; lower is better.

The results show that a trainable linear rollout alone does not recover the performance of the full model. K²SVD achieves lower mean NRMSE across all four datasets, with relative reductions of 8.7%, 0.5%, 5.7%, and 2.6%, respectively. The largest gain occurs on Electricity-L, while the smaller gain on MNIST is consistent with its more regular dynamics, which can already be modeled effectively by linear Koopman evolution. These results support the additional empirical value of the Kalman correction beyond a trainable linear rollout.

Q3. What is the actual update and gradient path?

Thank you for identifying these inconsistencies. We show the correct latest version aligned with other experiments in the following form and specify detailed settings and gradient map for all stage 1, stage 2, and inference sections. We will also revise the paper.

For the dynamic model, the update schedule is:

ComponentStage 1Stage 2Inference
Encoders f,gf,gOptimize with the LoRA objectiveFrozenFixed
Regression estimate KbOLSK_b^{\mathrm{OLS}}Estimated after representation learning--
Reference transition KθK_\thetaInactiveTrainableFixed
Kalman parameters A,H,Q,RA,H,Q,RInactiveTrainableFixed
Linear decoder DθD_\thetaInactiveTrainableFixed

Q4. Why do MNIST and Weather differ across tables?

We apologize for the inconsistent full-model entries. We correct this typo to align with the latest correct version of the model as the main comparison and the corresponding ablation tables to report:

  • MNIST: 1.7844 ± 0.0007
  • Weather-L: 0.3054 ± 0.0062
  • Exchange-L: 0.0419 ± 0.0004

Q5. What is the inference-time and GPU-memory measurement protocol?

Thank you for requesting this clarification.

  • Inference-time scope: We measure wall-clock time per evaluation batch, with a configured batch size of 32. We added a same variance decoder head for all point wise model including ours for fair comparison. The interval includes normalization, forecasting, output sampling, inverse scaling, and metric calculation, which is a full forward pass. Data fetching before the batch-start callback and epoch-end aggregation is excluded. Validation timings are collected during training with the model in evaluation mode. The current analysis averages the last ten logged validation batches per run, then averages across runs to reduce variability.
  • GPU memory: We record peak CUDA-allocated tensor memory during validation/test, resetting the peak counter at each phase’s start. Training memory is logged separately and not reported here.

We thank the reviewer again for these suggestions. They have helped us strengthen the experimental controls, clarify the training and inference procedures, and make the scope of our claims more precise.

Respected reviewer AfSs, thank you again for your careful review and constructive questions. As the discussion period ends in a few hours, we wanted to follow up briefly, as we have not heard your opinions from you on our responses. A revised manuscript containing your mentioned changes has been uploaded, with all revisions highlighted. If any point remains unclear, we would be glad to clarify it before the deadline. Thanks again for your review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.