acceptodds
Under review as a conference paper at ICLR 2027

Correct then Forecast: Observer State-Space Models for Time Series Forecasting

Abstract

Time series forecasting requires extrapolating the dynamics of an observed process beyond the last available measurement. Yet recurrent forecasting models typically treat observations as inputs that directly control their latent dynamics. It leads to a regime change when these observations become unavailable at prediction time. Following a state-estimation perspective, we introduce **Observer State-Space Models** (OSSMs), a class of recurrent models that **interprets the observed input time series as measurements of an underlying autonomous dynamical system**. OSSMs explicitly separate latent-state propagation from measurement assimilation: a single transition governs the dynamics across both context and forecasting intervals, while available observations **correct** the estimated state through an **observer**. This formulation naturally exposes classical control-theoretic properties, including observability and convergence of the state estimation error. We further show that conventional and recent SSMs can be recovered as particular instances of our OSSM framework, thereby providing a unified interpretation of their recurrent dynamics and revealing modeling inconsistencies. We perform experiments across several benchmarks showing that OSSM achieves substantial improvements while maintaining the same parameter count and training setup as the corresponding SSM baseline. These results support a simple principle for recurrent forecasting: observations should correct the estimated latent state, rather than control the dynamics used to propagate it.

Primary area:
learning on time series and dynamical systems
Keywords:
time series, forecasting, deep learning, control theory, state-space model, observer

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_E8RmOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Observer State-Space Models (OSSMs) for time series forecasting, motivated by the mismatch between observation-driven recurrent dynamics during the context window and free-running dynamics during forecasting. OSSM interprets observations as measurements that correct an autonomous latent state rather than as control inputs that determine its evolution. The authors connect this formulation to Luenberger observers, show that conventional closed-loop SSMs can be viewed as entangled instances of OSSM, and conduct a large controlled comparison across 14 datasets and 5,376 matched runs.

Critical strengths
  1. The paper provides a simple and well-motivated conceptual contribution: separating autonomous state propagation from observation-based state correction.
  2. The connection to classical observer theory gives the architecture a meaningful dynamical interpretation rather than introducing an ad-hoc forecasting module.
  3. The comparison with SSM is carefully controlled: encoders, parameter counts, seeds, optimization settings, and data splits are matched, with only the prediction-head parameterization changed.
Critical weaknesses
  1. Since OSSM and the closed-loop SSM are functionally equivalent model classes, the paper could characterize more precisely why the OSSM parameterization is easier to optimize. Currently, this is mostly inferred from the empirical results.
  2. The experiments focus on comparing OSSM with its matched SSM counterpart. This is appropriate for isolating the proposed principle, but leaves its competitiveness within stronger modern forecasting architectures less explored.
  3. OSSM receives an additional initialization constraint on the observer-error dynamics, ρ(AO−LOCO)=0.91\rho(A_O-L_OC_O)=0.91, whereas the SSM baseline only matches the free-running spectral radius. This makes it difficult to fully separate the benefit of the parameterization from the benefit of the initialization strategy.
Questions for authors
  1. Can the authors provide an ablation separating the effect of observer parameterization from the additional spectral initialization of AO−LOCOA_O-L_OC_O?
  2. Since OSSM and closed-loop SSMs represent the same function class, can the authors further analyze the optimization landscape or gradient conditioning to explain why OSSM converges more reliably?
2 comments

We sincerely thank the reviewer for the constructive assessment and for recognizing the motivation, observer-theoretic interpretation, and controlled evaluation. Your questions motivated a four-variant, function-matched ablation and a theoretical and empirical analysis of optimization. We summarize the most relevant findings below.

Q1.

We agree that matched initialization provides a direct way to disentangle the effects of parameterization (how OSSM explores the optimization landscape) and spectral initialization (how control theory guides the starting dynamics).

We therefore conducted the requested ablation using four variants:

ModelParameterizationInitialization
OSSM-originalOSSMOSSM
SSM-from-OSSMSSMMatched to OSSM
SSM-originalSSMSSM
OSSM-from-SSMOSSMMatched to SSM

The experiments cover the 14 datasets, four architectural configurations, multiple forecasting horizons, three seeds, and constant learning rates (10−5,10−4,10−3)(10^{-5},10^{-4},10^{-3}), alongside the original training schedule.

Matched variants share identical initial predictions, seeds, and batch ordering. We additionally monitor gradient evolution, effective transition matrices, spectral radii, innovation residuals, and optimization failures.

Some results at the paper's training setting:

DatasetParameterization gainInitialization gain
Electricity+14.9%+4.4%
Traffic+9.2%+3.6%
NYSE+7.4 %-1.7 %

Gains are averaged across five forecasting metrics.

The experiment demonstrates that the OSSM parameterization improves forecasting while both heads start from exactly the same function. The complementary initialization comparisons show a dataset-dependent effect: observer initialization helps on Traffic and Electricity but hurts on NYSE. Hence initialization alone does not explain OSSM's gains. Please find the setup and dataset results in the common response. Below, we provide further evidence of the different optimization trajectories induced by its parameterization.

Q2.

The above mapping is a bijection, so both heads represent exactly the same functions. They differ in which matrix a gradient step moves the most.

The prediction-head uses two transition matrices:

  • the context matrix, which runs over the context, up to 512 times;
  • the forecast matrix, which is applied once per forecast step, up to 720 times.

Because the forecast matrix is applied several times, the loss is most sensitive to it.

context matrixforecast matrix
SSMASA_SAS+BSCA_S + B_S C
OSSMAO−LOCA_O - L_O CAOA_O

Key point. A matrix built from several parameters moves faster under a gradient step, because the updates of all its parameters add up. In function space, the parameterization therefore acts as a preconditioner. It amplifies the step by 1+σmax⁡(G)2+σmax⁡(C)21+\sigma_{\max}(G)^2+\sigma_{\max}(C)^2, where GG is the gain (BSB_S or LOL_O), and σmax⁡\sigma_{\max} the largest singular value. This factor is ≃\simeq 6 from the SSM initialization.

  • The SSM applies this amplification to the forecast matrix, the sensitive direction.
  • The OSSM applies it to the context matrix, and the forecast matrix AOA_O receives a plain step.

Evidence from matched pairs (same initial function).

OSSM-from-SSM vs SSMOSSM vs SSM-from-OSSM
matrix that moves most during training, across all datasetscontext vs forecastcontext vs forecast
pairs with lower training loss (LR 10−410^{-4}), across all datasets392 vs 68359 vs 137
training-loss reduction, long-term datasets (LR 10−410^{-4})−41% vs −8%−40% vs −22%
training-loss reduction, long-term datasets (LR 10−510^{-5})−41% vs −37%−41% vs −40%
pairs with the larger gradient spike, long-term datasets (LR 10−410^{-4})44 vs 26067 vs 237
pairs with the larger gradient spike, long-term datasets (LR 10−510^{-5})95 vs 209156 vs 148

We observe, from the above table, that:

  • At a small learning rate, both heads optimize equally well.
  • At a larger one, the SSM stops making progress: its amplified steps on the forecast matrix produce large gradient spikes, after which training stalls.
  • A problem that tolerates only smaller step sizes is, by definition, worse conditioned.

The experiments and the theory support that OSSM gradients are better conditioned, explaining why OSSM converges more reliably, especially on long-term datasets.

These additional analyses will be part of the updated paper.

W2.

To complement Appendix B, we also tested the observer formulation within the original SpaceTime architecture, preserving its architecture and parameter budget. This preliminary study, based on short-term datasets (reported in the global response), provides evidence that the observer formulation is not tied to vanilla SSM. A comprehensive comparison with all modern forecasting architectures is indeed beyond the scope of the current controlled prediction head-level study.

Reviewer_zn7KOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Observer State-Space Models (OSSMs), which separate autonomous latent-state propagation from measurement correction using a Luenberger observer-inspired formulation. The authors establish three consistency properties and show that OSSM is equivalent in representational capacity to conventional closed-loop SSMs. Experiments across 14 datasets demonstrate improved forecasting accuracy and fewer divergent runs, suggesting benefits from the proposed parameterization.

Critical strengths
  1. The observer-based formulation provides an intuitive interpretation of context processing and forecasting through separate state propagation and measurement correction.

  2. The equivalence between OSSM and conventional SSMs is clearly derived. The analysis of SpaceTime's effective forecasting transition is also useful.

  3. The extensive evaluation across 14 datasets, multiple horizons, configurations, and seeds provides meaningful evidence of improvements under matched model capacity and training protocols.

  4. OSSM achieves promising forecasting improvements and fewer failed runs without increasing parameter count.

Critical weaknesses
  1. Limited methodological novelty. The authors acknowledge that OSSM "does not enlarge the class of linear recurrent prediction heads" (Section 3.2). Under the paper's own map (AO=AS+BSCSA_O = A_S + B_SC_S, LO=BSL_O = B_S, CO=CSC_O = C_S), OSSM's context transition (AO−LOCOA_O - L_OC_O) and forecasting transition (AOA_O) are identical to those of the closed-loop SSM. The consistency properties thus hold by construction and reflect which matrix is labeled "the dynamics" rather than a difference in computation. The contribution therefore appears to be a reparameterization rather than a fundamentally new forecasting mechanism.

  2. Insufficient positioning against prior observer-based forecasting work. The paper characterizes learning-based observers as primarily targeting state estimation, overlooking Deep KKL (Janny et al., 2021), Deep State Space Models (Rangapuram et al., 2018), and Ensinger et al. (2024), which use observer- or filter-based mechanisms for prediction. Moreover, Anand & Pappas (2026) already establish that SpaceTime can represent Luenberger-type predictors, including the steady-state Kalman predictor, substantially overlapping with Section 3.2 and Appendix B. The authors should clarify what OSSM contributes beyond these works, particularly whether the novelty lies in the observer formulation or its parameterization and initialization, and discuss its relationship to the classical Kalman innovations form.

  3. OSSM additionally constrains �0��0�, while the equivalent SSM context transition is not similarly controlled. This difference could explain part of the improved forecasting stability. Without initialization-matched comparisons, the benefits cannot be attributed solely to the observer parameterization.

  4. The observer-error analysis assumes �1��1�, which is not enforced during training. Generic observability does not guarantee well-conditioned estimation, and the learned spectra and observer-error behavior are not examined. Thus, the theoretical guarantees are not established for the learned model.

  5. Comparisons with stronger forecasting and observer-based baselines are missing. The proposed correction mechanism is not tested under noisy or intermittent observations. Additionally, the divergence criterion includes large test errors, and the results do not sufficiently separate optimization failures from forecasting improvements across configurations.\

References

  1. Janny, S., Andrieu, V., Nadri, M., & Wolf, C. (2021). Deep KKL: Data-driven Output Prediction for Non-Linear Systems. IEEE Conference on Decision and Control (CDC). https://arxiv.org/abs/2103.12443

  2. Rangapuram, S. S., Seeger, M. W., Gasthaus, J., Stella, L., Wang, Y., & Januschowski, T. (2018). Deep State Space Models for Time Series Forecasting. NeurIPS 31. https://proceedings.neurips.cc/paper_files/paper/2018/hash/5cf68969fb67aa6082363a6d4e6468e2-Abstract.html

  3. Ensinger, K., Ziesche, S., & Trimpe, S. (2024). Learning Hybrid Dynamics Models with Simulator-Informed Latent States. AAAI, 38(11), 11892–11900. https://doi.org/10.1609/aaai.v38i11.29075

  4. Anand, S. C., & Pappas, G. J. (2026). Adversarial Robustness of Deep State Space Models for Forecasting. arXiv:2604.03427. https://arxiv.org/abs/2604.03427

  5. Ljung, L. (1999). System Identification: Theory for the User. 2nd ed., Prentice Hall. https://www.rt.isy.liu.se/en/books/sysid/

Questions for authors
  1. Given the exact equivalence between OSSM and conventional closed-loop SSMs, what substantive modeling distinction is established by the proposed consistency properties? How does OSSM differ beyond reparameterizing the same effective context and forecasting dynamics?

  2. How much of the improvement comes from parameterization versus initialization? Can the authors initialize both models to identical functions using AS=AO−LOCOA_S=A_O-L_OC_O, BS=LOB_S=L_O, and CS=COC_S=C_O, then compare their forecasting performance and optimization stability?

  3. How do the theoretical observer convergence properties apply to the learned model, where E(yt)=COxtE(y_t)=C_Ox_t is not explicitly enforced? Can the authors analyze learned eigenvalue spectra, observability conditioning, and innovation residuals to validate the proposed state-estimation mechanism?

  4. How does OSSM differ from existing observer-based forecasting methods, particularly Deep KKL, Deep State Space Models, and Anand & Pappas (2026)? Can the authors clarify the novelty relative to classical Kalman innovations-form predictors and provide a comparison with an observer-parameterized SpaceTime baseline?

  5. Can the authors evaluate OSSM under noisy, missing, or intermittent observations to directly test the proposed correction mechanism? Additionally, can they separate genuine numerical divergence from poor forecasting performance and report whether the gains persist across learning-rate configurations and successfully converged runs?

3 comments

We sincerely thank the reviewer for their thoughtful, detailed, and constructive feedback. We greatly appreciate their careful evaluation of our work and valuable suggestions, which have helped us further strengthen the paper. Below, we address each concern and provide additional theoretical analysis and experimental evidence demonstrating the benefits of OSSM.

Q1.

Indeed, we show in the paper that SSMs can be expressed as OSSMs. This equivalence establishes that OSSM preserves the expressive power of conventional SSMs while providing an interpretable decomposition between autonomous dynamics and observation-driven correction. This decomposition also enables targeted control-theoretic inductive biases, such as spectral conditioning, whose benefits are further supported by our additional experiments.

Crucially, despite their equivalent expressive power, OSSM and SSM induce different parameter updates during training, leading to distinct learned functions.

To investigate this theoretically, we derive the gradients of both models using Backpropagation Through Time (BPTT). For functionally matched parameters, where both models initially produce identical latent trajectories and predictions, we obtain:

\begin{aligned} \nabla_{\mathbf A_S}\mathcal L =\nabla_{\mathbf A_O}\mathcal L, \quad \ \nabla_{\mathbf B_S}\mathcal L =\nabla_{\mathbf L_O}\mathcal L +(\nabla_{\mathbf A_O}\mathcal L)\mathbf C_O^\top, \quad \ \nabla_{\mathbf C_S}\mathcal L =\nabla_{\mathbf C_O}\mathcal L +\mathbf L_O^\top(\nabla_{\mathbf A_O}\mathcal L). \end{aligned}

Here, OO and SS denote OSSM and SSM parameters, respectively, and L\mathcal L is the loss function.

These expressions establish that SSM gradients contain additional contributions from the autonomous dynamics, resulting in different parameter updates even when both models start from identical functions. Our experiments further show that OSSM achieves more stable optimization, with smaller gradient spikes and fewer optimization stalls, particularly at larger learning rates.

Thus, the contribution extends beyond representational equivalence: OSSM introduces a control-theoretically structured parameterization that changes the optimization geometry and the resulting learned forecasting dynamics.

Q2.

We agree that matched initialization provides a direct way to disentangle the effects of parameterization (how OSSM explores the optimization landscape) and initialization (how control theory guides the starting dynamics).

We therefore conducted the requested ablation using four variants:

ModelParameterizationInitialization
OSSM-originalOSSMObserver-based
SSM-from-OSSMSSMMatched to OSSM
SSM-originalSSMConventional SSM
OSSM-from-SSMOSSMMatched to SSM

The experiments cover 14 datasets, four architectural configurations, multiple forecasting horizons, three seeds, and constant learning rates (10−5,10−4,10−3)(10^{-5},10^{-4},10^{-3}), alongside the original training schedule.

Matched variants share identical initial predictions, seeds, and batch ordering. We additionally monitor gradient evolution, effective transition matrices, spectral radii, innovation residuals, and optimization failures.

Some results at the paper's training setting:

DatasetParameterization gainInitialization gain
Electricity+14.9%+4.4%
Traffic+9.2%+3.6%
NYSE+7.4 %-1.7 %

Gains are averaged across five forecasting metrics.

These results confirm that both components can improve forecasting. The parameterization effect becomes particularly pronounced as the learning rate increases: OSSM handles large learning rates better, presenting fewer gradient spikes than SSM. Indeed, when SSM starts from the OSSM initialization, it presents the same number of gradient spikes across long-term datasets at lr=10−510^{-5},148 vs 156, whereas at lr=10−410^{-4}, it shifts to 237 vs 67.

Please find the complete details and dataset results in our general response.

Q3.

We thank the reviewer for highlighting the importance of examining whether the theoretical observer assumptions remain valid after training.

Our formulation, derived from the consistency properties, deliberately uses observer theory as an architectural inductive bias, without explicitly enforcing the classical observer assumptions throughout optimization.

To investigate the learned behavior, we additionally tracked the spectral radii of the autonomous and corrected dynamics, observability conditioning, and the evolution of innovation residuals and observer corrections across the four variants.

The results reveal a clear distinction in the learned dynamics. For example, starting from identical SSM initializations (AO−LOCO=ASA_O-L_OC_O=A_S), we have the following spectral radius median value after training:

DatasetOSSM-from-SSM ρ(AO−LOCO)\rho(A_O-L_OC_O)SSM-original ρ(AS)\rho(A_S)
Traffic0.8550.615
Electricity0.8930.623
ETTm10.8760.633

The SSM ρ(AS)\rho(A_S) remains close to its initial spectral radius (around 0.63-0.65), whereas OSSM learns substantially different dynamics, reflecting its distinct optimization behavior: from the same initialization, the observer-based parametrization carries more of the innovation state along the context window. These diagnostics provide further empirical evidence that the observer formulation influences the learned dynamics and correction behavior, beyond simply changing their mathematical representation.

We have monitored the post-training value of ρ(AO−LOCO)\rho(A_O-L_OC_O) during the ablation study. In the original OSSM setup, all the learned ρ(AO−LOCO)\rho(A_O-L_OC_O) are below 11 (with a maximum value of 0.9960.996). Even without explicitly imposing this constraint, the spectral-radius condition still holds. In addition, the observability condition is satisfied, thereby validating the observer-based framework.

We have tracked the evolution of the innovation term during training. OSSM does not necessarily reduce the innovation term much more than SSM. As an alternative, one could define a separate decoder that maps its predictions back to the original observation space while OSSM operates entirely in the encoder space, following Equation 3. We leave the investigation of this alternative architecture, including explicit consistency regularization, for future work.

Q4.

Positioning. We thank the reviewer for these valuable references, which help clarify our contribution.

We acknowledge that observer-based prediction and innovations-form models are established concepts. However, our work differs in its objective and formulation:

  • Deep KKL [1] studies output prediction through KKL observer theory, emphasizing the construction of an observer-based latent representation. OSSM instead introduces an explicitly trainable observer parameterization for general time-series forecasting architectures.

  • Deep State Space Models [2] employ RNN-conditioned, time-varying probabilistic SSMs. OSSM learns autonomous latent dynamics and an observer gain jointly through a forecasting objective, with an explicit distinction between context correction and autonomous forecasting.

  • Ensinger et al. [3] incorporate simulator information into learned latent trajectories. OSSM introduces its inductive bias directly through the recurrent architecture, without requiring simulator-generated data.

  • Anand and Pappas [4] establish that decoder-only SpaceTime can represent Luenberger observers. Our contribution is complementary: we introduce an observer-based reparameterization of SpaceTime, preserving its original structural assumptions through the additional matrix KSTK_{ST} (Appendix B.2).

Regarding classical Kalman innovations-form predictors, our correction mechanism builds on the same principle. Unlike a classical Kalman filter, OSSM learns both the autonomous dynamics and observer gain from data.

Comparison against SpaceTime. Following the reviewer's suggestion, we implemented an observer-parameterized SpaceTime and evaluated it under the same four-variant protocol, using three seeds and two learning rates.

DatasetParameterization effectInitialization effect
NN5+11.07 % (84.2 %)+29.11 % (92.5 %)
NYSE+3.08 % (57.9 %)-5.83 % (32.9 %)
ILI-0.50 % (48.3 %)-1.36 % (38.8 %)
NASDAQ+3.90 % (66.2 %)+8.46 % (88.3 %)
Mean (4 datasets)+4.39 % (64.2 %)+7.59 % (63.1 %)

These results demonstrate that the OSSM formulation can substantially improve an established forecasting architecture, with complementary benefits from its parameterization and observer-informed initialization.

We provide the complete SpaceTime comparison in the general response.

Q5.

Correction mechanism under corrupted observations.

Following the reviewer's suggestion, we conducted additional experiments assessing robustness to noisy, missing, and intermittent observations.

We evaluated previously trained models without retraining, corrupting only the input context using Gaussian noise, outliers, randomly missing observations, and contiguous missing intervals.

At the paper's training setting, the results on four short-horizon datasets (ILI, NASDAQ, NYSE, NN5) are:

Observation settingOSSM gain over SSM
Clean observations+8.8%
Gaussian noise, σ=1\sigma=1+13.9%
5% outliers+13.4%
10% missing interval before forecasting+21.7%

Gains are measured in normalized test MSE; bold corrupted-setting gains correspond to statistically significant robustness improvements.

These improvements persist when restricting the comparison to successfully converged models. The four-variant analysis further shows that observer-informed initialization is particularly beneficial for robustness to noise and structured missing observations.

Clarifying divergence.

We agree that numerical divergence and poor forecasting should be distinguished. In the original experiments, the reported failure counts combine two different outcomes:

Failure typeOSSMSSM
Numerical divergence (NaN/Inf)49117
Poor forecast (finite MSE > 10)47108
Total96225

Thus, OSSM exhibits approximately 2.4 times fewer numerical failures and 2.3 times fewer poor forecasts in the original evaluation.

Learning-rate stability.

We further examined these outcomes under multiple learning rates, separately tracking early numerical divergence and finite but poor forecasts.

At the more aggressive learning rate 10−310^{-3}, we have:

OutcomeOSSMSSM
Early numerical divergence3178
Poor forecast (MSE > 10)4473

OSSM substantially reduces early numerical failure and poor forecast counts.

At the original learning rate, the additional experiments exhibit substantially fewer failures, and OSSM retains its forecasting advantages on key datasets. These results provide additional evidence of OSSM's optimization benefits.

References

[1] Janny, S., Andrieu, V., Nadri, M., & Wolf, C. (2021). Deep KKL: Data-driven Output Prediction for Non-Linear Systems. IEEE Conference on Decision and Control (CDC). https://arxiv.org/abs/2103.12443

[2] Rangapuram, S. S., Seeger, M. W., Gasthaus, J., Stella, L., Wang, Y., & Januschowski, T. (2018). Deep State Space Models for Time Series Forecasting. NeurIPS 31. https://proceedings.neurips.cc/paper_files/paper/2018/hash/5cf68969fb67aa6082363a6d4e6468e2-Abstract.html

[3] Ensinger, K., Ziesche, S., & Trimpe, S. (2024). Learning Hybrid Dynamics Models with Simulator-Informed Latent States. AAAI, 38(11), 11892–11900. https://doi.org/10.1609/aaai.v38i11.29075

[4] Anand, S. C., & Pappas, G. J. (2026). Adversarial Robustness of Deep State Space Models for Forecasting. arXiv:2604.03427. https://arxiv.org/abs/2604.03427

Reviewer_31kgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Observer State-Space Models (OSSMs), a recurrent framework for time series forecasting that separates autonomous state propagation from measurement correction. The authors introduce three consistency properties, analyze the relationship between OSSM and conventional SSMs, and investigate the theoretical properties of the observer formulation. Extensive experiments across 14 benchmark datasets demonstrate improvements in forecasting accuracy and training robustness compared with conventional SSMs. However, the theoretical advantages over existing formulations remain insufficiently justified, and further analysis is needed to clarify the sources of the observed performance improvements.

Critical strengths

This paper addresses the discrepancy between observation-driven and free-running dynamics in recurrent time series forecasting. The proposed Observer State-Space Models (OSSMs) explicitly separate autonomous state propagation from measurement correction, providing a control-theoretic perspective on recurrent forecasting. The framework is relatively simple and offers a unified interpretation of conventional SSMs and SpaceTime. In addition, extensive experiments across 14 benchmark datasets, multiple forecasting horizons, model configurations, and random seeds demonstrate improvements in forecasting accuracy and training robustness under controlled comparisons.

Critical weaknesses
  1. Unclear novelty and theoretical advantages. OSSM builds on classical Luenberger observers, and the authors show that it has the same expressive power as conventional closed-loop SSMs for linear recurrent prediction heads. Although three consistency properties are introduced, it remains unclear whether they establish substantive distinctions beyond a different parameterization. Therefore, the theoretical advantages of OSSM over conventional SSMs are not sufficiently justified.

  2. Insufficient analysis of the performance improvements. Although extensive paired experiments demonstrate the advantages of OSSM, the proposed approach also introduces an additional initialization of the observer-error dynamics. The current experiments do not sufficiently distinguish the effects of parameterization and initialization, nor do they directly examine the role of measurement correction. Consequently, the main sources of the performance improvements remain unclear.

Questions for authors
  1. Given the mathematical equivalence between OSSM and conventional closed-loop SSMs, how do the three proposed consistency properties establish substantive differences between the two formulations?

  2. What are the key methodological differences between OSSM and existing learning-based observers, beyond their different application objectives?

  3. In Equation (4), how is the consistency between the encoded observation E(yt)E(y_t) and the predicted output y^t\hat{y}_t ensured?

  4. Do the learned OSSM parameters still satisfy the spectral radius condition required for state-estimation error convergence after training?

  5. How much of the performance improvement remains when the conventional SSM baseline adopts a matched initialization of its context-state dynamics?

  6. Section 4.1 describes the results using median improvements, while Table 2 reports mean ± standard deviation. Which aggregation method is actually used?

2 comments

Response to Reviewer 31kg

We sincerely thank the reviewer for their thoughtful, detailed, and constructive feedback. We greatly appreciate their positive assessment of our work and valuable suggestions, which have helped us further strengthen the paper. Below, we address each concern and present additional theoretical and experimental evidence supporting the benefits of OSSM.

Q1.

Indeed, OSSM and conventional closed-loop SSMs have the same expressive power as linear recurrent forecasting heads. This equivalence is an important part of our contribution: it shows that the three consistency properties can be satisfied without sacrificing expressivity or increasing the parameter count.

The substantive distinction is in how the same forecasting functions are learned. OSSM directly parameterizes the autonomous forecast transition as AOA_O, independently of the observer gain, whereas the equivalent SSM learns it through the coupled expression AS+BSCSA_S+B_SC_S. To investigate the consequences, we derived the gradients of both formulations using Backpropagation Through Time (BPTT). For two functionally matched models, the chain rule yields

\begin{aligned} \nabla_{A_S}\mathcal L = \nabla_{A_O}\mathcal L, \quad \ \nabla_{B_S}\mathcal L = \nabla_{L_O}\mathcal L+(\nabla_{A_O}\mathcal L)C_O^\top, \quad \ \nabla_{C_S}\mathcal L = \nabla_{C_O}\mathcal L+L_O^\top(\nabla_{A_O}\mathcal L). \end{aligned}

Thus, the SSM input and readout gradients contain additional contributions from the autonomous dynamics. Consequently, even when OSSM and SSM start from functionally identical parameters, their gradient updates follow different directions, leading to distinct optimization trajectories. Our experiments further show that OSSM achieves more stable optimization, with smaller gradient spikes and fewer optimization stalls, particularly at larger learning rates.

The novelty is not a larger function class, but an observer-based choice of learning coordinates that explores the parameter space differently.

Q2.

We thank the reviewer for the opportunity to clarify our positioning.

Existing approaches differ in what is learned from data and what is set by the knowledge. KalmanNet [1] learns the Kalman gain given known or partially known dynamics; Niazi et al. [2] learn an encoder for a fixed latent dynamical system; and Pathak et al. [3] learn a readout over fixed reservoir dynamics. Conversely, Herz et al. [4] learn the dynamics using a correction mechanism set by the partial observation setup, while Mikhaeil et al. [5] investigate learning strategies to tackle the context-forecast discrepancy in RNNs.

OSSM's contribution is therefore neither the identification of the context-forecast discrepancy nor the leverage of observer theory. OSSM's novelty relies on two aspects:

  • bringing control theory to address the context-forecast discrepancy and,
  • jointly learning from data the latent-state propagation and measurement correction for general time series forecasting.

We will revise the Related Work section to clarify OSSM positioning within the existing literature.

Q3.

We thank the reviewer for raising this important point. Modern time series forecasting models, including conventional SSMs, commonly employ an encoder (e.g., a Transformer backbone [6,7]) to map observations into a latent representation, followed by a prediction head that produces forecasts. We chose to follow this established architectural design in the OSSM formulation presented in Equation (4), where the encoded observations and predicted outputs are defined in a shared prediction head.

To further investigate this aspect, we monitored the evolution of the innovation residual during training, defined as

=\mathcal{E}(\mathbf{y}_t)-\mathbf{C}_O\hat{\mathbf{x}}_t.$$ Empirically, the correction applied by the OSSM does not enforce $\mathcal{E}(\mathbf{y}_t)$ and $\hat{\mathbf{y}}_t$ to be equal (they should not because they do not represent the same quantity). As an alternative, one could define a separate decoder that maps its predictions back to the original observation space while OSSM operates entirely in the encoder space, following Equation 3. We leave the investigation of this alternative architecture, including explicit consistency regularization, for future work. ## Q4. We have tracked the post-training value of $\rho(A_O-L_OC_O)$ during the ablation study. In the original OSSM setup, all the learned $\rho(A_O-L_OC_O)$ are below $1$ (with a maximum value of $0.996$). Even without explicitly imposing this constraint, the spectral-radius condition still holds.

Q5.

We agree that function-matched initialization is necessary to separate the effect of the OSSM parameterization (how the model explores the parameter space) from the effect of its observer-based initialization (how control theory guides the starting dynamics). We therefore conducted the following ablation:

ModelParameterizationInitialization
OSSM-originalOSSMOSSM
SSM-from-OSSMSSMOSSM
SSM-originalSSMSSM
OSSM-from-SSMOSSMSSM

Using AO=AS+BSCA_O=A_S+B_SC, LO=BSL_O=B_S, and AS=AO−LOCA_S=A_O-L_OC, the crossed variants have identical initial forecasting functions. Comparing models with the same initial function isolates parameterization; comparing with the same parameterization isolates initialization.

The ablation spans four architectural configurations, multiple horizons, up to three seeds, and constant learning rates 10−510^{-5}, 10−410^{-4}, and 10−310^{-3}, alongside the original training schedule. We additionally tracked gradient evolution, effective dynamics, and optimization failures.

Some results under the paper's training schedule:

DatasetParameterization gainInitialization gain
Electricity+14.9%+4.4%
Traffic+9.2%+3.6%
NYSE+7.4 %-1.7 %

Gains are averaged across five forecasting metrics.

The results establish that OSSM's gains cannot be attributed to its initialization alone: the ablation study reports statistically significant gains produced by OSSM parametrization.

Please find the complete tables of the ablation study in the general comment.

Q6.

Thank you for catching this typo. The reported results are indeed the mean, as stated in the Table 2 caption. It will be fixed in the revised version.

References

[1] Revach et al., KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics. Trans. Sig. Proc. 70 (2022), 1532–1547. https://doi.org/10.1109/TSP.2022.3158588

[2] Niazi et al., Learning-based design of luenberger observers for autonomous nonlinear systems, 2022. https://arxiv.org/abs/2210.01476

[3] Pathak et al., Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach. Phys. Rev. Lett., 120:024102, Jan 2018. https://doi.org/10.1103/PhysRevLett.120.024102

[4] Herz et al., Teacher forcing as generalized bayes: Optimization geometry mismatch in switching surrogates for chaotic dynamics. https://arxiv.org/abs/2604.25904

[5] Mikhaeil et al., On the difficulty of learning chaotic dynamics with rnns. In Advances in Neural Information Processing Systems, volume 35, pp. 11297–11312. Curran Associates, Inc., 2022. doi: 10.52202/068431-0821. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/495e55f361708bedbab5d81f92048dcd-Paper-Conference.pdf.

[6] Liu et al., iTransformer: Inverted transformers are effective for time series forecasting. ICLR, 2024. https://openreview.net/forum?id=JePfAI8fah

[7] Nie et al., A time series is worth 64 words: Long-term forecasting with transformers. ICLR, 2023. URL https://openreview.net/forum?id=Jbdc0vTOcol

Comments on the paper

3 comments
AuthorsOpenReview ↗

Global response

We sincerely thank Reviewers 31kg, zn7K, and E8Rm for their thoughtful and constructive assessments of OSSM. We greatly appreciate their recognition of the importance of the problem, the clarity of the observer-based interpretation, and the rigor of our empirical comparisons. Most importantly, their questions highlight two key aspects that can further strengthen our contribution: distinguishing the benefits of the observer parameterization from those of its initialization, and explaining why two functionally equivalent models can exhibit different training behaviors.

Following the reviewers' suggestions, we have carried out a targeted set of new theoretical analyses and controlled experiments. The results strengthen our central claim: despite having the same theoretical expressive power as conventional SSMs, OSSM provides a distinct and empirically valuable approach to autoregressive time series forecasting.

Additional evidence supporting OSSM

1. A precise mathematical explanation of the novelty. We derive the Backpropagation Through Time (BPTT) gradients for OSSM and SSM. Although both models can start from exactly the same initialization, their parameter gradients differ. The learned trajectories are different from the first update onward.

2. An ablation study to isolate confounding effects. Following the reviewers' proposed experiment, we perform an ablation study to isolate the impact of OSSM parametrization and initialization on performance, and we additionally train two variants: OSSM-from-SSM, an OSSM starting from the equivalent SSM initialization, and SSM-from-OSSM, an SSM starting from the equivalent OSSM initialization. Overall, the results demonstrate two distinct benefits from the OSSM parametrization and initialization, and the separation itself answers the principal concern raised independently by all three reviewers.

3. Optimization evidence supporting the gradient analysis. Our BPTT analysis shows that, despite their functional equivalence, OSSM and SSM distribute gradients differently across their parameters, leading to distinct optimization trajectories. We confirm this experimentally by initializing both models identically and tracking their parameter updates throughout training. OSSM consistently induces larger updates to the context transition, whereas SSM favors the autonomous forecasting transition during training. This asymmetry persists across datasets and learning rates, resulting in distinct learned spectral dynamics. Together with the larger gradient peaks, optimization plateaus, and struggles at large learning rates observed for SSM, these findings provide empirical evidence that explicitly decoupling autonomous dynamics from measurement correction changes the optimization behavior, leading to a more stable training behavior.

4. Additional experiments. We additionally evaluated already-trained checkpoints under noisy, outlier-contaminated, and intermittent test-time observations. The results provide further insights into OSSM's behavior under imperfect observations, highlighting the role of observer initialization in improving robustness under certain conditions. We also extended our evaluation to an observer-parameterized SpaceTime architecture, where the results show promising improvements on short-term benchmarks, providing additional evidence of the potential benefits of the OSSM formulation.

All these points will be part of the updated version of the paper. We thank the reviewers again for their valuable feedback and suggestions, which provide additional evidence supporting OSSM.

AuthorsOpenReview ↗

Ablation study details

Model variants.

variantparameterizationinitialization
OSSM-originalOSSMOSSM
SSM-from-OSSMSSMOSSM
SSM-originalSSMSSM
OSSM-from-SSMOSSMSSM

Comparisons.

Training protocol :

  • optimizer: Adam, MSE loss, batch size 32;
  • at most 30 epochs, early stopping on the validation loss with patience 5;

Grid. 4 architectures (2 encoder layers; (dmodeld_{\rm model}, dxd_x) ∈\in {(64, 64), (128, 64), (128, 128), (128, 256)}) × 4 horizons × 3 seeds (2021,2022,2023) × learning rate = 48 matched pairs per comparison and dataset, i.e., 768 runs per dataset.

  • learning rate: 1e-4 with the type1 schedule; 1e-3, 1e-4, and 1e-5 with a constant schedule.
  • Long-term datasets: look-back 512, horizons 96 / 192 / 336 / 720.
  • Short-term datasets (ILI, NASDAQ, NYSE, NN5): look-back 104, horizons 24 / 36 / 48 / 60.

Each cell = 5-metric mean gain, with the win rate in parentheses. Positive = the OSSM side is better. Bold = positive gain or win rate above 50 %.

  • Parameterization gain: OSSM parameterization vs SSM parameterization, from both starting points.
  • Initialization gain: OSSM init vs SSM init, under both parameterizations.
  • The gain is the average of the 5-metric means.

Results at the paper learning rate setting.

DatasetParameterization gainInitialization gain
Short-term forecasting
NN5+3.13 % (45.5 %)+11.49 % (88.1 %)
NYSE+7.39 % (62.1 %)-1.65 % (50.9 %)
ILI+0.15 % (48.2 %)+3.43 % (62.6 %)
NASDAQ+1.69 % (62.9 %)-3.88 % (22.3 %)
Long-term forecasting
Electricity+14.89 % (74.3 %)+4.35 % (69.3 %)
Solar+5.44 % (53.3 %)+0.91 % (47.5 %)
Traffic+9.17 % (71.3 %)+3.63 % (71.1 %)
ETTh1+2.72 % (61.1 %)-0.51 % (49.3 %)
ETTm1+0.97 % (58.9 %)-2.32 % (36.6 %)
Wind+1.59 % (62.7 %)-0.99 % (32.3 %)
ETTm2+0.39 % (59.4 %)+0.03 % (54.1 %)
Exchange+1.03 % (56.6 %)+0.90 % (45.5 %)
ETTh2-0.26 % (45.9 %)-0.17 % (46.0 %)
Weather+0.70 % (47.1 %)-0.65 % (42.3 %)
Mean (14 datasets)+3.50 % (57.8 %)+1.04 % (51.3 %)

Parameterization gain, constant learning rate schedule.

Datasetlr 1e-5lr 1e-4lr 1e-3
Short-term forecasting
NN5-2.64 % (27.8 %)+1.37 % (74.5 %)+15.92 % (70.1 %)
NYSE-0.43 % (37.5 %)+8.50 % (69.8 %)+19.03 % (81.2 %)
ILI-3.42 % (39.8 %)-1.50 % (45.4 %)+9.14 % (75.6 %)
NASDAQ-0.29 % (46.7 %)+1.56 % (63.8 %)+4.81 % (79.2 %)
Long-term forecasting
Electricity+1.81 % (67.5 %)+22.97 % (85.4 %)+13.29 % (63.3 %)
Solar-3.86 % (42.5 %)+9.38 % (68.1 %)+10.80 % (49.2 %)
Traffic+0.94 % (62.1 %)+13.68 % (79.0 %)+11.43 % (61.7 %)
ETTh1+0.17 % (50.0 %)+3.33 % (65.5 %)+5.68 % (70.0 %)
ETTm1-0.53 % (51.0 %)+3.34 % (68.0 %)-1.12 % (52.5 %)
Wind-0.17 % (45.0 %)+1.53 % (75.2 %)+4.52 % (67.1 %)
ETTm2-0.46 % (47.5 %)+0.91 % (72.0 %)+0.98 % (59.5 %)
Exchange-2.07 % (45.2 %)+0.53 % (57.3 %)+3.62 % (66.7 %)
ETTh2-0.12 % (47.1 %)-0.73 % (43.1 %)+0.27 % (54.5 %)
Weather-1.64 % (28.7 %)+1.18 % (51.5 %)+2.39 % (57.5 %)
Mean (14 datasets)-0.91 % (45.6 %)+4.72 % (65.6 %)+7.20 % (64.9 %)

Initialization gain, constant learning rate schedule

Datasetlr 1e-5lr 1e-4lr 1e-3
Short-term forecasting
NN5+19.19 % (100.0 %)+7.86 % (95.8 %)+7.40 % (56.7 %)
NYSE-1.50 % (46.9 %)-5.30 % (48.3 %)-3.38 % (47.5 %)
ILI+6.21 % (72.7 %)+2.28 % (57.7 %)-0.54 % (49.8 %)
NASDAQ-4.66 % (12.3 %)-4.42 % (22.9 %)-3.19 % (42.1 %)
Long-term forecasting
Electricity+3.28 % (60.8 %)+4.20 % (75.4 %)+3.29 % (60.4 %)
Solar-5.06 % (31.2 %)+3.95 % (55.0 %)+3.85 % (56.2 %)
Traffic+2.56 % (76.5 %)+3.04 % (67.9 %)+2.90 % (54.9 %)
ETTh1-2.53 % (43.0 %)+1.30 % (59.5 %)-1.53 % (47.5 %)
ETTm1-8.32 % (13.5 %)-0.73 % (44.0 %)+0.48 % (51.5 %)
Wind-1.56 % (15.0 %)-0.54 % (38.3 %)-0.83 % (43.5 %)
ETTm2-0.15 % (51.5 %)+0.33 % (58.5 %)+0.04 % (57.5 %)
Exchange+6.28 % (45.4 %)-1.29 % (37.3 %)-0.88 % (52.1 %)
ETTh2-0.66 % (41.2 %)+0.28 % (54.2 %)-0.38 % (47.7 %)
Weather-2.02 % (29.6 %)-0.56 % (39.8 %)+0.20 % (54.5 %)
Mean (14 datasets)+0.79 % (45.7 %)+0.74 % (53.9 %)+0.53 % (51.6 %)
AuthorsOpenReview ↗

SpaceTime comparison and noisy-missing data setup experiment

SpaceTime comparison

Goal. The same 2 × 2 ablation (parameterization × initialization), run inside a published SSM forecaster. We keep the full SpaceTime architecture (Zhang et al., 2023) and replace only its closed-loop decoder head by an OSSM head.

Four variants.

variantparameterizationinitialization
OSSM-originalOSSMOSSM
ST-from-OSSMSpaceTimeOSSM
SpaceTime-originalSpaceTimeSpaceTime
OSSM-from-STOSSMSpaceTime

Initializations.

  • SpaceTime init: SpaceTime's own random draws (ρ(K) ≈ ρ(F) ≈ 0.98).
  • OSSM init: the same draws for A, C and K. The gain L is set so that ρ(A − L K) = 0.91.

Training protocol. Follows SpaceTime protocol.

  • Optimizer and loss: AdamW (weight decay 10⁻⁴), RMSE loss, dropout 0.25, batch size 32.
  • Learning rate: 2 warm-up epochs at 10⁻⁶, a 10-epoch linear ramp to the peak learning rate, then cosine decay. Peak learning rate 10⁻³ or 10⁻² (10⁻² is SpaceTime's default and was the best in our tuning).
  • At most 100 epochs, with early stopping on the validation loss (patience 20); the test set is scored with the best-validation checkpoint.

Grid. The four short-term datasets (ILI, NASDAQ, NYSE, NN5), look-back 104, horizons 24 / 36 / 48 / 60 × 3 seeds (2021,2022,2023). That gives 12 matched pairs per comparison, dataset and learning rate, or 384 runs in total.

Metrics and aggregation. These are identical to the OSSM-vs-SSM ablation: 5-metric mean gain of the means, positive = the OSSM side is better; win rate pooled over pairs × 5 metrics; a diverged run (non-finite or test MSE > 10) loses all five metrics.

Results

Each cell gives the 5-metric mean gain, with the win rate in parentheses. Positive = the OSSM side is better. Bold = positive gain or win rate above 50 %.

Summary, averaged over the two learning rates

DatasetParameterization effectInitialization effect
NN5+11.07 % (84.2 %)+29.11 % (92.5 %)
NYSE+3.08 % (57.9 %)-5.83 % (32.9 %)
ILI-0.50 % (48.3 %)-1.36 % (38.8 %)
NASDAQ+3.90 % (66.2 %)+8.46 % (88.3 %)
Mean (4 datasets)+4.39 % (64.2 %)+7.59 % (63.1 %)

Parameterization effect, each learning rate

Datasetlr 1e-3lr 1e-2
NN5+10.64 % (95.8 %)+11.50 % (72.5 %)
NYSE+2.81 % (57.5 %)+3.36 % (58.3 %)
ILI+0.66 % (52.5 %)-1.65 % (44.2 %)
NASDAQ+1.60 % (63.3 %)+6.21 % (69.2 %)
Mean (4 datasets)+3.93 % (67.3 %)+4.85 % (61.0 %)

Initialization effect, each learning rate

Datasetlr 1e-3lr 1e-2
NN5+37.83 % (100.0 %)+20.39 % (85.0 %)
NYSE-8.69 % (34.2 %)-2.97 % (31.7 %)
ILI-0.53 % (41.7 %)-2.18 % (35.8 %)
NASDAQ+8.60 % (95.8 %)+8.32 % (80.8 %)
Mean (4 datasets)+9.30 % (67.9 %)+5.89 % (58.3 %)

Robustness to noisy, missing and intermittent observations

We only describe here the protocol. Full tables and experiments will be provided in the appendix of the update paper.

Setup. No retraining: we reload the trained checkpoints of the four variants (lr 1e-4 type1; constant 1e-5/1e-4/1e-3; 3 seeds) and corrupt only the test look-back windows, identically for all models.

  • Noisy observations: we add Gaussian noise to every value of the window, with a standard deviation of 0.1, 0.3 or 1 times that of the channel, or we replace 5 % of the values by outliers of ±5 standard deviations.
  • Missing observations: we remove 10 %, 30 % or 50 % of the time steps at random, or one contiguous block covering 10 % of the window, placed at random, a quarter of the window before the forecast origin, or right before it. Missing values are either forward-filled (last observed value repeated) or masked.
  • Data: ILI, NASDAQ, NYSE, NN5 (h24/60), ETTh2, Exchange (h96/720).
  • Metric: OSSM gain in mean test MSE, and each model's degradation vs its own clean score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.