acceptodds
Withdrawn from ICLR 2027

SPOT-VSA: Efficient Stability-Plasticity Tunable On-Line Time-Series Forecasting With Vector Symbolic Architectures

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

16 views

Abstract

Online time-series forecasting requires rapid adaptation to non-stationary data while preserving previously acquired patterns, confronting the stability–plasticity dilemma. Current approaches struggle along this frontier, either suffering catastrophic forgetting through aggressive adaptation or incurring heavy computational overhead through redundant or frozen backbones, and none quantifies its position on the tradeoff. We propose SPOT-VSA, an efficient neuro-vector-symbolic architecture for task-free online forecasting. By projecting inputs into a latent space via Vector Symbolic Architectures, the model introduces an explicit inductive bias where short- and long-term temporal dependencies are bound into a single representation with negligible interference. Forecasts are produced by a hybrid sequence-to-sequence and autoregressive strategy whose granularity sets the model's position on the stability–plasticity frontier. We ground this claim theoretically, deriving bounds on stability and plasticity that identify forecasting granularity as a calibration knob and show that the step-size tradeoff between the two emerges only under distribution drift. Across diverse benchmarks and horizons, SPOT-VSA matches or surpasses state-of-the-art online forecasters in accuracy, exhibits the lowest forgetting among methods adapting their full model, and validates theoretical predictions on synthetic drift streams. In addition, it achieves up to speedup and higher energy efficiency on edge devices, with up to fewer parameters.We will make our code available after paper acceptance.

Primary area:
learning on time series and dynamical systems
Keywords:
Online Time Series Forecasting, Vector Symbolic Architectures

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_W3Z6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper targets task-free online multivariate forecasting under a leak-free, horizon-strided protocol (stride = H, one update per window). Its central idea is to produce the H-step forecast in ss chunks, each fed back as input for the next, and to use this granularity ss as a knob between plasticity and stability. Section 2.8 derives worst-case bounds on both as functions of ss, HH, and the learning rate. SPOT-VSA applies: instance normalization, an encoder with learnable WW and frozen Gaussian PP, a per-channel "persistent" gain, SiLU activation, an optional channel mixer, and a linear decoder that outputs one chunk. Experiments cover 8 datasets with H∈{24,168,336}H \in \{ 24, 168, 336 \} against 8 online baselines, scored by online RMSE/MAE, and a probe-based forgetting ratio (FR). SPOT-VSA is among the top two in 16/23 RMSE cells and forgets least among methods that update all their parameters

Critical strengths
  1. A tiny online forecaster that is competitive under a strict protocol.

    • With 0.17–0.43M parameters, SPOT-VSA has the best RMSE in 10/23 cells and is best or second in 16/23 (Table 2)
    • Among fully adapted methods, it has the lowest mean FR: 1.09, against 1.20–13.9 for the baselines (Table 3)
    • It is 18–404x faster on real edge hardware, measured with a documented power protocol (Table 4, Appendix A.7)
  2. A uniform, leak-free evaluation reported transparently.

    • All methods are scored under the same stride-H protocol, and retention is measured
    • The appendix reports seed standard deviations, frozen offline references (Table 7), swaps of the update rule on a fixed backbone (Table 17), and failure cases (ETTm2 336 ECL, DD is not a stability knob)
Critical weaknesses
  1. The experimental setup differs across methods in ways that favor SPOT-VSA.

    • Normalization. SPOT-VSA uses a validation-selected instance-normalization anchor. FSNet, OneNet, ER, DER++ and LSTD run without instance normalization (Table 12)
    • Lookback. SPOT-VSA uses L=2HL = 2H, against L=HL = H for the baselines at H≥168H \geq 168 (Appendix A.4.2). This contradicts Appendix A.6.4, which states that baselines are evaluated under "identical splits, scalers, look-back contexts"
    • Learning rate. SPOT-VSA's η\eta is tuned per dataset. The baselines keep their published stride-1 configurations
    • Update steps. On ECL, SPOT-VSA takes 10 extra decoder steps per window despite the stated ninner=1n_{\mathrm{inner}} = 1
    • Frozen models. Frozen offline models (FR=1\mathrm{FR} = 1 by construction) beat SPOT-VSA in 11/23 RMSE and 14/23 MAE cells, and beat every online baseline in 13/23 RMSE and 17/23 MAE cells (Table 7 vs Table 2). Therefore, the baselines appear under-configured
  2. The paper's two signature ideas are not what produces its results.

    • Granularity as a stability–plasticity knob is switched off in every headline result. Tables 2 and 3 use s=1s = 1, which is plain one-shot forecasting. Appendix A.6.3 states that this "is not the optimal chunk size". Where chunking is tested, it rarely helps:
      • In the component ablation, s=2s = 2 and s=4s = 4 are never better than s=1s = 1 on any of the five cells (Table 5a)
      • In the dedicated sweeps, the best ss improves MSE by 3–13% on three real cells, even with ss chosen on the test stream (Table 16). On synthetic streams, the gain is 1–2%, and it appears only at large learning rates
    • Vector-symbolic components add little:
      • Removing the frozen matrix PP changes MSE by 0.7–6% on three cells (Table 5a)
      • Φl\Phi_l changes MSE by 0.6–8%, and its effect on FR goes in both directions (Table 5b)
      • D=50D = 50, smaller than the input window LL, is within 1% of D=500D = 500, or better, on all eight cells (Table 13). This contradicts the high-dimensional, quasi-orthogonality rationale of Section 2.4
      • The "short-term" and "long-term" losses (LsL_s, LlL_l) are the same MSE on the same chunk, with the same η\eta
Questions for authors
  1. Table 17's SPOT-VSA MSE differs from Table 15's by up to 2.4×2.4\times (ETTm2 336: 1.705 vs 4.131), although Appendix A.7.4 says the two may differ "slightly". Which settings differ, and which numbers should readers rely on?
  2. Apart from weight decay, what distinguishes Φl\Phi_l from the other parameters? Section 2.4 says that making PP trainable induces forgetting (Appendix A.6.5), but no trainable-PP results are reported. Please add them
  3. Does the advantage in Table 2 persist when the baselines (at least DSOF, LSTD, OneNet, ER/DER++) get the same instance normalization, look-back (2H2H) and validation-tuned η\eta, and SPOT-VSA uses ninner=1n_{\mathrm{inner}} = 1 on ECL?
  4. How do the headline results change with ss selected on validation instead of s=1s = 1? How does SPOT-VSA compare with RLinear/NLinear (same normalization, validation-tuned η\eta)? Since W⊙PW \odot P is Frobenius-normalized (Listing 1), pre-activations start near zero, where SiLU is nearly linear. Do they stay there after warm-up?
  5. Do Theorems 2.2 and 2.3, derived for plain gradient descent, still hold under AdamW, which all experiments use? And why does stability worsen with ss on synthetic streams (Figure 4) but improve with ss on real ones (Figure 5a)?
Reviewer_p7y5OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces an online learning method for time series forecasting aiming at striking a balance between stability and plasticity. The problem is relevant. However, the presentation of the paper and discussion of the results make it very difficult to appreciate the contributions of the work and raise several concerns regarding correctness. Several results are overstated and misrepresented.

Critical strengths

Online/continual learning in modern time series forecasting architectures is a relevant problem which definitely deserves more attention and research efforts.

Critical weaknesses

Main issues:

  • The presentation of the paper has serious issues. The paper often uses loosely defined terms, and it is extremely vague in describing the motivations for the proposed design. A few examples of senteces/terms with no clear technical meaning understandable by a human reader:

    • From the abstract: "short- and long-term temporal dependencies are bound into a single representation with negligible interference. Forecasts are produced by a hybrid sequence-to-sequence and autoregressive strategy whose granularity sets the model’s position on the stability–plasticity frontier."
    • "P insulates input-space geometry from online updates."
    • "Table 1 states the bounds as directional predictions, where an interior optimum lies strictly inside a knob’s range."
    • "We design a vector-symbolic forecaster that binds a dynamic window encoding to a persistent latent representation"
    • "function-space displacement"
  • Many results are overstated and misrepresented throughout the paper. Again, a few examples:

    • Section 2.4: Multiplying the weights of a linear layer by a random matrix does not change the class of models that can be learned (it's just a scaling). At initialization, Proposition 2.1 is also true for a standard linear layer with randomly initialized weights. Moreover, the proposition assumes the weights to be fixed, which means that the result does not say anything about the stability of the method in an online learning setup. It is not clear what "P insulates input-space geometry from online updates" is supposed to mean. Moreover, guaranteeing some properties in the encoder does not imply that such properties are preserved as representations go through downstream processing.
    • Section 2.5: it is not clear to what "relational inductive bias" the paper is referring to. Again, the paper hints at properties of the proposed design without enough support and discussion of the claims. The operation described in 2.5 is again just a scaling.
    • The paper characterized the method as "symbolic", however, it doesn't seem to rely on any symbolic representations (again, the encoder behaves like a standard linear layer) and does not discuss why that should be the case.
    • Section 2.8: the presentation makes it again quite difficult to understand whether the theoretical results are simply derived from existing work or, if not, how the obtained insights are technically novel. The role of the introduced components in granting such properties is also not clear (are these properties unique to the proposed architecture?). I would encourage the authors to simplify the terminology and ensure that the technical contributions are appropriately and clearly discussed. Again, the presentation is disconnected, and the discussion is difficult to follow. For example, Theorem 2.2 relies on a term described as "rollout compuding noise", however its formal definition is not given anywhere, not even in the appendix. As a side note, bounds can be quite loose and do not necessarily describe practical scenarios. I'd avoid translating them directly into predictions as in Tab. 1.
  • Empirical results have several issues:

    • Results in Table 7 show that the proposed method is often outperformed by standard frozen methods that are simply trained on the warm-up split. This raises several doubts about the effectiveness of the method and should be clearly discussed. Comparisons with standard adaptive linear predictors would also be interesting (e.g., by using recursive least squares): the closest available results are in Table 15, but the tuned version of online DLinear is described in the text but omitted in the table.
    • The paper claims large speedups over baselines, but simpler, more scalable baselines are not considered for comparison.

In summary, this paper falls well below the acceptance threshold for ICLR.

Minor issues:

  • The main body of the paper is not self-contained, as relevant results are postponed to the appendix, which is frequently referenced throughout the text.
  • Some acronyms (e.g., CLS) are never defined.
Questions for authors

See weaknesses.

Reviewer_Mf8jOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes SPOT-VSA, an efficient online forecasting model using high-dimensional (VSA) representations. The architecture binds a frozen random projection with a learnable matrix, gates it with a persistent inductive bias, and uses a linear decoder. It also formalizes and analyzes the stability–plasticity tradeoff, bounding it in terms of horizon, chunk count, and learning rate. The model is benchmarked on standard online forecasting datasets using RMSE, MAE, and forgetting ratio (FR), under a strict leakage-free protocol motivated by [1]. The authors claim it matches or surpasses state-of-the-art online forecasters, has the lowest forgetting among fully adapted models, and achieves up to 404× speedup and 390× energy savings on edge devices.

[1] Lau et al., Fast and Slow Streams for Online Time Series Forecasting Without Information Leakage, ICLR 2025.

Critical strengths
  1. Formalization and analysis of the stability–plasticity trade-off. The paper provides a much-needed formalization of the stability–plasticity trade-off for online time-series forecasting, together with an interesting analysis of how horizon, chunk count, and learning rate affect it.

  2. Efficient model for online forecasting. SPOT-VSA uses far fewer parameters than the baselines and substantially reduces latency and energy on edge hardware.

Critical weaknesses
  1. The trade-off claim in Section 3.2 is not supported by clear analysis. In Section 3.2, the paper claims that "SPOT-VSA yields the best stability-plasticity trade-off with fewer parameters." However, it provides no clear analysis showing where SPOT-VSA sits on the trade-off relative to the other methods, or whether a better trade-off leads to better benchmark results. Tables 2 and 3 show that SPOT-VSA does not dominate on all datasets. For example, on WTH, LSTD achieves better RMSE at all horizons but has higher FR, so neither model clearly has the better trade-off.

  2. The benchmarks do not reflect the efficiency motivation. All benchmark datasets have long sampling intervals (15 minutes to daily), far longer than all latencies reported in Table 4. Efficiency therefore gives little practical advantage on these benchmarks. Since efficiency is one of the main contributions, its practical benefit is not demonstrated.

Questions for authors

Clarifications:

  1. The paper states that quasi-orthogonality bounds inter-lag interference, but lacks a detailed explanation of how this benefits the stability–plasticity trade-off, or online forecasting more generally. Can you clarify this?

  2. Section 3.2.2 does not reference Appendix A.7 for the measurement protocol, and the reported speedup and energy numbers do not state which baseline they are compared against. Can you add the reference and specify the baseline for each number to remove ambiguity?

Suggestions:

  1. Can you add an analysis showing where SPOT-VSA and all baselines sit on the stability–plasticity trade-off? For example, a figure plotting FR against RMSE/MAE for all methods would make this comparison clear.

  2. Are there existing benchmarks more relevant to the setting where efficiency matters? Can you add an evaluation on them?

  3. Section 2 currently combines the related work, preliminaries, proposed method, and theoretical analysis, which makes the paper hard to follow. Can you separate the related work, preliminaries, and method into different sections? Can you also add a notation table to the appendix to make the math easier to follow?

  4. Can you increase the text size of the figures and tables? Figures 1, 2, 4, 5, and 6 and Tables 4 and 5 are the hardest to read, but text in all tables is too small.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.