- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper targets task-free online multivariate forecasting under a leak-free, horizon-strided protocol (stride = H, one update per window). Its central idea is to produce the H-step forecast in chunks, each fed back as input for the next, and to use this granularity as a knob between plasticity and stability. Section 2.8 derives worst-case bounds on both as functions of , , and the learning rate. SPOT-VSA applies: instance normalization, an encoder with learnable and frozen Gaussian , a per-channel "persistent" gain, SiLU activation, an optional channel mixer, and a linear decoder that outputs one chunk. Experiments cover 8 datasets with against 8 online baselines, scored by online RMSE/MAE, and a probe-based forgetting ratio (FR). SPOT-VSA is among the top two in 16/23 RMSE cells and forgets least among methods that update all their parameters
Critical strengths
-
A tiny online forecaster that is competitive under a strict protocol.
- With 0.17–0.43M parameters, SPOT-VSA has the best RMSE in 10/23 cells and is best or second in 16/23 (Table 2)
- Among fully adapted methods, it has the lowest mean FR: 1.09, against 1.20–13.9 for the baselines (Table 3)
- It is 18–404x faster on real edge hardware, measured with a documented power protocol (Table 4, Appendix A.7)
-
A uniform, leak-free evaluation reported transparently.
- All methods are scored under the same stride-H protocol, and retention is measured
- The appendix reports seed standard deviations, frozen offline references (Table 7), swaps of the update rule on a fixed backbone (Table 17), and failure cases (ETTm2 336 ECL, is not a stability knob)
Critical weaknesses
-
The experimental setup differs across methods in ways that favor SPOT-VSA.
- Normalization. SPOT-VSA uses a validation-selected instance-normalization anchor. FSNet, OneNet, ER, DER++ and LSTD run without instance normalization (Table 12)
- Lookback. SPOT-VSA uses , against for the baselines at (Appendix A.4.2). This contradicts Appendix A.6.4, which states that baselines are evaluated under "identical splits, scalers, look-back contexts"
- Learning rate. SPOT-VSA's is tuned per dataset. The baselines keep their published stride-1 configurations
- Update steps. On ECL, SPOT-VSA takes 10 extra decoder steps per window despite the stated
- Frozen models. Frozen offline models ( by construction) beat SPOT-VSA in 11/23 RMSE and 14/23 MAE cells, and beat every online baseline in 13/23 RMSE and 17/23 MAE cells (Table 7 vs Table 2). Therefore, the baselines appear under-configured
-
The paper's two signature ideas are not what produces its results.
- Granularity as a stability–plasticity knob is switched off in every headline result. Tables 2 and 3 use , which is plain one-shot forecasting. Appendix A.6.3 states that this "is not the optimal chunk size". Where chunking is tested, it rarely helps:
- In the component ablation, and are never better than on any of the five cells (Table 5a)
- In the dedicated sweeps, the best improves MSE by 3–13% on three real cells, even with chosen on the test stream (Table 16). On synthetic streams, the gain is 1–2%, and it appears only at large learning rates
- Vector-symbolic components add little:
- Removing the frozen matrix changes MSE by 0.7–6% on three cells (Table 5a)
- changes MSE by 0.6–8%, and its effect on FR goes in both directions (Table 5b)
- , smaller than the input window , is within 1% of , or better, on all eight cells (Table 13). This contradicts the high-dimensional, quasi-orthogonality rationale of Section 2.4
- The "short-term" and "long-term" losses (, ) are the same MSE on the same chunk, with the same
- Granularity as a stability–plasticity knob is switched off in every headline result. Tables 2 and 3 use , which is plain one-shot forecasting. Appendix A.6.3 states that this "is not the optimal chunk size". Where chunking is tested, it rarely helps:
Questions for authors
- Table 17's SPOT-VSA MSE differs from Table 15's by up to (ETTm2 336: 1.705 vs 4.131), although Appendix A.7.4 says the two may differ "slightly". Which settings differ, and which numbers should readers rely on?
- Apart from weight decay, what distinguishes from the other parameters? Section 2.4 says that making trainable induces forgetting (Appendix A.6.5), but no trainable- results are reported. Please add them
- Does the advantage in Table 2 persist when the baselines (at least DSOF, LSTD, OneNet, ER/DER++) get the same instance normalization, look-back () and validation-tuned , and SPOT-VSA uses on ECL?
- How do the headline results change with selected on validation instead of ? How does SPOT-VSA compare with RLinear/NLinear (same normalization, validation-tuned )? Since is Frobenius-normalized (Listing 1), pre-activations start near zero, where SiLU is nearly linear. Do they stay there after warm-up?
- Do Theorems 2.2 and 2.3, derived for plain gradient descent, still hold under AdamW, which all experiments use? And why does stability worsen with on synthetic streams (Figure 4) but improve with on real ones (Figure 5a)?