acceptodds
Withdrawn from ICLR 2027

SSVT: Industrial Multivariate Time Series Forecasting with Hybrid State Space and Variate-Token Attention

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

As industrial systems operate at higher sampling rates and deploy increasingly dense sensor networks, multivariate time series forecasting in industrial settings faces two central challenges, namely slow cross-cycle drift that accumulates over long temporal spans and abrupt local fluctuations across variables. To address these challenges, we propose SSVT, a new paradigm for industrial multivariate time series forecasting that integrates hybrid state-space modeling with variate-token attention. Specifically, a temporal state-space encoder extracts cross-cycle historical dynamics for each variable and maps the resulting state representation into a corresponding variate token, allowing the tokens to retain long-term drift information along the temporal dimension. Variate-token attention then propagates information across variables and models local inter-variable dependencies, enabling SSVT to jointly capture long-term trends and short-term fluctuations. Building on this architecture, we further introduce physical constraints into the training process and use an LLM to assist in configuring the constraint weights. This design provides a more effective balance between data-driven learning and physical priors, improving forecasting stability and generalization in complex industrial environments. Experiments on six datasets show that the advantage of SSVT becomes increasingly pronounced as the forecasting horizon extends. In the 720-step forecasting task, SSVT achieves the best MSE and MAE across all six datasets. On the Boiler Equipment dataset, SSVT obtains an average MSE of 0.395, reducing the error by 6.95% compared with the strongest baseline. These results indicate that SSVT offers an effective paradigm for modeling cross-cycle slow drift and local variable dependencies in industrial time series. The code and experimental configurations will be made publicly available at https://github.com/CollaborativeResearch/SSVT.

Primary area:
learning on time series and dynamical systems
Keywords:
Industrial Multivariate Time Series Forecasting, Hybrid State-Space Models, Variate-Token Attention.

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_AFPLOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

SSVT runs a Mamba block over each variable and feeds the last hidden states, one token per variable, to an iTransformer-style encoder. Optional physics penalties are added to the loss, with weights proposed by an LLM. On five public benchmarks and one boiler dataset, the paper claims the best results at H = 720 and an advantage that grows with the horizon.

Critical strengths
  1. The model is small (about 131k131k parameters) and fast.
Critical weaknesses
  1. Little is new, and the advertised extras are not in the main results. SSVT is a standard Mamba encoder per variable (Sec. 3.2 says the SSM is not redesigned) followed by an iTransformer encoder, and no Mamba-based forecaster is compared. The boiler results in Table 1 (0.102 at H = 96, 0.648 at H = 720) equal the unconstrained results in Sec. 4.4, so the physics penalties and the LLM search play no role. Where they are used, they raise the smoothness violation at H = 720 (0.0715 to 0.0945), and the unnamed LLM is worse than manual tuning (0.1001 vs 0.0984).

  2. The setup cannot test the stated problem. The paper is about slow drift from equipment aging, but the input window is 96 steps (App. A.2): 8 minutes of history on the boiler data, which itself covers only five days (App. A.1). No experiment isolates drift.

  3. The results contradict the claims and are not controlled. The advantage shrinks with the horizon: on ETTh1 the margin is 16% at H = 96 (0.306 vs 0.366) but 0.4% at H = 720 (0.457 vs 0.459). The H = 720 "wins on all six datasets" are 0.001 to 0.008 MSE from a single seed, and Table 1 mixes re-run and copied baseline numbers without marking them (App. B.3). In Table 2, three different variants give the same boiler error (0.780, 0.779, 0.779).

  4. The code link breaks anonymity. The abstract links to a public GitHub repository under a named account. Even if it is currently an empty placeholder with no identifying information, such a link can reveal the authors at any time and should not appear in a double-blind submission. Anonymized links exist for this purpose.

Questions for authors

See weaknesses.

Reviewer_KFhdOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SSVT for industrial multivariate time series forecasting. It combines a Mamba3-based temporal encoder with variate-token attention to model temporal dynamics and cross-variable dependencies. The method also introduces soft industrial constraints and LLM-assisted tuning of constraint weights. Experiments are conducted on six datasets using standard forecasting metrics.

Critical strengths

The architecture is straightforward and easy to follow. The reported results show competitive forecasting performance, particularly at longer prediction horizons.

Critical weaknesses
  1. The paper focuses on cross-cycle slow drift and abrupt local fluctuations, but the experiments do not directly examine either phenomenon. All experiments use an input length of 96, which corresponds to only eight minutes on the Boiler dataset. It is unclear how these experiments support the claims about long-term drift.

  2. The method mainly combines Mamba3 with standard variate-token attention. Both components are existing techniques, and the paper offers limited novelty beyond their combination.

  3. The ablation study only covers two datasets at the 720-step horizon. While removing either module degrades performance, the results do not show whether TSSE actually captures slow drift or VTA captures local fluctuations.

  4. The role of the LLM is questionable. It is only used to suggest constraint weights, and the resulting configuration performs worse than manual tuning. No comparison with conventional hyperparameter optimization methods is provided.

  5. Five of the six datasets are standard forecasting benchmarks, and only one comes from industrial equipment. The evaluation does not sufficiently reflect the industrial scenarios emphasized throughout the paper.

  6. The figures could be improved.

  7. The GitHub repository linked in the paper is empty, so the code and experimental configurations are not available for reproduction.

Questions for authors
  1. What evidence shows that TSSE captures cross-cycle slow drift and VTA captures abrupt local fluctuations, rather than simply improving overall forecasting accuracy?
  2. What is the main methodological novelty compared with directly combining Mamba3 and iTransformer-style attention?
  3. Why is an LLM needed for constraint-weight tuning when simpler search methods are available?
  4. Can the authors provide the complete code and configurations needed to reproduce the results?
Reviewer_iKx7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SSVT, which employs a state-space encoder to extract temporal representations for individual variables, models inter-variable relationships via variable-level attention, and incorporates soft physical constraints alongside LLM-assisted weight tuning. The authors evaluate the model across six datasets and four prediction horizons, reporting strong performance in long-term forecasting.

Critical strengths
  1. The architecture is clearly designed, separating temporal modeling from variable interaction to avoid the computational burden of full-sequence self-attention.
  2. The controlled ablation study in Table 2 supports the complementary nature of the two modules, and the results across multiple datasets demonstrate practical potential.
Critical weaknesses
  1. While Section 4.1 states that baselines follow a consistent data split, Appendix B.3 indicates a mix of results from various literature sources. Given the discussion on original-scale evaluation in Appendix A.3, the sources and scales of the results in Table 1 require clarification to ensure a fair comparison.
  2. Appendix A.2 specifies a fixed historical window of 96 time steps—which, for the boiler dataset, corresponds to only about 8 minutes. The absence of definitions for periodicity or mechanisms for state propagation across windows makes the current experimental setup insufficient to support the core claim regarding the "modeling of slow drifts across cycles."
Questions for authors
  1. Appendix B.1 relies on a single random seed, and for the 720-step prediction on the ETTm1 dataset, the MSE improvement is marginal (from 0.435 to 0.434). Could you provide the mean and standard deviation across multiple seeds to confirm the stability of these performance gains?
  2. Appendix F mentions that an LLM-based candidate selection step was introduced only after five manual trials. Does the "reduction in search cost" account for these preliminary trials? Please clarify the total computational/experimental budget and specify the LLM model and prompts used.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.