- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes M²TSF, an LLM-based time series forecasting method that replaces conventional single-stream multi-scale modeling with a multi-stream architecture. Each temporal scale is processed independently, while a constrained intermediate interaction module enables progressive cross-scale communication aligned with LLM depth. A pattern-aware fusion module further uses a hyperedge-centric MoE mechanism to select and combine informative temporal patterns.
Critical strengths
S1: The paper identifies a clear limitation of existing multi-scale LLM forecasting methods, and propose an effective method.
S2: Code is released.
S3: Experiments on multiple datasets show its effectiveness.
Critical weaknesses
W1: It's not clear whether performance gain is from LLM. The model makes limited use of explicit time-series semantics, and the current experiments do not isolate the benefit of LLM pretraining. A comparison with an identically structured Transformer trained from random initialization would help.
W2: The multi-stream formulation appears to reuse the same frozen Transformer parameters across all streams, with only the inputs differing.
Questions for authors
Q1: Tha author assumes that shallow LLM layers correspond to fine-grained temporal patterns, while deeper layers correspond to coarse-grained semantics. Is this true ans is there papers or experimental results supporting it? More importantly, evidence from probing language representations does not directly imply that the same layer-wise hierarchy remains valid after numerical time-series embeddings are fed into the LLM.
4 comments
Q1: Justify the assumption that shallow LLM layers capture fine-grained temporal patterns while deeper layers capture coarse-grained semantics, and provide experimental evidence that this layer-wise hierarchy remains valid when numerical time-series embeddings are fed into the LLM.
Probing studies in NLP suggest that lower layers of pre-trained LLMs tend to encode local features (e.g., lexical and part-of-speech information), middle layers capture syntactic structure, and higher layers reflect more abstract, global semantic features [1, 2]. Similar observations have also been reported in LLM-based time-series forecasting methods [3–5]. For example, Logo-LLM visualizes the cosine similarity of hidden representations across six Transformer layers and finds that shallow LLM layers attend primarily to local temporal patterns, whereas deeper layers focus on global, coarse-grained temporal patterns.
Motivated by these findings, M$^2$TSF adopts a fine-to-coarse alignment between layer depth and temporal scale. As reported in Table 13 of the , both reversing this alignment to coarse-to-fine and applying uniform scale weights yield worse forecasting performance than M$^2$TSF. These results provide indirect empirical support for our alignment strategy.
[1] Jawahar G, Sagot B, Seddah D. What does BERT learn about the structure of language? ACL, 2019.
[2] Skean O, Arefin M R, Zhao D, et al. Layer by layer: Uncovering hidden representations in language models. ICML, 2025.
[3] Zhang F, Fan S, Wang H. TimeSAF: Towards LLM-guided semantic asynchronous fusion for time series forecasting. ACL, 2026.
[4] Liu P, Guo H, Dai T, et al. CALF: Aligning LLMs for time series forecasting via cross-modal fine-tuning. AAAI, 2025.
[5] Ou W, Zhao Z, Chen C, et al. Logo-LLM: local and global modeling with large language models for time series forecasting. arXiv, 2025.
To further investigate whether the layer-wise hierarchy observed in NLP remains valid when numerical time-series embeddings are fed into the LLM, we conduct a probing experiment on ETTh1 dataset with a prediction horizon of 96. Specifically, we first apply the Fast Fourier Transform (FFT) to each input sequence and partition the spectrum into three frequency bands, from which the trend (low-frequency, coarse-grained), seasonal (mid-frequency), and residual (high-frequency, fine-grained) components are reconstructed via the inverse FFT. We then keep the backbone parameters frozen and train an independent linear regression probe for each layer to reconstruct the corresponding component from that layer’s hidden representations. Finally, we compute the score of each probe to measure the reconstruction ability, where R² is one minus the ratio of the residual sum of squares to the total sum of squares. A higher indicates that the corresponding temporal component is more linearly recoverable.
The visualization results are shown in Figure 4 of the . The solid lines denote the probing results on the pre-trained LLM backbone, while the dashed lines denote those on a randomly initialized backbone of the same architecture. We can observe that: (1) The values of the residual component decrease as depth increases, indicating that fine-grained, local temporal features are most linearly accessible at shallow layers and are progressively abstracted away as depth increases. (2) In contrast, the values of the trend component increase as depth increases, showing that coarse-grained, global temporal features become increasingly prominent at deeper layers, while the values of the seasonal component peak at intermediate layers, consistent with its mid-granularity nature. (3) All values for the randomly initialized architecture remain flat and low across layers, confirming that the layer-wise hierarchy indeed remains valid for numerical time-series inputs, and that it originates from the learned representations rather than from the architecture itself or the probe's fitting capacity.
Thank you for your reply. Most of my questions are solved. I will raise my score to 3.
Dear Reviewer wFVy,
Thank you for your thoughtful feedback. We are pleased that our response has addressed most of your concerns. All of your comments, including assessing whether the performance gains stem from LLM pretraining, adding visualization analysis, and clarifying the detailed experimental settings, have been included in the .
Thanks again for the time and effort you dedicated to reviewing our paper.
Best regards,
The Authors
W1: Comparison with an identically structured Transformer trained from random initialization to assess whether the performance gains stem from LLM pretraining.
Thanks for your valuable feedback and scientific rigor. To further investigate the effectiveness of LLM pretraining, we conduct ablation studies by carefully designing the following two variants:
-w/o LLM: Removing the LLM and directly using the numerical embeddings as multi-stream representations.
-RanInit: Replacing the LLM with a randomly initialized Transformer of identical architecture and depth.
Table 1
The experimental results on ETTh1 dataset are shown in Table 1, which has also been included in Appendix F of the revised paper. We can observe that: (1) -w/o LLM performs worst, suggesting that directly using numerical embeddings as multi-scale representations may provide insufficient contextual information and introduce noise interference. (2) M$^2$TSF outperforms -RanInit despite sharing the same architecture, confirming that the pre-trained semantic knowledge is the genuine driver of performance.
W2: The multi-stream formulation appears to reuse the same frozen Transformer parameters across all streams, with only the inputs differing.
Thank you for the valuable comment. You are correct that all streams share the same frozen LLM backbone while receiving scale-specific token representations as inputs. However, this does not mean that all parameters are frozen. After every stage, CII models higher-order interactions among scale-specific representations through a hypergraph structure and progressively controls cross-scale information exchange. This design allows the streams to retain their scale-specific representations while exchanging information at different stages. The progressive interaction is designed to align cross-scale information exchange with the hierarchical representations learned at different depths of the LLM. The parameters of the hypergraph structures, together with the soft prompts and projection layers, remain trainable.