acceptodds
Under review as a conference paper at ICLR 2027

SFCATime: Soft Spectral Representation and Adaptive Cross-Modal Fusion for Time Series Forecasting

Abstract

Multivariate time-series forecasting can benefit from complementary frequency-domain and language-derived representations, yet their effectiveness depends on how frequency structure is organized and how semantic information interacts with numerical features. A shared spectral projection leaves frequency specialization to downstream encoders, while standard cross-modal attention relies primarily on query–key affinity to determine numerical–semantic interactions. We propose SFCATime, an adaptive spectral–semantic forecasting framework that addresses both issues. First, SFCATime learns soft, overlapping assignments of frequency bins to latent spectral components and uses them to construct frequency-dependent corrections to a retained full-spectrum pathway. This preserves the complete magnitude spectrum while introducing spectral specialization without hard frequency boundaries. Second,AdaptiveMask Cross-Modal Attention augments base numerical–semantic affinity with a learnable signed relation correction before attention normalization, while adaptive branch aggregation and residual modulation separately control the contribution of the aligned representation. The pretrained language model remains frozen throughout forecasting training. Across eight widely used multivariate forecasting benchmarks, SFCATime achieves the best or tied-best average MAE on all eight datasets and the best average MSE on six. Compared with the most directly related multimodal baseline, it obtains lower average MSE and MAE on seven datasets each. Ablation studies and representation analyses further support the complementary effects of adaptive spectral representation and relation-aware cross-modal regulation.

Primary area:
learning on time series and dynamical systems
Keywords:
multivariate time series forecasting, soft spectral representation, cross-modal alignment, adaptive attention, pretrained language models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_g5TeOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes SFCATime, a spectral–semantic framework for multivariate time-series forecasting. It introduces soft spectral assignments for frequency-dependent representations and AdaptiveMask attention to regulate interactions between numerical and language-derived features. Experiments on eight benchmark datasets report competitive MSE and MAE results, supported by ablation studies and visualizations. The overall design is interesting, but I have serious concerns about the evaluation protocol and the reliability of some reported results.

Critical strengths
  1. The paper addresses an interesting forecasting problem. Combining spectral representations with semantic information is a reasonable direction.

  2. I appreciate the adaptive design of the spectral assignments and cross-modal attention. The two components have clear motivations and are relatively easy to understand.

  3. The evaluation covers eight datasets, with encouraging forecasting results. The ablations and visualizations also provide some useful insights into the proposed modules.

Critical weaknesses
  1. The evaluation protocol is a major concern. Appendix A.2 explicitly states that test MSE is used for hyperparameter optimization and checkpoint selection. This makes the reported test results unreliable as independent generalization evidence.

  2. I noticed inconsistencies in Table 1. The ETTm2 UniTime averages do not match the reported values, and the ILI iTransformer MSE–MAE pair appears mathematically inconsistent.

  3. Novelty is somewhat limited. The framework largely extends T3Time, while adaptive frequency modeling and cross-modal attention have already been explored. I am not fully convinced that the proposed modifications represent a substantial methodological advance.

  4. The contribution of individual mechanisms is not sufficiently isolated. In particular, the AdaptiveMask ablation removes multiple operations together, making their separate effects difficult to judge.

Questions for authors
  1. Why was test MSE used for checkpoint and hyperparameter selection? Could the authors rerun the experiments using validation-only selection and report the updated results?

  2. Could the authors clarify the numerical inconsistencies in Table 1, particularly the ETTm2 averages and the ILI iTransformer MSE–MAE values?

  3. What is the main methodological difference between SFCATime and T3Time? Could simpler frequency-adaptive projections achieve similar improvements?

  4. Could the authors separately ablate the signed attention correction and residual modulation? I would also like to see whether the language representations provide meaningful improvements over simpler alternatives.

Reviewer_mQaFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes SFCATime, aiming to improve the forecasting performance when incorporating multi-modality information, including spectral, temporal, and textual info. The authors aim to enhance the framework from two aspects, how to structure each view and how to capture the relations between views. Specifically, it introduces soft frequency-band assignments for the former while AdaptiveMask to adjust numerical–semantic attention scores for the latter. Experiments on eight benchmarks show competitive forecasting performance, with ablation studies examining the proposed components.

Critical strengths

I think the authors present a complete framework with reasonably extensive experiments

Critical weaknesses

My main concern is the limited novelty and depth of the contributions. The method largely follows an existing backbone and introduces two module-level refinements. Meanwhile, the presentation needs to be improved. It seems the authors present large motivations but did not fully address them. Details are as follows.

  1. The work appears incremental at both the motivation and contribution levels. The motivations concern local limitations within an existing forecasting framework, while the main contributions are a soft spectral correction and an AdaptiveMask correction to cross-modal attention. I think the overall framework largely follows the existing backbone, and the proposed changes appear to be module-level refinements.

  2. The presentation needs to be improved. a. The introduction is not well organized, and the scope of the motivations is unclear. The authors mention “how each view should be structured,” but only discuss limitations of spectral representations. Why is the spectral view singled out? The motivation is too large. b. The second motivation discusses 'how information ... across them', but then only focuses on numerical–semantic alignment, i.e., the view between the textual view and others. Still, the motivation is too large. c. Figure 1 is not informative enough. It would be better to clearly highlight the proposed changes compared with the existing framework, particularly T3Time, and distinguish them from inherited components. d. In Section 3.1, the authors should distinguish between the problem formulation and the overview of the proposed framework.

Questions for authors

See above

Reviewer_4UZJOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

SFCATime extends T3Time with soft frequency-dependent spectral projections and signed corrections to cross-modal attention, using frozen GPT-2 representations of historical observations. Experiments cover eight forecasting benchmarks, reporting best or tied-best average MAE on all eight and best average MSE on six. Component ablations and representation visualizations provide additional analysis.

Critical strengths

The spectral design has a clear structural motivation: the paper shows that a position-agnostic spectral encoder with invariant pooling cannot distinguish permutations of frequency magnitudes. SSC introduces frequency-dependent projections that can break this symmetry while retaining all magnitude bins. This provides a principled motivation for the proposed modification and identifies a concrete limitation of the inherited backbone.

Critical weaknesses
  1. Test-set-driven model selection. Appendix A.2 (p. 12, lines 641–646) explicitly states that hyperparameter tuning minimizes test MSE and that test performance influences checkpoint selection after epoch ten. This compromises the independence of the evaluation, so the reported improvements cannot establish generalization gains. Validation-only model selection followed by independent testing is necessary.
  2. Configuration-dependent test coverage. Appendix A.1 (p. 12, lines 618–622) uses drop_last=True for testing, while batch size varies across configurations and is sometimes tuned. Consequently, configurations can be evaluated on different subsets of test windows, confounding model quality with evaluation coverage. Reliable comparisons require all methods to use the same complete test-window set.
Questions for authors

Can you provide results comparing SFCATime with T3Time using validation-only hyperparameter and checkpoint selection, with both methods evaluated on the same complete test-window set?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.