acceptodds
Under review as a conference paper at ICLR 2027

SIFT: Enhancing Time Series Foundation Models via Semantic Invariance and Structural Fidelity Fine-Tuning

Abstract

Time Series Foundation Models (TSFMs) have achieved remarkable zero-shot performance through extensive pre-training on massive time series datasets. Nevertheless, due to the low-dimensional properties and diverse structural patterns of time series data, performing naive fine-tuning on TSFMs often leads to overfitting and falling into the mean-prediction trap. To address these challenges, we propose SIFT, a robust adaptation method that enhances time series foundation models by preserving Semantic Invariance and structural Fidelity throughout the fine-Tuning process. We employ semantic-invariant adversarial augmentation, which utilizes semantic spectrum decomposition to partition the semantic space and then generates perturbations within the non-core semantic subspace to bolster the model’s robustness against these perturbations, mitigating overfitting. We implement a component-based structural fidelity enhancement, which facilitates component-wise mixup and imposes a reconstruction objective to improve the model's ability to preserve structural fidelity, alleviating the mean-prediction trap. Extensive experiments on representative TSFMs covering 10 real-world datasets demonstrate that SIFT can significantly enhance the performance of TSFMs.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
time series foundation models, model adaptation, time series forecasting, transfer learning, temporal representation learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_EdbkOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SIFT to address two problems in downstream fine-tuning of time series foundation models (TSFMs): overfitting and overly smooth predictions. The method comprises two modules. First, Semantic-Invariant Adversarial Augmentation (SIAA) uses SVD of the trajectory matrix to partition the spectrum into core, subordinate, and noise subspaces, generates adversarial perturbations within the subordinate semantic subspace, and regularizes the model through prediction consistency. Second, Component-based Structural Fidelity Enhancement (CSFE) selects high-volatility training samples, cross-pairs their trend and seasonal components, and preserves temporal structure through a component-wise fidelity reconstruction loss.

The experiments cover 10 datasets, four foundation models, and four prediction horizons. The paper reports results averaged over multiple independent runs and provides module ablations, parameter sensitivity analyses, and spectral component visualizations. The results demonstrate that SIFT improves forecasting performance across multiple backbones without introducing additional computation during inference.

Critical strengths

1. The experimental evaluation across backbones is rigorous and solid.

The paper evaluates Timer, UniTS, Moirai, and Sundial, rather than demonstrating results on a single model. In Table 1, Timer’s average MSE decreases from 0.405 under full fine-tuning to 0.357, while the other three backbones also achieve improvements in average MSE. These results indicate that SIFT has meaningful general applicability to fine-tuning time series foundation models.

2. The two modules have clearly complementary designs and introduce no additional inference computation.

By design, SIAA addresses sensitivity to input perturbations, whereas CSFE addresses the loss of structural details. They provide consistency regularization and structural supervision, respectively, during training. Both modules target practical problems in downstream fine-tuning of TSFMs and offer promising solutions. The ablation study also demonstrates their effectiveness.

Critical weaknesses

1. The central claim of “semantic invariance” is not sufficiently supported by the mathematical construction or empirical evidence.

The Eckart–Young–Mirsky theorem guarantees the best low-rank approximation, and it does not guarantee that high-energy directions correspond to the core semantics required for forecasting, nor that intermediate-rank components represent spurious patterns that should be ignored. Although the cross-domain affinity analysis in the appendix provides evidence of association, it does not establish that the forecasting target should remain unchanged when these components are perturbed. Should additional theoretical justification or a formal proof be provided?

Furthermore, Eq. (5) first projects the perturbation in the trajectory-matrix space and then reconstructs it through anti-diagonal averaging. After the resulting one-dimensional perturbation is re-embedded into a trajectory matrix, it is not generally guaranteed to remain orthogonal to the core subspace. However, Eq. (6) penalizes the core projection of the raw perturbation, rather than the core leakage of the final reconstructed perturbation. Eq. (6) also does not explicitly constrain the perturbation magnitude within the subordinate subspace. These issues directly affect whether consistency training operates on valid inputs that preserve the target semantics, and therefore constitute a fundamental methodological concern.

2. The experiments do not sufficiently distinguish the contributions of the proposed mechanisms from those of generic augmentation and additional supervision.

Table 2 provides useful ablations, but it lacks a direct forecasting-performance comparison between random and learned perturbations within the same subordinate subspace and at the same perturbation magnitude. It also does not separately isolate the contributions of high-volatility sample selection, structural component mixup, and component-wise fidelity reconstruction. Consequently, the current results do not sufficiently establish whether the complexity of the complete method is necessary.

Moreover, although the two modules are complementary by design and each individually provides clear improvements, the additional gains from combining them are only on the order of a few thousandths in the reported MSE values. The stability of these incremental gains needs to be confirmed. This limits the assessment of the mechanisms’ contributions and the magnitude of their benefits.

3. The method introduces no additional inference computation, but its training costs are not reported. Although the paper states that inference requires no additional computation, it does not report whether training costs increase relative to PEFT methods such as LoRA, including the number of trainable parameters and GPU memory usage. These are also important metrics for evaluating downstream fine-tuning of pretrained models.

Questions for authors
  1. Does generator training use a perturbation norm constraint, output clipping, or normalization? Please provide the complete training objective and perturbation magnitude settings, and explain how the penalty in Eq. (6) controls core-subspace leakage in the final reconstructed perturbation, rather than merely the core projection of the raw perturbation.

  2. In the actual experiments, is r2r_2 determined by the fixed ratio ρ2\rho_2, or by the Marchenko–Pastur (MP) threshold? What are the specific configurations for the trajectory embedding window width and stride, and how are projection matrices shared across multivariate channels?

  3. Under the same subspace, perturbation magnitude, and training budget, do learned perturbations outperform random perturbations? For CSFE, how much improvement is attributable separately to high-volatility sample selection, structural component mixup, and component-wise fidelity reconstruction?

  4. Please explain why several LoRA configurations in Table 1 exhibit substantial performance degradation, and provide the hyperparameter search ranges and training compute budgets for each baseline.

  5. Can you explain why the empirical gains from combining the two complementary modules are less pronounced than the preceding discussion suggests?

7 comments

Thank you for your suggestion. We try to address your concerns in the following ways:

For W1 and Q1. Theoretical and Empirical Justification of Semantic Invariance

We provide additional theoretical clarification and empirical analyses regarding the interpretation of spectral subspaces and the motivation for subordinate-space perturbations.

1. Semantic Interpretation and Predictive Relevance.

1) Theoretical Motivation of Spectral Subspace Decomposition.

Our justification therefore draws on complementary findings from spectral transferability and classical SSA, together with a norm-based analysis of prediction sensitivity.

Our spectral decomposition is motivated by prior findings on spectral transferability. **Previous studies [1] suggest that dominant spectral components tend to exhibit stronger transferability, whereas weaker components may encode less transferable variations. **Although these findings do not directly establish the transferability of spectral components in raw time series, they motivate us to investigate whether different temporal spectral components exhibit distinct structural and predictive characteristics.

To this end, we introduce classical Singular Spectrum Analysis (SSA) to characterize temporal representations through the singular spectrum of Hankel trajectory matrices [2]. Given a time series XX, its trajectory matrix admits the decomposition H(X)=∑i=1dσiuivi⊤\mathcal H(X)=\sum_{i=1}^{d}\sigma_i u_i v_i^\top, where σ1≥σ2≥⋯≥σd≥0\sigma_1\geq\sigma_2\geq\cdots\geq\sigma_d\geq0. We denote the ii-th elementary spectral component as Hi=σiuivi⊤\mathcal H_i=\sigma_i u_i v_i^\top, whose energy is ∣Hi∣F2=σi2|\mathcal H_i|_F^2=\sigma_i^2. Based on this spectral ordering, we define the leading eigentriples (i≤r1i\leq r_1) as the core subspace and the intermediate eigentriples (r1<i≤r2r_1<i\leq r_2) as the subordinate subspace.

The structural interpretation of these spectral components is supported by classical SSA theory, particularly the concepts of separability and spectral dominance [2, 3]. When an underlying temporal component admits a relatively low-rank representation, dominates the residual components in energy, and is approximately separable from other components, its energy tends to concentrate in a small number of leading eigentriples. Consequently, these leading components tend to capture dominant temporal representations, such as long-term trends and long-period oscillations. In contrast, intermediate eigentriples retain non-dominant temporal variations. Importantly, we do not assume that these intermediate components are purely noise or entirely devoid of predictive information; rather, they may encode sample-specific or domain-dependent variations with relatively weaker structural dominance.

We further characterize the potential influence of individual spectral components on forecasting through a norm-based bound. Let Xi=Ψ−1(Hi)X_i=\Psi^{-1}(\mathcal H_i) denote the reconstructed time-series component of the ii-th eigentriple, where Ψ−1\Psi^{-1} performs anti-diagonal averaging. Since each reconstructed entry is obtained by averaging the corresponding entries along an anti-diagonal, this reconstruction operator is non-expansive with respect to the Frobenius and Euclidean norms, yielding ∣Xi∣2≤∣Hi∣F=σi|X_i|_2\leq|\mathcal H_i|_F=\sigma_i. For a linear predictor f(X)=WX+bf(X)=WX+b, the prediction difference induced by removing the ii-th component satisfies ∣f(X)−f(X−Xi)∣2=∣WXi∣2≤∣W∣2∣Xi∣2≤∣W∣2σi|f(X)-f(X-X_i)|_2=|WX_i|_2\leq|W|_2|X_i|_2\leq|W|_2\sigma_i. More generally, for a nonlinear forecaster that is locally LfL_f-Lipschitz along the segment connecting XX and X−XiX-X_i, we obtain ∣f(X)−f(X−Xi)∣2≤Lf∣Xi∣2≤Lfσi|f(X)-f(X-X_i)|_2\leq L_f|X_i|_2\leq L_f\sigma_i. These bounds establish an energy-dependent upper bound on the change in model predictions induced by removing an individual reconstructed spectral component. These bounds establish a connection between spectral energy and the potential impact of individual components on forecasting, providing a mathematical justification for preserving dominant spectral representations during fine-tuning.

Taken together, prior findings on spectral transferability, the interpretation provided by SSA, and the above perturbation bounds motivate our spectral decomposition. The core subspace captures dominant temporal representations to be preserved, whereas the subordinate subspace contains non-dominant variations that may be less stable or transferable. Accordingly, SIAA applies adversarial regularization within the subordinate subspace to encourage prediction consistency under non-core variations while preserving dominant temporal representations, whose predictive relevance is further examined through empirical analyses.

[1] Chen, Xinyang, et al. "Catastrophic forgetting meets negative transfer: Batch spectral shrinkage for safe transfer learning." Advances in neural information processing systems 32 (2019).

[2] Golyandina, Nina. "Particularities and commonalities of singular spectrum analysis as a method of time series analysis and signal processing." Wiley Interdisciplinary Reviews: Computational Statistics 12.4 (2020): e1487.

[3] Golyandina, Nina, Vladimir Nekrutkin, and Anatoly A. Zhigljavsky. Analysis of time series structure: SSA and related techniques. CRC press, 2001.

2) Analysis of Predictive Information in Different Subspaces.

To investigate whether the spectral distinction is reflected in future-sequence similarity, we construct three groups of naturally occurring sample pairs based on the pairwise distances between their corresponding subspace representations: (A) similar core and similar subordinate, (B) similar core but different subordinate, and (C) different core but similar subordinate. For each group, we report the mean pairwise distances within the core and subordinate subspaces, together with the corresponding ground-truth future distances (dyd_y).

DatasetA: Similar Core & Similar Sub (Core / Sub / Future)B: Similar Core & Different Sub (Core / Sub / Future)C: Different Core & Similar Sub (Core / Sub / Future)
ETTh10.755 / 0.705 / 0.8210.760 / 1.233 / 0.9831.265 / 0.804 / 2.708
ETTm10.816 / 0.728 / 0.6990.884 / 1.262 / 0.7961.266 / 0.786 / 2.021
Exchange0.805 / 0.906 / 0.4470.713 / 1.365 / 0.4601.462 / 0.842 / 2.388
ZafNoo0.818 / 0.727 / 0.9430.773 / 1.182 / 1.1861.183 / 0.813 / 2.035
Mean Future Distance0.7280.8562.288

Across all four datasets, future distances consistently follow dy(A)<dy(B)<dy(C)d_y(A)<d_y(B)<d_y(C). When core distances remain comparable (A vs. B), increasing subordinate distances leads to only modest increases in future distances. In contrast, larger core distances (A vs. C) are accompanied by substantially greater future differences. These results suggest that core representations exhibit stronger associations with future patterns, while subordinate representations provide comparatively weaker predictive discrimination.

3) Analysis of Predictive Contributions from Different Spectral Subspaces.

To further examine the predictive contributions of different spectral subspaces, we compare forecasting performance using the full input, core-only components, and core+sub components under the same forecasting model and evaluation protocol.

DatasetFull MSECore-only MSECore+Sub MSE
ETTh10.4500.4620.464
Exchange0.3960.3710.384
ZafNoo0.5230.5290.540
Weather0.2450.2520.247

The results show that core-only inputs retain forecasting performance comparable to that of full inputs across all four datasets, indicating that the core subspace preserves most of the predictive capability. Moreover, incorporating subordinate components does not yield consistent improvements. These findings suggest that the subordinate subspace may still contain some predictive information, but can also encode spurious or non-transferable variations. Our objective is therefore not to discard these components, but to improve the model's robustness to perturbations within this subspace while preserving its core semantics.

Overall, the SVD-based semantic spectrum decomposition and complementary predictive analyses support the distinction between core and subordinate subspaces. The core subspace exhibits stronger associations with future patterns, whereas subordinate components retain structured but comparatively less predictive-discriminative variations. This provides a theoretically motivated and empirically supported basis for our spectral decomposition.

We have added the theoretical motivation in Section 3.1 and predictive-relevance analyses and predictive contributions in Appendix A.6 and A.7.

2. Mathematical Construction and Perturbation Constraints.

For perturbation magnitude control, our implementation bounds the perturbation magnitude through tanh-based scaling, with a maximum element-wise magnitude of 0.3 times the input's temporal standard deviation, preventing excessively large perturbations.

For generator objective, our objective is to guide perturbations toward the subordinate subspace, rather than enforce strict orthogonality to the core subspace. Specifically, the subordinate projection in Eq. (5) directs perturbation generation toward intermediate spectral components, while the core penalty in Eq. (6) discourages excessive core components in the raw generator output. We adopt this soft guidance to retain flexibility in perturbation generation, rather than imposing a restrictive orthogonality constraint on the final reconstructed perturbations. Consequently, some core leakage may arise after anti-diagonal averaging and re-embedding.

To quantify the spectral energy distribution of perturbations, we define the Core Leakage Ratio (CLR) and Subordinate Energy Ratio (SER) as: CLR(δ)=∣∣UcoreUcore⊤Ψ(δ)∣∣F2∣∣Ψ(δ)∣∣F2+ϵ,\mathrm{CLR}(\delta)=\frac{||U_{\mathrm{core}}U_{\mathrm{core}}^\top\Psi(\delta)||_F^2}{||\Psi(\delta)||_F^2+\epsilon}, SER(δ)=∣∣UsubUsub⊤Ψ(δ)∣∣F2∣∣Ψ(δ)∣∣F2+ϵ,\mathrm{SER}(\delta)=\frac{||U_{\mathrm{sub}}U_{\mathrm{sub}}^\top\Psi(\delta)||_F^2}{||\Psi(\delta)||_F^2+\epsilon}, where ϵ\epsilon is a small numerical constant. CLR measures the proportion of perturbation energy projected onto the Core subspace, while SER measures the proportion projected onto the Subordinate subspace. Here, Raw denotes the generator output, Projected denotes the perturbation after subordinate-space projection in the trajectory-matrix space, and Final denotes the reconstructed perturbation after anti-diagonal averaging.

DatasetRaw CLRProjected CLRFinal CLRRaw SERFinal SER
ETTh10.06599.11e-150.02610.59000.9173
ETTm10.09815.63e-150.02740.51940.8962
ZafNoo0.04098.28e-150.02620.58080.8975

As shown in the table, although the projected perturbations have negligible core leakage, the final perturbations exhibit small but non-zero leakage, with CLR remaining below 0.03 across all evaluated datasets. Meanwhile, the final Subordinate Energy Ratio (SER) ranges from 0.8962 to 0.9173. These results empirically suggest that the final perturbations predominantly preserve subordinate-subspace energy.

We describe the perturbation-generation procedure and its optimization objective, and further analyze the spectral energy distribution of generated perturbations at different stages in Appendix A.9 in the revised paper.

Reviewer_r3wQOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SIFT, a fine tuning framework for TSFMs that aims to mitigate overfitting and overly smooth forecasts issues. It combines SVD based adversarial augmentation to improve robustness with structural reconstruction using trend and seasonal components to preserve temporal patterns. Experiments on four pretrained models and ten datasets evaluate SIFT against several fine tuning baselines and show improvements in forecasting performance.

Critical strengths

S1. Relevant problem and targeted design

  • The paper addresses overfitting and overly smooth forecasts issues in TSFM fine tuning through a plug-in framework with two complementary components. It provides a practical approach to improving robustness and preserving temporal structures during fine tuning.

S2. Evaluation across multiple TSFMs

  • The evaluation covers four pretrained TSFMs and ten datasets, comparing SIFT with full fine tuning and several parameter efficient methods. The results show improvements across multiple model and dataset combinations.
Critical weaknesses

W1. Insufficient justification of semantic interpretation

  • The motivation of SIFT relies on distinguishing core and subordinate semantics in time series. However, the SVD based formulation partitions the signal by spectral structure without establishing a clear correspondence between the resulting subspaces and paper claimed semantic roles.

W2. Validity of cross sample reconstruction

  • CSFE combines the trend of one sample with the seasonal component of another and uses their respective future components as reconstruction targets. However, the validity of this cross sample supervision remains unclear, particularly when trend and seasonal patterns interact or the underlying dynamics are nonlinear.

W3. Limited empirical support for the claimed mechanisms

  • Although SIFT improves forecasting performance in several settings, the experiments provide limited evidence for the claimed mechanisms. For example, Figure 3 shows that learned perturbations alter predictions but does not establish their semantic invariance. The evaluation also lacks direct evidence that CSFE improves structural fidelity.
Questions for authors

Q1. Definition of core and subordinate semantics

  • How are core and subordinate semantics defined in this work? Why do the leading singular components correspond to core semantics, while the intermediate components represent subordinate semantics? Are there other factors beyond spectral energy that support these interpretations?

Q2. Empirical basis of the motivation

  • Are the visualizations in Figure 1 based on real data, controlled experiments, or illustrative examples? Can you provide the corresponding experimental settings and quantitative evidence to establish the prevalence of overfitting and overly smooth forecasts across different TSFMs and datasets?
11 comments

Thank you for your suggestion. We try to address your concerns in the following ways:

For W1/Q1. Definition and Justification of Semantic Subspaces

We further clarify the definitions of core, subordinate, and noise components and provide additional theoretical and empirical evidence to support their interpretations.

1) Theoretical Motivation and Semantic Definition.

We first clarify that our three subspaces are defined according to the ordered singular values of the Hankel trajectory matrix.

Given a time series XX, we perform singular value decomposition on its trajectory matrix, yielding H(X)=UΣV⊤=∑i=1dHi\mathcal H(X)=U\Sigma V^\top=\sum_{i=1}^{d}\mathcal H_i, where Hi=σiuivi⊤\mathcal H_i=\sigma_i u_i v_i^\top denotes the ii-th elementary spectral component, with σ1≥σ2≥⋯≥σd≥0\sigma_1\geq\sigma_2\geq\cdots\geq\sigma_d\geq0 and energy ∣Hi∣F2=σi2|\mathcal H_i|_F^2=\sigma_i^2. Based on the ordered singular spectrum, we define the leading eigentriples (i≤r1i\leq r_1) as the core subspace, the intermediate eigentriples (r1<i≤r2r_1<i\leq r_2) as the subordinate subspace, and the remaining low-energy components (i>r2i>r_2) as the noise subspace. The core subspace captures dominant semantic structures, typically corresponding to long-term trends and seasonal patterns, whereas the subordinate subspace retains less dominant variations that are more likely to encode spurious or non-transferable patterns.

Our decomposition is motivated by two complementary perspectives. First, prior studies on spectral transferability [1] suggest that dominant spectral components tend to exhibit stronger transferability, whereas weaker components may be more susceptible to negative transfer. Second, classical Singular Spectrum Analysis (SSA) [2, 3] provides a structural interpretation of the singular spectrum of Hankel trajectory matrices. Under suitable dominance and separability conditions, the energy of a low-rank temporal component tends to concentrate in a small number of leading eigentriples. Consequently, leading eigentriples tend to capture dominant temporal representations, such as long-term trends, while oscillatory patterns are often represented by pairs of eigentriples with similar singular values. Intermediate eigentriples, in contrast, retain non-core temporal variations, which may exhibit weaker stability and transferability.

We further characterize the potential influence of individual spectral components on forecasting through a norm-based bound. Let Xi=Ψ−1(Hi)X_i=\Psi^{-1}(\mathcal H_i) denote the reconstructed time-series component of the ii-th eigentriple, where Ψ−1\Psi^{-1} is the anti-diagonal averaging operator that reconstructs a time series from a trajectory matrix by averaging along its anti-diagonals. Since this reconstruction operator is non-expansive, we have ∣Xi∣2≤∣Hi∣F=σi|X_i|_2\leq|\mathcal H_i|_F=\sigma_i. For a linear predictor f(X)=WX+bf(X)=WX+b, the prediction difference induced by removing this component satisfies ∣f(X)−f(X−Xi)∣2≤∣W∣2σi|f(X)-f(X-X_i)|_2\leq|W|_2\sigma_i. More generally, for a nonlinear forecaster that is locally LfL_f-Lipschitz along the segment connecting XX and X−XiX-X_i, we obtain ∣f(X)−f(X−Xi)∣2≤Lfσi|f(X)-f(X-X_i)|_2\leq L_f\sigma_i. This bound suggests that higher-energy components which constitute the core subspace, are associated with potentially larger upper bounds on prediction variation. Accordingly, we encourage the model to focus on stable core semantics while maintaining prediction consistency under non-core variations.

Taken together, spectral transferability findings, the structural interpretation provided by SSA, and the above perturbation bounds motivate our spectral stratification. The core subspace captures dominant temporal representations that we aim to preserve, whereas the subordinate subspace retains structured intermediate variations that may be less stable or transferable. SIAA therefore applies adversarial regularization within the subordinate subspace to improve robustness to non-core variations while preserving dominant core representations.

[1] Chen, Xinyang, et al. "Catastrophic forgetting meets negative transfer: Batch spectral shrinkage for safe transfer learning." Advances in neural information processing systems 32 (2019).

[2] Golyandina, Nina. "Particularities and commonalities of singular spectrum analysis as a method of time series analysis and signal processing." Wiley Interdisciplinary Reviews: Computational Statistics 12.4 (2020): e1487.

[3] Golyandina, Nina, Vladimir Nekrutkin, and Anatoly A. Zhigljavsky. Analysis of time series structure: SSA and related techniques. CRC press, 2001.

2) Domain-Relationship Analysis of Spectral Subspaces.

The core subspace exhibits very high affinity across related ETT datasets (0.995), while its affinity decreases to 0.812 when compared with external datasets. In contrast, subordinate and noise subspaces exhibit smaller affinity gaps. These findings suggest that the core subspace provides a more distinctive representation of dominant temporal semantics within related domains.

3) Predictive Relevance of Different Spectral Subspaces.

To investigate the predictive relevance of the core and subordinate subspaces, we construct three groups of naturally occurring sample pairs based on the pairwise distances between their corresponding subspace representations: (A) similar core and similar subordinate, (B) similar core but different subordinate, and (C) different core but similar subordinate. For each group, we report the mean pairwise distances within the core and subordinate subspaces, together with the corresponding ground-truth future distances (dyd_y).

DatasetA: Similar Core & Similar Sub (Core / Sub / Future)B: Similar Core & Different Sub (Core / Sub / Future)C: Different Core & Similar Sub (Core / Sub / Future)
ETTh10.755 / 0.705 / 0.8210.760 / 1.233 / 0.9831.265 / 0.804 / 2.708
ETTm10.816 / 0.728 / 0.6990.884 / 1.262 / 0.7961.266 / 0.786 / 2.021
Exchange0.805 / 0.906 / 0.4470.713 / 1.365 / 0.4601.462 / 0.842 / 2.388
ZafNoo0.818 / 0.727 / 0.9430.773 / 1.182 / 1.1861.183 / 0.813 / 2.035
Mean Future Distance0.7280.8562.288

Across all four datasets, future distances consistently follow dy(A)<dy(B)<dy(C)d_y(A)<d_y(B)<d_y(C). When core distances remain comparable (A vs. B), increasing subordinate distances leads to only modest increases in future distances. In contrast, larger core distances (A vs. C) are accompanied by substantially greater future differences, suggesting that core semantics exhibit stronger associations with future patterns, while subordinate semantics provide comparatively weaker predictive discrimination.

Overall, the theoretical motivation and empirical analyses consistently support the distinction between core and subordinate subspaces. The core subspace exhibits stronger cross-domain consistency and closer association with future patterns, providing a basis for SIAA to preserve stable core semantics while improving robustness.

We have expanded the theoretical discussion in Section 3.1 and added predictive-relevance analysis, in Appendix A.6 in the revised paper.

For W2 . Validity of Cross-Sample Reconstruction

We clarify the validity of our cross-sample supervision from two complementary perspectives.

1) Preservation of Component-wise Historical-to-Future Correspondence.

CSFE combines the historical trend component of one sample with the historical seasonal component of another, while retaining their respective original future components as supervision targets. Specifically, we uniformly sample an ordered pair of distinct instances (a,b)(a,b) from the same high-volatility subset Itop\mathcal{I}_{\mathrm{top}}, and construct Xmix=Xt(a)+Xs(b)X_{\mathrm{mix}}=X_t^{(a)}+X_s^{(b)}. We supervise the predicted trend and seasonal components using Yt(a)Y_t^{(a)} and Ys(b)Y_s^{(b)}, respectively. Since each historical component remains paired with its corresponding future component from the same source instance, uniform sampling preserves the marginal joint distributions of the original component-wise historical-future pairs. Thus, the mixing changes the cross-component pairing while retaining the original source correspondence within each component.

2) Statistical Interpretation of the CSFE Objective under Nonlinear Forecasting.

To further characterize the statistical validity of cross-sample component-wise supervision under forecasting, let Z=XmixZ=X_{\mathrm{mix}} and define ht(Z)=SDt(f(Z))h_t(Z)=\mathrm{SD}_t(f(Z)) and hs(Z)=SDs(f(Z))h_s(Z)=\mathrm{SD}_s(f(Z)) as the predicted trend and seasonal components, respectively. Here, SD(⋅)\mathrm{SD}(\cdot) denotes the moving-average-based series decomposition operator. The component-wise reconstruction losses define an auxiliary population risk under the induced mixed-data distribution: RCFR=Emix[∣∣hs(Z)−Ys(b)∣∣22+α∣∣ht(Z)−Yt(a)∣∣1]\mathcal R_{\mathrm{CFR}}=\mathbb E_{\mathrm{mix}}\left[||h_s(Z)-Y_s^{(b)}||_2^2+\alpha||h_t(Z)-Y_t^{(a)}||_1\right], where α\alpha is the adaptive scaling factor defined in Eq. (12). This objective is defined directly over the joint distribution of the mixed historical input and its corresponding component-wise future targets.

Specifically, the population-optimal seasonal prediction minimizes the conditional squared-error risk and is therefore given by the conditional mean, hsBayes(Z)=Emix[Ys(b)∣Z]h_s^{\mathrm{Bayes}}(Z)=\mathbb E_{\mathrm{mix}}[Y_s^{(b)}\mid Z]. Similarly, the population-optimal trend prediction minimizes the conditional weighted absolute-error risk and is given by the conditional weighted median, htBayes(Z)=arg⁡min⁡uEmix[α∣∣u−Yt(a)∣∣1∣Z]h_t^{\mathrm{Bayes}}(Z)=\arg\min_u\mathbb E_{\mathrm{mix}}[\alpha||u-Y_t^{(a)}||_1\mid Z].

This statistical characterization does not impose linearity assumptions on the forecasting function or the underlying temporal dynamics. The conditional distributions are defined directly over the joint distribution of the mixed historical input and its corresponding future component targets, without assuming statistical independence between trend and seasonal components. Thus, the conditional risk formulation provides a statistical interpretation of component-wise supervision. Finally, the same-subset restriction confines component recombination to the selected high-volatility subset, while the standard forecasting loss in Eq. (13) continues to supervise original historical-future pairs (X,Y)(X,Y), retaining direct learning from naturally observed temporal dynamics.

Accordingly, CSFE is designed as an auxiliary component-wise structural supervision mechanism. By imposing reconstruction constraints on the future trend and seasonal components under recombined historical inputs, it encourages the forecaster to preserve the structural characteristics of both components, complementing the standard forecasting objective on naturally observed sequences.

In the revised paper, we provide the detailed statistical characterization of historical-to-future source correspondence and the population-optimal component-wise prediction objectives in Appendix A.11.

Reviewer_gkr1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SIFT, a fine-tuning method for time series foundation models that aims to reduce overfitting and overly smooth predictions. It introduces two main components: semantic-invariant adversarial augmentation (SIAA), which improves robustness through subspace-constrained perturbations, and component-based structural fidelity enhancement (CSFE), which uses component mixup and reconstruction to better preserve temporal patterns. Experiments on four foundation models and ten datasets show that SIFT generally improves forecasting performance over existing fine-tuning methods.

Critical strengths
  1. The paper addresses two practical issues in fine-tuning time series foundation models: overfitting and overly smooth predictions. The proposed SIAA and CSFE modules are designed to tackle these issues separately, making the overall approach easy to follow.
  2. The authors evaluate SIFT on four time series foundation models and ten datasets, showing consistent improvements over several fine-tuning baselines. They also provide ablation studies and sensitivity analyses to better understand the contributions of different components.
Critical weaknesses
  1. The paper assumes that the dominant singular components capture core semantics, while the intermediate components mainly represent less useful variations. However, this assumption is not sufficiently justified, as the intermediate components may also contain important information for forecasting.
  2. The paper aims to reduce overfitting and the mean-prediction trap, but most experiments focus on improvements in MSE and MAE. While these results are encouraging, they do not directly show how much SIFT reduces overfitting or prevents overly smooth predictions.
Questions for authors
  1. The authors assume that the dominant singular components capture the core semantics, while the intermediate components are less useful for forecasting. However, these intermediate components may still contain important predictive information. Could the authors provide more evidence that perturbing them does not affect useful forecasting signals?
  2. The proposed component mixup combines the trend from one sample with the seasonal component from another. However, these components may be closely related in real-world time series, and mixing them across different samples could lead to unrealistic patterns. Could the authors further explain whether the mixed samples still preserve valid relationships between historical observations and future targets?
  3. SIFT requires training an additional perturbation generator, along with spectral decomposition and component-wise reconstruction during fine-tuning. Could the authors compare its total training time and GPU memory usage with standard fine-tuning and MSFT?
8 comments

Thank you for your suggestion. We try to address your concerns in the following ways:

For W1/Q1. Theoretical and Empirical Justification of Spectral Subspace Decomposition We provide additional theoretical motivation and complementary analyses from three perspectives:

1) Theoretical Motivation of Spectral Subspace Decomposition: Our spectral decomposition is motivated by prior findings on spectral transferability. Previous studies [1] suggest that dominant spectral components tend to exhibit stronger transferability, whereas weaker components may encode less transferable variations. Although these findings do not directly establish the transferability of spectral components in raw time series, they motivate us to investigate whether different temporal spectral components exhibit distinct semantics and predictive characteristics.

To this end, we introduce classical Singular Spectrum Analysis (SSA) to characterize the temporal structure through the singular spectrum of Hankel trajectory matrices [2]. Given a time series XX, its trajectory matrix admits the decomposition H(X)=∑i=1dσiuivi⊤\mathcal H(X)=\sum_{i=1}^{d}\sigma_i u_i v_i^\top, where σ1≥σ2≥⋯≥σd≥0\sigma_1\geq\sigma_2\geq\cdots\geq\sigma_d\geq0. We denote the ii-th elementary spectral component as Hi=σiuivi⊤\mathcal H_i=\sigma_i u_i v_i^\top, whose energy is ∥Hi∥F2=σi2\|\mathcal H_i\|_F^2=\sigma_i^2. Based on this spectral ordering, we define the leading eigentriples (i≤r1i\leq r_1) as the core subspace and the intermediate eigentriples (r1<i≤r2r_1<i\leq r_2) as the subordinate subspace.

The interpretation of these spectral components is supported by classical SSA theory, particularly the concepts of separability and spectral dominance [2, 3]. When an underlying temporal component admits a relatively low-rank representation, dominates the residual components in energy, and is approximately separable from other components, its energy tends to concentrate in a small number of leading eigentriples. Consequently, these leading components tend to capture dominant temporal representations, such as long-term trends and long-period oscillations. In contrast, intermediate eigentriples retain non-dominant temporal variations.

We further characterize the potential influence of individual spectral components on forecasting through a norm-based bound. Let Xi=Ψ−1(Hi)X_i=\Psi^{-1}(\mathcal H_i) denote the reconstructed time-series component of the ii-th eigentriple, where Ψ−1\Psi^{-1} performs anti-diagonal averaging. Since each reconstructed entry is obtained by averaging the corresponding entries along an anti-diagonal, this reconstruction operator is non-expansive with respect to the Frobenius and Euclidean norms, yielding ∥Xi∥2≤∥Hi∥F=σi\|X_i\|_2\leq\|\mathcal H_i\|_F=\sigma_i. For a linear predictor f(X)=WX+bf(X)=WX+b, the prediction difference induced by removing the ii-th component satisfies ∥f(X)−f(X−Xi)∥2=∥WXi∥2≤∥W∥2∥Xi∥2≤∥W∥2σi\|f(X)-f(X-X_i)\|_2=\|WX_i\|_2\leq\|W\|_2\|X_i\|_2\leq\|W\|_2\sigma_i. More generally, for a nonlinear forecaster that is locally LfL_f-Lipschitz along the segment connecting XX and X−XiX-X_i, we obtain ∥f(X)−f(X−Xi)∥2≤Lf∥Xi∥2≤Lfσi\|f(X)-f(X-X_i)\|_2\leq L_f\|X_i\|_2\leq L_f\sigma_i. These bounds establish an energy-dependent upper bound on the change in model predictions induced by removing an individual reconstructed spectral component. These bounds establish a connection between spectral energy and the potential impact of individual components on forecasting, providing a mathematical justification for preserving dominant spectral representations during fine-tuning.

Taken together, prior findings on spectral transferability, the interpretation provided by SSA, and the above perturbation bounds motivate our spectral decomposition. The core subspace captures dominant temporal representations to be preserved, whereas the subordinate subspace contains non-dominant variations that may be less stable or transferable. Accordingly, SIAA applies adversarial regularization within the subordinate subspace to encourage prediction consistency under non-core variations while preserving dominant temporal representations, whose predictive relevance is further examined through empirical analyses.

[1] Chen, Xinyang, et al. "Catastrophic forgetting meets negative transfer: Batch spectral shrinkage for safe transfer learning." Advances in neural information processing systems 32 (2019).

[2] Golyandina, Nina. "Particularities and commonalities of singular spectrum analysis as a method of time series analysis and signal processing." Wiley Interdisciplinary Reviews: Computational Statistics 12.4 (2020): e1487.

[3] Golyandina, Nina, Vladimir Nekrutkin, and Anatoly A. Zhigljavsky. Analysis of time series structure: SSA and related techniques. CRC press, 2001.

2) Analysis of Predictive Information in Different Subspaces:

To investigate the predictive relevance of the core and subordinate subspaces, we construct three groups of naturally occurring sample pairs based on the pairwise distances between their corresponding subspace representations: (A) similar core and similar subordinate, (B) similar core but different subordinate, and (C) different core but similar subordinate. For each group, we report the mean pairwise distances within the core and subordinate subspaces, together with the corresponding ground-truth future distances (dyd_y).

DatasetA: Similar Core & Similar Sub (Core / Sub / Future)B: Similar Core & Different Sub (Core / Sub / Future)C: Different Core & Similar Sub (Core / Sub / Future)
ETTh10.755 / 0.705 / 0.8210.760 / 1.233 / 0.9831.265 / 0.804 / 2.708
ETTm10.816 / 0.728 / 0.6990.884 / 1.262 / 0.7961.266 / 0.786 / 2.021
Exchange0.805 / 0.906 / 0.4470.713 / 1.365 / 0.4601.462 / 0.842 / 2.388
ZafNoo0.818 / 0.727 / 0.9430.773 / 1.182 / 1.1861.183 / 0.813 / 2.035
Mean Future Distance0.7280.8562.288

Across all four datasets, future distances consistently follow dy(A)<dy(B)<dy(C)d_y(A)<d_y(B)<d_y(C). When core distances remain comparable (A vs. B), increasing subordinate distances leads to only modest increases in future distances. In contrast, larger core distances (A vs. C) are accompanied by substantially greater future differences, suggesting that core representations exhibit stronger associations with future patterns, while subordinate representations provide comparatively weaker predictive discrimination.

3) Analysis of Predictive Contributions from Different Spectral Subspaces:

To further examine the predictive contributions of different spectral subspaces, we compare forecasting performance using the full input, core-only components, and core+sub components under the same forecasting model and evaluation protocol.

DatasetFull MSEcore-only MSEcore+sub MSE
ETTh10.4500.4620.464
Exchange0.3960.3710.384
ZafNoo0.5230.5290.540
Weather0.2450.2520.247

The results show that core-only inputs retain forecasting performance comparable to that of full inputs across all four datasets, indicating that the core subspace preserves most of the predictive capability. Moreover, incorporating subordinate components does not yield consistent improvements. These findings suggest that subordinate components may retain some predictive information, but their marginal contributions are relatively limited and dataset-dependent, whereas the core components provide a more consistent basis for forecasting. Our objective is therefore not to discard these components, but to improve the model's robustness to perturbations within this subspace while preserving its core semantics.

In the revised paper, we have expanded the theoretical discussion in Section 3.1 and added rank-wise spectral visualizations, predictive-relevance analysis and predictive contributions in Appendix A.6 and A.7.

For W2. Evidence of Mitigating Overfitting and Over-Smoothing

1) Overfitting:

We conduct controlled experiments on five representative datasets using UniTS and Timer, with a context length of 96 and a prediction horizon of 96. We track the clean forecasting loss LpredL_{\mathrm{pred}} on unaugmented training samples and the validation MSE over 20 training epochs. For a fair comparison, LpredL_{\mathrm{pred}} excludes the additional consistency regularization introduced by SIAA. Each entry reports training LpredL_{\mathrm{pred}} / validation MSE.

TSFMDatasetMethodEpoch 1Epoch 10Epoch 20
UniTSETTh1Full FT0.6514 / 0.69130.6442 / 0.71310.5996 / 0.7439
SIAA0.6634 / 0.69130.6227 / 0.69050.5916 / 0.6871
ETTm1Full FT0.3081 / 0.40340.2241 / 0.41830.2221 / 0.3989
SIAA0.3239 / 0.41210.2448 / 0.38760.2434 / 0.3797
ExchangeFull FT0.1350 / 0.14020.0937 / 0.18110.0938 / 0.1805
SIAA0.1399 / 0.14190.1082 / 0.13620.1082 / 0.1357
ZafNooFull FT0.5360 / 0.40840.4762 / 0.39830.4766 / 0.4091
SIAA0.5470 / 0.41570.5087 / 0.40140.5089 / 0.3908
WeatherFull FT0.4575 / 0.41430.4014 / 0.41180.3998 / 0.4431
SIAA0.4772 / 0.42890.4258 / 0.41100.4242 / 0.3916
TimerETTh1Full FT0.7076 / 0.72550.6760 / 0.67870.6660 / 0.6876
SIAA0.7082 / 0.72450.6558 / 0.68910.6526 / 0.6686
ETTm1Full FT0.3278 / 0.40680.2638 / 0.41460.2638 / 0.4151
SIAA0.3433 / 0.40870.2759 / 0.39650.2738 / 0.3765
ExchangeFull FT0.1387 / 0.13600.1195 / 0.13640.1194 / 0.1364
SIAA0.1433 / 0.13690.1217 / 0.13250.1216 / 0.1308
ZafNooFull FT0.5687 / 0.42400.5245 / 0.41990.5242 / 0.4168
SIAA0.5658 / 0.43510.5229 / 0.41540.5231 / 0.4052
WeatherFull FT0.5154 / 0.45940.4601 / 0.46140.4600 / 0.4610
SIAA0.5423 / 0.47770.4722 / 0.45190.4722 / 0.4503

Evidence of overfitting. Under Full FT, training loss decreases while validation MSE increases on multiple datasets, indicating overfitting during fine-tuning. For example, on Exchange, training loss decreases from 0.1350 to 0.0938, whereas validation MSE increases from 0.1402 to 0.1805.

Effectiveness of SIAA. With SIAA, training loss continues to decrease while validation MSE also decreases on most datasets, indicating that SIAA mitigates overfitting and improves generalization during fine-tuning.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of rebuttals and revisions

Thank you very much for taking the time to review our paper. Our paper proposes SIFT, a robust fine-tuning framework for Time Series Foundation Models (TSFM) that preserves semantic invariance and structural fidelity to mitigate overfitting and the mean-prediction trap.

The reviewers positively recognized the motivation and significance of our work, noting that SIFT addresses "two practical issues" in TSFM fine-tuning and presents a "relevant problem and targeted design". The proposed SIAA and CSFE modules were appreciated for their "clearly complementary designs". The experimental evaluation was also praised as "rigorous and solid", with comprehensive results across four TSFMs and ten datasets demonstrating "consistent improvements over several fine-tuning baselines".

We sincerely thank all the reviewers for their insightful comments and constructive suggestions. We have made substantial revisions and conducted additional analyses and experiments to address their concerns, as summarized below.

1. Methodology:

  • Theoretical justification of spectral decomposition (Reviewer gkr1 W1/Q1, Reviewer r3wQ W1/Q1, Reviewer Edbk W1): We strengthened the theoretical motivation of our SIAA.

    Revised: Theoretical justification (Section 3.1, lines 255–265).

  • Predictive relevance of spectral subspaces (Reviewer gkr1 W1/Q1): We analyze predictive information at the data level.

    Newly added: Appendix A.6: Analysis of Predictive Information in Different Subspaces.

  • Predictive contributions of spectral subspaces (Reviewer gkr1 W1/Q1, Reviewer r3wQ follow up): We examine predictive contributions at the model level.

    Newly added: Appendix A.7: Predictive Contributions of Different Spectral Subspaces.

  • Adversarial perturbation mechanism (Reviewer Edbk W1): We clarify the perturbation generation process and quantify its spectral properties.

    Newly added: Appendix A.9: Spectral Subspace Analysis of Generated Perturbations.

  • Validity of cross-sample component mixup (Reviewer gkr1 Q2, Reviewer r3wQ W2/follow up): We justify the preserved historical-to-future correspondence within each component and the validity of cross-sample component mixup as auxiliary supervision.

    Newly added: Appendix A.11: Statistical Characterization of Cross-Sample Component Mixup.

2. Experiments:

  • Mechanism validation (Reviewer gkr1 W2, Reviewer r3wQ W3/Q2): We add training-validation diagnostics and structural fidelity metrics for SIAA and CSFE.

    Newly added: Appendix A.10: Diagnostics of the Two Challenges and Evidence for the Proposed Mechanisms.

  • Controlled ablation studies (Reviewer Edbk W2/Q3/Q5): We strengthen the ablation study to isolate component contributions and demonstrate their complementarity.

    Revised: Ablation study (Section 4.2, Table 2, lines 427-468).

  • Computational efficiency (Reviewer gkr1 Q3, Reviewer Edbk W3): We add comparisons of training time and peak GPU memory across fine-tuning methods.

    Newly added: Appendix A.12: Computational Efficiency Comparison.

3. Additional Clarifications:

  • Implementation details (Reviewer Edbk Q2): We clarify spectral rank selection, trajectory embedding, and multivariate channel processing.

    Revised: Appendix A.4: Hyperparameter and Implementation Details.

  • LoRA degradation and hyperparameter search (Reviewer Edbk Q4): We clarify baseline hyperparameter search.

    Revised: Appendix A.3: Fine-tuning baselines.

  • Distinction between SIFT and existing periodicity-modeling methods (Reviewer gkr1 follow-up):

    We clarify the fundamental difference from periodicity-modeling methods in both motivation and mechanism. Existing periodicity-modeling methods model periodicity explicitly for forecasting, while SIFT is a TSFM fine-tuning method using spectral decomposition for identifying core/subordinate semantics. We apply SIAA to mitigate overfitting and CSFE to preserve structural fidelity to alleviate the mean-prediction trap. Details are in our official comments.

  • Applicability to high-dimensional multivariate series (Reviewer gkr1 follow-up):

    We have supplemented the evaluation with the Traffic dataset. Methodologically, our channel-clustering-based spectral decomposition strategy is specifically designed to adapt to high-dimensional settings, and experimental results on high-dimensional multivariate series such as Traffic, Electricity, and Solar confirm SIFT's applicability to high-dimensional multivariate series.

The reviewers' constructive suggestions have substantially strengthened our paper. We have added extensive theoretical analyses and experimental evidence to address the above concerns. All corresponding revisions are highlighted in blue in the revised manuscript.

We sincerely hope these revisions and additional evidence can help you better evaluate our work. Thank you once again for your time and thoughtful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.