- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SIFT to address two problems in downstream fine-tuning of time series foundation models (TSFMs): overfitting and overly smooth predictions. The method comprises two modules. First, Semantic-Invariant Adversarial Augmentation (SIAA) uses SVD of the trajectory matrix to partition the spectrum into core, subordinate, and noise subspaces, generates adversarial perturbations within the subordinate semantic subspace, and regularizes the model through prediction consistency. Second, Component-based Structural Fidelity Enhancement (CSFE) selects high-volatility training samples, cross-pairs their trend and seasonal components, and preserves temporal structure through a component-wise fidelity reconstruction loss.
The experiments cover 10 datasets, four foundation models, and four prediction horizons. The paper reports results averaged over multiple independent runs and provides module ablations, parameter sensitivity analyses, and spectral component visualizations. The results demonstrate that SIFT improves forecasting performance across multiple backbones without introducing additional computation during inference.
Critical strengths
1. The experimental evaluation across backbones is rigorous and solid.
The paper evaluates Timer, UniTS, Moirai, and Sundial, rather than demonstrating results on a single model. In Table 1, Timer’s average MSE decreases from 0.405 under full fine-tuning to 0.357, while the other three backbones also achieve improvements in average MSE. These results indicate that SIFT has meaningful general applicability to fine-tuning time series foundation models.
2. The two modules have clearly complementary designs and introduce no additional inference computation.
By design, SIAA addresses sensitivity to input perturbations, whereas CSFE addresses the loss of structural details. They provide consistency regularization and structural supervision, respectively, during training. Both modules target practical problems in downstream fine-tuning of TSFMs and offer promising solutions. The ablation study also demonstrates their effectiveness.
Critical weaknesses
1. The central claim of “semantic invariance” is not sufficiently supported by the mathematical construction or empirical evidence.
The Eckart–Young–Mirsky theorem guarantees the best low-rank approximation, and it does not guarantee that high-energy directions correspond to the core semantics required for forecasting, nor that intermediate-rank components represent spurious patterns that should be ignored. Although the cross-domain affinity analysis in the appendix provides evidence of association, it does not establish that the forecasting target should remain unchanged when these components are perturbed. Should additional theoretical justification or a formal proof be provided?
Furthermore, Eq. (5) first projects the perturbation in the trajectory-matrix space and then reconstructs it through anti-diagonal averaging. After the resulting one-dimensional perturbation is re-embedded into a trajectory matrix, it is not generally guaranteed to remain orthogonal to the core subspace. However, Eq. (6) penalizes the core projection of the raw perturbation, rather than the core leakage of the final reconstructed perturbation. Eq. (6) also does not explicitly constrain the perturbation magnitude within the subordinate subspace. These issues directly affect whether consistency training operates on valid inputs that preserve the target semantics, and therefore constitute a fundamental methodological concern.
2. The experiments do not sufficiently distinguish the contributions of the proposed mechanisms from those of generic augmentation and additional supervision.
Table 2 provides useful ablations, but it lacks a direct forecasting-performance comparison between random and learned perturbations within the same subordinate subspace and at the same perturbation magnitude. It also does not separately isolate the contributions of high-volatility sample selection, structural component mixup, and component-wise fidelity reconstruction. Consequently, the current results do not sufficiently establish whether the complexity of the complete method is necessary.
Moreover, although the two modules are complementary by design and each individually provides clear improvements, the additional gains from combining them are only on the order of a few thousandths in the reported MSE values. The stability of these incremental gains needs to be confirmed. This limits the assessment of the mechanisms’ contributions and the magnitude of their benefits.
3. The method introduces no additional inference computation, but its training costs are not reported. Although the paper states that inference requires no additional computation, it does not report whether training costs increase relative to PEFT methods such as LoRA, including the number of trainable parameters and GPU memory usage. These are also important metrics for evaluating downstream fine-tuning of pretrained models.
Questions for authors
-
Does generator training use a perturbation norm constraint, output clipping, or normalization? Please provide the complete training objective and perturbation magnitude settings, and explain how the penalty in Eq. (6) controls core-subspace leakage in the final reconstructed perturbation, rather than merely the core projection of the raw perturbation.
-
In the actual experiments, is determined by the fixed ratio , or by the Marchenko–Pastur (MP) threshold? What are the specific configurations for the trajectory embedding window width and stride, and how are projection matrices shared across multivariate channels?
-
Under the same subspace, perturbation magnitude, and training budget, do learned perturbations outperform random perturbations? For CSFE, how much improvement is attributable separately to high-volatility sample selection, structural component mixup, and component-wise fidelity reconstruction?
-
Please explain why several LoRA configurations in Table 1 exhibit substantial performance degradation, and provide the hyperparameter search ranges and training compute budgets for each baseline.
-
Can you explain why the empirical gains from combining the two complementary modules are less pronounced than the preceding discussion suggests?
7 comments
[1] Chen, Xinyang, et al. "Catastrophic forgetting meets negative transfer: Batch spectral shrinkage for safe transfer learning." Advances in neural information processing systems 32 (2019).
[2] Golyandina, Nina. "Particularities and commonalities of singular spectrum analysis as a method of time series analysis and signal processing." Wiley Interdisciplinary Reviews: Computational Statistics 12.4 (2020): e1487.
[3] Golyandina, Nina, Vladimir Nekrutkin, and Anatoly A. Zhigljavsky. Analysis of time series structure: SSA and related techniques. CRC press, 2001.
2) Analysis of Predictive Information in Different Subspaces.
To investigate whether the spectral distinction is reflected in future-sequence similarity, we construct three groups of naturally occurring sample pairs based on the pairwise distances between their corresponding subspace representations: (A) similar core and similar subordinate, (B) similar core but different subordinate, and (C) different core but similar subordinate. For each group, we report the mean pairwise distances within the core and subordinate subspaces, together with the corresponding ground-truth future distances ().
| Dataset | A: Similar Core & Similar Sub (Core / Sub / Future) | B: Similar Core & Different Sub (Core / Sub / Future) | C: Different Core & Similar Sub (Core / Sub / Future) |
|---|---|---|---|
| ETTh1 | 0.755 / 0.705 / 0.821 | 0.760 / 1.233 / 0.983 | 1.265 / 0.804 / 2.708 |
| ETTm1 | 0.816 / 0.728 / 0.699 | 0.884 / 1.262 / 0.796 | 1.266 / 0.786 / 2.021 |
| Exchange | 0.805 / 0.906 / 0.447 | 0.713 / 1.365 / 0.460 | 1.462 / 0.842 / 2.388 |
| ZafNoo | 0.818 / 0.727 / 0.943 | 0.773 / 1.182 / 1.186 | 1.183 / 0.813 / 2.035 |
| Mean Future Distance | 0.728 | 0.856 | 2.288 |
Across all four datasets, future distances consistently follow . When core distances remain comparable (A vs. B), increasing subordinate distances leads to only modest increases in future distances. In contrast, larger core distances (A vs. C) are accompanied by substantially greater future differences. These results suggest that core representations exhibit stronger associations with future patterns, while subordinate representations provide comparatively weaker predictive discrimination.
3) Analysis of Predictive Contributions from Different Spectral Subspaces.
To further examine the predictive contributions of different spectral subspaces, we compare forecasting performance using the full input, core-only components, and core+sub components under the same forecasting model and evaluation protocol.
| Dataset | Full MSE | Core-only MSE | Core+Sub MSE |
|---|---|---|---|
| ETTh1 | 0.450 | 0.462 | 0.464 |
| Exchange | 0.396 | 0.371 | 0.384 |
| ZafNoo | 0.523 | 0.529 | 0.540 |
| Weather | 0.245 | 0.252 | 0.247 |
The results show that core-only inputs retain forecasting performance comparable to that of full inputs across all four datasets, indicating that the core subspace preserves most of the predictive capability. Moreover, incorporating subordinate components does not yield consistent improvements. These findings suggest that the subordinate subspace may still contain some predictive information, but can also encode spurious or non-transferable variations. Our objective is therefore not to discard these components, but to improve the model's robustness to perturbations within this subspace while preserving its core semantics.
Overall, the SVD-based semantic spectrum decomposition and complementary predictive analyses support the distinction between core and subordinate subspaces. The core subspace exhibits stronger associations with future patterns, whereas subordinate components retain structured but comparatively less predictive-discriminative variations. This provides a theoretically motivated and empirically supported basis for our spectral decomposition.
We have added the theoretical motivation in Section 3.1 and predictive-relevance analyses and predictive contributions in Appendix A.6 and A.7.
2. Mathematical Construction and Perturbation Constraints.
For perturbation magnitude control, our implementation bounds the perturbation magnitude through tanh-based scaling, with a maximum element-wise magnitude of 0.3 times the input's temporal standard deviation, preventing excessively large perturbations.
For generator objective, our objective is to guide perturbations toward the subordinate subspace, rather than enforce strict orthogonality to the core subspace. Specifically, the subordinate projection in Eq. (5) directs perturbation generation toward intermediate spectral components, while the core penalty in Eq. (6) discourages excessive core components in the raw generator output. We adopt this soft guidance to retain flexibility in perturbation generation, rather than imposing a restrictive orthogonality constraint on the final reconstructed perturbations. Consequently, some core leakage may arise after anti-diagonal averaging and re-embedding.
To quantify the spectral energy distribution of perturbations, we define the Core Leakage Ratio (CLR) and Subordinate Energy Ratio (SER) as: where is a small numerical constant. CLR measures the proportion of perturbation energy projected onto the Core subspace, while SER measures the proportion projected onto the Subordinate subspace. Here, Raw denotes the generator output, Projected denotes the perturbation after subordinate-space projection in the trajectory-matrix space, and Final denotes the reconstructed perturbation after anti-diagonal averaging.
| Dataset | Raw CLR | Projected CLR | Final CLR | Raw SER | Final SER |
|---|---|---|---|---|---|
| ETTh1 | 0.0659 | 9.11e-15 | 0.0261 | 0.5900 | 0.9173 |
| ETTm1 | 0.0981 | 5.63e-15 | 0.0274 | 0.5194 | 0.8962 |
| ZafNoo | 0.0409 | 8.28e-15 | 0.0262 | 0.5808 | 0.8975 |
As shown in the table, although the projected perturbations have negligible core leakage, the final perturbations exhibit small but non-zero leakage, with CLR remaining below 0.03 across all evaluated datasets. Meanwhile, the final Subordinate Energy Ratio (SER) ranges from 0.8962 to 0.9173. These results empirically suggest that the final perturbations predominantly preserve subordinate-subspace energy.
We describe the perturbation-generation procedure and its optimization objective, and further analyze the spectral energy distribution of generated perturbations at different stages in Appendix A.9 in the revised paper.
Thank you for your suggestion. We try to address your concerns in the following ways:
For W1 and Q1. Theoretical and Empirical Justification of Semantic Invariance
We provide additional theoretical clarification and empirical analyses regarding the interpretation of spectral subspaces and the motivation for subordinate-space perturbations.
1. Semantic Interpretation and Predictive Relevance.
1) Theoretical Motivation of Spectral Subspace Decomposition.
Our justification therefore draws on complementary findings from spectral transferability and classical SSA, together with a norm-based analysis of prediction sensitivity.
Our spectral decomposition is motivated by prior findings on spectral transferability. **Previous studies [1] suggest that dominant spectral components tend to exhibit stronger transferability, whereas weaker components may encode less transferable variations. **Although these findings do not directly establish the transferability of spectral components in raw time series, they motivate us to investigate whether different temporal spectral components exhibit distinct structural and predictive characteristics.
To this end, we introduce classical Singular Spectrum Analysis (SSA) to characterize temporal representations through the singular spectrum of Hankel trajectory matrices [2]. Given a time series X, its trajectory matrix admits the decomposition H(X)=∑i=1dσiuivi⊤, where σ1≥σ2≥⋯≥σd≥0. We denote the i-th elementary spectral component as Hi=σiuivi⊤, whose energy is ∣Hi∣F2=σi2. Based on this spectral ordering, we define the leading eigentriples (i≤r1) as the core subspace and the intermediate eigentriples (r1<i≤r2) as the subordinate subspace.
The structural interpretation of these spectral components is supported by classical SSA theory, particularly the concepts of separability and spectral dominance [2, 3]. When an underlying temporal component admits a relatively low-rank representation, dominates the residual components in energy, and is approximately separable from other components, its energy tends to concentrate in a small number of leading eigentriples. Consequently, these leading components tend to capture dominant temporal representations, such as long-term trends and long-period oscillations. In contrast, intermediate eigentriples retain non-dominant temporal variations. Importantly, we do not assume that these intermediate components are purely noise or entirely devoid of predictive information; rather, they may encode sample-specific or domain-dependent variations with relatively weaker structural dominance.
We further characterize the potential influence of individual spectral components on forecasting through a norm-based bound. Let Xi=Ψ−1(Hi) denote the reconstructed time-series component of the i-th eigentriple, where Ψ−1 performs anti-diagonal averaging. Since each reconstructed entry is obtained by averaging the corresponding entries along an anti-diagonal, this reconstruction operator is non-expansive with respect to the Frobenius and Euclidean norms, yielding ∣Xi∣2≤∣Hi∣F=σi. For a linear predictor f(X)=WX+b, the prediction difference induced by removing the i-th component satisfies ∣f(X)−f(X−Xi)∣2=∣WXi∣2≤∣W∣2∣Xi∣2≤∣W∣2σi. More generally, for a nonlinear forecaster that is locally Lf-Lipschitz along the segment connecting X and X−Xi, we obtain ∣f(X)−f(X−Xi)∣2≤Lf∣Xi∣2≤Lfσi. These bounds establish an energy-dependent upper bound on the change in model predictions induced by removing an individual reconstructed spectral component. These bounds establish a connection between spectral energy and the potential impact of individual components on forecasting, providing a mathematical justification for preserving dominant spectral representations during fine-tuning.
Taken together, prior findings on spectral transferability, the interpretation provided by SSA, and the above perturbation bounds motivate our spectral decomposition. The core subspace captures dominant temporal representations to be preserved, whereas the subordinate subspace contains non-dominant variations that may be less stable or transferable. Accordingly, SIAA applies adversarial regularization within the subordinate subspace to encourage prediction consistency under non-core variations while preserving dominant temporal representations, whose predictive relevance is further examined through empirical analyses.