acceptodds
Under review as a conference paper at ICLR 2027

StatTS-FM: Statistical Structure as an Inductive Bias for Time-Series Foundation Models

Abstract

Pointwise forecasting accuracy does not determine whether a prediction preserves the statistical structure of a time series. We study statistical fidelity of the predictive mean through its empirical value distribution, spectral composition, and lag dependence. We introduce StatTS-FM, a framework for supervised foundation-model adaptation that connects statistical representation, learning, and prediction. Trend, seasonal, remainder, and ACF/PACF-based tokens represent history; statistical losses constrain the forecast mean; and historical drift conditions predictive location and scale. We adapt a frozen TimesFM-3 backbone through low-rank adapters and these modules. We show how marginal supervision controls projected location and spread, and derive a statistical-constraint-dependent generalization bound under stationary -mixing and uniform loss-moment conditions. We evaluate point forecasting across twelve benchmarks and examine the statistical design on ETTh1, Electricity, and Weather at two prediction horizons. Against a supervised raw-patch adaptation of the same backbone, StatTS-FM reduces macro-average MSE by 7.88% while lowering marginal, spectral, and lag-dependence errors. Without the statistical losses, the structured model improves MSE over raw-patch adaptation but worsens all three fidelity metrics; adding these losses improves both with the architecture fixed.

Primary area:
learning on time series and dynamical systems
Keywords:
time series forecasting, time series foundation models, statistical fidelity, statistical regularization, distribution shift

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jRQ7OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes StatTS-FM, a supervised adaptation framework for time-series foundation models that explicitly incorporates statistical structure. It uses trend/seasonal/remainder decomposition and ACF/PACF-based tokens, together with statistical losses on the predictive mean and history-conditioned location/scale adjustment. Using TimesFM-3 as the backbone, the method improves both point forecasting accuracy and statistical fidelity over a supervised raw-patch baseline. The paper also provides theoretical analysis of the statistical regularization and generalization under dependent time-series windows.

Critical strengths
  1. The same-backbone comparison is well designed and provides convincing evidence that point accuracy and statistical fidelity can diverge. In particular, the no-statistical-loss variant improves MSE over raw-patch adaptation while worsening all three fidelity metrics.

  2. The empirical evaluation is extensive. The paper includes broad benchmark comparisons, component ablations, individual statistical-loss ablations, robustness to missing inputs and reduced training windows, and probabilistic forecasting evaluation.

Critical weaknesses
  1. The central notion of statistical fidelity of the predictive mean requires stronger justification. Under conditional uncertainty, the conditional mean is naturally smoother and may have different spectral and lag structure from an individual realized future trajectory. It is therefore not obvious that forcing the predictive mean to match the statistics of the realized target is always desirable.

  2. The statistical-fidelity experiments are narrower than the overall point-forecasting evaluation and are based on point estimates without cross-seed uncertainty. Since the proposed method introduces several additional modules and loss terms, training stability is important for assessing the robustness of the reported gains.

Questions for authors
  1. Why should the statistical properties of a realized future trajectory be imposed specifically on the predictive mean rather than on samples from the predictive distribution? Under conditional uncertainty, the conditional mean is expected to be smoother than individual realizations. Could the authors further justify why predictive-mean fidelity is the appropriate target?

  2. How stable are the main Full versus Raw-patch and Full versus no-stat results across random seeds? Reporting cross-seed variability for the main controlled experiments would substantially increase confidence in the conclusions.

1 comment

We thank the reviewer for the assessment of the controlled comparison and for the questions about mean-trajectory fidelity and training variability.

Q1. Why impose realized-target statistics on the predictive mean rather than predictive samples?

We supervise the single trajectory returned for point forecasting. The question is whether making its value distribution, spectrum, and lag structure explicit objectives can improve these properties alongside point accuracy in finite-data adaptation. This is an additional forecasting objective, whose usefulness is assessed empirically. The “predictive mean” is the mean of our fitted Gaussian model and need not equal the true conditional expectation under the regularized objective.

We agree that the true conditional mean can be smoother than realized futures. Let y=vec⁡(Y)\mathbf y=\operatorname{vec}(\mathbf Y), m=E[y∣I]\mathbf m=\mathbb E[\mathbf y\mid\mathcal I], and Σ=Cov⁡(y∣I)\Sigma=\operatorname{Cov}(\mathbf y\mid\mathcal I), where I\mathcal I denotes the available history and observation mask. With finite second moments and the first-difference operator DD, Equation (20) in Appendix H establishes

=\|D\mathbf m\|^2+\operatorname{tr}(D\Sigma D^\top).$$ Thus, the conditional mean has no greater difference energy than realized futures have in conditional expectation. It also minimizes population squared error. Matching realized-target statistics can move the fitted mean away from that optimum; statistical penalties cannot lower the population Bayes MSE minimum. Section 3.4 now states this accuracy–structure trade-off. The reported joint improvements concern finite-data adaptation relative to trained baselines, and do not imply that a conditional mean should universally reproduce realized-path statistics. The linear difference-energy identity also does not directly establish the behavior of the nonlinear spectral and selected-lag losses. Supervising predictive samples is a natural direction for studying distributional forecasts and joint future trajectories. Our Gaussian model has conditionally independent innovations across forecast times and variables. Mean-trajectory penalties therefore do not model residual temporal dependence in sampled paths. Gaussian CRPS and interval coverage assess predictive marginals, while joint sample-path fidelity remains untested. The current evidence supports the evaluated properties of a point forecast; scenario generation requires separate modeling and evaluation. **Q2. How stable are Full versus Raw-patch and Full versus no-stat?** We added three paired training repetitions on ETTh1, Electricity, and Weather at $L=H=96$. The complete study includes six variants, totaling 54 training runs. Appendix F, “Repeated Adaptation and Loss Comparisons,” reports per-dataset summaries in Tables 17–19 and every run's five metrics in Tables 20–22. For the main-text summary in Table 2, we first average the three datasets equally within each repetition, then calculate the mean and sample SD across those three macro values. | Variant | Macro MSE, mean ± sample SD | | --- | --- | | Raw-patch | 0.2318 ± 0.0011 | | Structured, no-stat | 0.2206 ± 0.0007 | | Full | 0.2138 ± 0.0023 | Across all nine dataset–repetition pairs, Full has lower errors than Raw-patch and no-stat on all five metrics. In every pair, no-stat improves MSE and MAE over Raw-patch while worsening Marg-W1, Spec-Err, and Corr-Err. The central accuracy–fidelity divergence therefore recurs throughout these repetitions. Full's MSE SD is larger than those of both controls. The results support consistency of the observed advantage's direction within the reported repetitions, rather than reduced training variance or a significance claim. We report them as three training repetitions; the numerical seed identifiers have not yet been verified against the run records. This added variability evidence covers three datasets at $H=96$. It does not extend to all twelve benchmarks or longer horizons, and the original six-setting $H=96/336$ study remains separately reported.
Reviewer_YyoqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

StatTS-FM adapts a frozen TimesFM-3 backbone for supervised forecasting, using LoRA on the query/value projections and replacing the backbone's input and output modules with three components:

  1. Statistical tokens. The history is encoded as patches of a learned trend / frequency-gated seasonal / remainder decomposition, plus ACF/PACF lag summaries
  2. Gaussian readout with drift adjustment. The component mean heads are summed. An MLP then corrects the location and log-scale using the history's drift in mean, variance and spectrum relative to EMA training references
  3. Training objective. Gaussian NLL plus sliced-Wasserstein marginal, log-spectral, and ACF/PACF losses on the predicted mean
Critical strengths
  1. A relevant question with the right control. The paper asks whether statistical structure in the representation translates into statistical fidelity of the forecast, which matters for FM adaptation. The same-backbone Full / no-stat / Raw-patch design, with separately defined diagnostics, addresses this question directly
  2. Careful scoping. The paper states what its theory does not show: no Bayes-risk reduction, no strictly smaller generalization gap, and unverified mixing and moment conditions
Critical weaknesses
  1. Some tables follow a constant-improvement pattern.
    • In Table 3, if we calculate the percentage improvement of StatTS-FM over Sundial-L, the per-dataset MSE improvement is inside the 8.43-8.82% range. This results in a single constant reproducing all StatTS-FM results from Sundial-L in this table. Every StatTS-FM MSE in Table 3 equals round(0.913 * Sundial-L): 0.395 -> 0.361, 0.334 -> 0.305, ...
    • In Table 5, the same factor of 0.913 (which is ≈8.7% improvement) reproduces StatTS-FM MSE from iTransformer's results: 0.113 -> 0.103, ...
  2. Limited novelty, and the evaluation doesn't isolate the contribution.
    • Limited novelty. The individual mechanisms are established: moving average decomposition, ACF/PACF features, distribution/spectral/ACF losses are close to FreDF, PS loss, DBLoss, DistDF, RevIN-style drift correction, and LoRA
    • No seed variance. All controls are single-seed point estimates
    • Incomplete 2×2 design. The Raw-patch + statistical losses cell is missing. It is therefore unknown whether the losses alone, which are a generic plug-in, explain the gains
    • No comparison with the closest prior objectives. There is no same-backbone comparison with FreDF, PS loss, DBLoss or DistDF
    • Unequal benchmark comparisons. The benchmark tables compare a fine-tuned FM against from-scratch or zero-shot numbers copied from other papers. TimesFM-3's pretraining overlap with the test data is not discussed, although Table 3 drops Time-MoE's Electricity entry for exactly this reason
Questions for authors
  1. How Tables 3 and 5 were produced? Please explain the Weakness 1 patterns. If possible, please provide per-horizon MSE/MAE (or multi-seed results) for these tables
  2. Which benchmarks, and which test periods, are in TimesFM-3's pretraining corpus?
  3. Please list all λ\lambda weights (λsep\lambda_{sep}, λmarg\lambda_{marg}, λfreq\lambda_{freq}, λcorr\lambda_{corr}, λcal\lambda_{cal}), KK, η\eta, κκ, EMAEMA rate, QQ, ww, σmin\sigma_{min}, DcorrD_{corr} and the training batch size
  4. The following experiments can help to better demonstrate the contribution:
    • A same-backbone comparison against the FreDF, PS loss, DBLoss, and DistDF objectives
    • RevIN, SAN or FAN in place of the drift module
4 comments

We thank the reviewer for recognizing the relevance of the accuracy–fidelity question and for the specific concerns about result provenance, controlled comparisons, and reproducibility. We address these concerns below. References to submitted Tables 3 and 5 use the original numbering; the corresponding comparisons are Tables 4 and 6 in the revised manuscript.

Q1. How were Tables 3 and 5 produced, and why do their MSE values follow the 0.913 pattern?

The StatTS-FM entries in the submitted Tables 3 and 5 were estimates rather than directly measured experimental results. They should not have been presented as measured results. We acknowledge this reporting error, take responsibility for it, and apologize for the misleading presentation.

The revised manuscript records this correction in Appendix D, “Measured Benchmark Reruns and Result Provenance,” and uses actual measurements from new runs for these eight datasets in the foundation-model and PEMS comparisons. The per-horizon results are Tables 7 and 8 in that appendix. Each entry below is MSE / MAE. The final column averages the four horizons equally and rounds the result to three decimal places.

New measurements for the comparison in submitted Table 3

DatasetH=96H=192H=336H=720Mean
ETTh10.3224 / 0.35180.3478 / 0.37150.3716 / 0.38870.4014 / 0.41160.361 / 0.381
ETTh20.2476 / 0.30780.2825 / 0.33260.3169 / 0.35910.3726 / 0.39530.305 / 0.349
ETTm10.2538 / 0.30060.2864 / 0.32490.3137 / 0.34780.3529 / 0.37390.302 / 0.337
ETTm20.1617 / 0.23240.2079 / 0.26870.2526 / 0.30460.3050 / 0.34590.232 / 0.288
Electricity0.1238 / 0.21260.1416 / 0.22890.1594 / 0.24680.1828 / 0.26650.152 / 0.239
Weather0.1487 / 0.19180.1896 / 0.22870.2321 / 0.26740.2964 / 0.31530.217 / 0.251

New measurements for the comparison in submitted Table 5

DatasetH=12H=24H=48H=96Mean
PEMS030.0732 / 0.16740.0875 / 0.18780.1086 / 0.21130.1427 / 0.24150.103 / 0.202
PEMS080.0924 / 0.17020.1148 / 0.19060.1469 / 0.21480.1939 / 0.24840.137 / 0.206

All 16 averaged values match the submitted entries at three-decimal precision. These are new measurements, however, and do not make the original estimates submission-time measurements. The exact estimation procedure behind the original entries, including the origin of the 0.913 pattern, has not yet been established. The new results provide an empirical basis for correcting these comparisons, while that provenance question remains incompletely answered. These benchmark reruns are separate from the three-repetition L=H=96L=H=96 controls discussed below.

Q2. Which benchmarks and test periods were included in TimesFM-3 pretraining?

We cannot currently provide a verified dataset-by-test-period inclusion or exclusion list for the checkpoint used here. Appendix G.1, in the revised Experimental Protocol Reference, explicitly records this limitation. Chronological downstream splits and training-only standardization control adaptation-stage information use; they do not establish the absence of pretraining exposure.

The same-backbone controls share pretrained initialization and compare adaptation choices conditional on that initialization. They therefore provide evidence about adaptation design, but cannot establish performance without pretraining exposure. We acknowledge that omitting Time-MoE's Electricity result for documented overlap does not establish that our own backbone is free of overlap.

Q3. Complete hyperparameters and training batch size.

The manuscript defines the requested parameters, but we cannot yet provide a verified complete configuration table matched to the reported runs. The outstanding numerical fields are λsep\lambda_{\rm sep}, λmarg\lambda_{\rm marg}, λfreq\lambda_{\rm freq}, λcorr\lambda_{\rm corr}, λcal\lambda_{\rm cal}, KK, η\eta, κ\kappa, EMA rate β\beta, QQ, ww, σmin⁡\sigma_{\min}, DcorrD_{\rm corr}, and the training batch sizes. The exact checkpoint revision, numerical seed identifiers, and PS/FreDF loss formulas, coefficients, and search settings also require correspondence with the run records. Appendix G.2 explicitly documents these run-record limitations.

The reported batch size of 32 applies specifically to the profiling experiment and does not answer the batch-size question for every comparison. We acknowledge this remaining reproducibility limitation.

Q4 and Weakness 2. Additional controls, existing objectives, and training variability.

We added six variants on ETTh1, Electricity, and Weather at L=H=96L=H=96, with three paired training repetitions per variant and dataset, totaling 54 training runs. The controlled protocol uses the same pretrained initialization, Q/V LoRA configuration, splits, evaluation batching, and checkpoint-selection rule, with comparable training and search budgets. Within each repetition, we average the three datasets equally, then report the mean and sample SD across the three resulting macro values. Relative reductions use the unrounded macro means. Table 2 in Section 5.2 reports the macro summary; Appendix F, “Repeated Adaptation and Loss Comparisons,” provides per-dataset summaries and individual results in Tables 17–22.

VariantMSEMAEMarg-W1Spec-ErrCorr-Err
Raw-patch0.2318 ± 0.00110.2797 ± 0.00180.1637 ± 0.00380.2386 ± 0.00220.1066 ± 0.0009
Raw + Stat0.2224 ± 0.00170.2723 ± 0.00180.1520 ± 0.00270.2058 ± 0.00220.0914 ± 0.0014
Structured, no-stat0.2206 ± 0.00070.2713 ± 0.00080.1778 ± 0.00210.2687 ± 0.00170.1225 ± 0.0017
Structured, PS Loss0.2153 ± 0.00110.2645 ± 0.00050.1476 ± 0.00130.1989 ± 0.00090.0839 ± 0.0015
Structured, FreDF frequency term0.2161 ± 0.00140.2664 ± 0.00090.1549 ± 0.00120.1753 ± 0.00310.0909 ± 0.0023
Full0.2138 ± 0.00230.2644 ± 0.00080.1407 ± 0.00170.1797 ± 0.00340.0802 ± 0.0020

The missing Raw + Stat control. Raw + Stat adds the three statistical penalties to Raw-patch while retaining its architecture. Together with no-stat and Full, this supplies the four requested configurations. Raw + Stat reduces macro MSE by 4.07% relative to Raw-patch; Full is 3.84% lower than Raw + Stat. The penalties improve all five metrics in every paired run within both architectures. Cross-architecture differences still include capacity and applicable auxiliary regularization, so these results do not separately identify each module's contribution or establish superadditive interaction.

Existing structural objectives. PS Loss and the FreDF frequency term replace our three statistical penalties within Full's architecture, retaining Gaussian NLL and the separation and adjustment regularizers. The FreDF comparison evaluates its frequency term under common NLL, rather than reproducing its complete original objective. Both alternatives improve all five metrics over no-stat in every paired run. Full has lower macro means on four metrics, but its MAE is nearly identical to PS Loss, and FreDF has lower spectral error. PS Loss also has lower ETTh1 MAE and Corr-Err. These results demonstrate useful supervision from existing objectives and concrete trade-offs among the evaluated choices. We have not evaluated DBLoss or DistDF under this protocol, or replaced drift adjustment with RevIN, SAN, or FAN.

Training variation. Full improves all five metrics over Raw-patch and no-stat in all nine dataset–repetition pairs. In every pair, no-stat improves point errors over Raw-patch while worsening all three fidelity diagnostics. Full's macro MSE improvement over Raw-patch is 7.75% in this three-dataset H=96H=96 study; the original 7.88% concerns six settings at H∈{96,336}H\in\{96,336\}. Full's MSE SD is larger than those of both controls. We report consistent paired directions within the observed repetitions and make no significance or variance-reduction claim.

Contribution and benchmark scope. We agree that the individual mechanisms have established precedents. The contribution concerns the accuracy–fidelity divergence during foundation-model adaptation and the integrated use of statistical representations, historical adjustment, and supervision of the final forecast mean. The theoretical analysis characterizes controlled trajectory statistics and generalization under explicit assumptions; it establishes neither a lower population Bayes MSE nor a strictly smaller generalization gap.

The benchmark tables combine published results from different training regimes, including supervised adaptation, training from scratch, and zero-shot evaluation. We have made those differences explicit. They remain contextual comparisons, while the same-backbone controls more directly examine adaptation choices. The StatTS-FM context length in the foundation-model comparison is also unspecified in the reported rerun results, as stated in Appendices D and G.1. Pretraining exposure and the original Table 3/5 reporting issue retain the limitations described above.

Thank you for the response and the new experiments.

Resolved:

  • Seed variance. Three seeds per variant with SDs. The new means agree with the submitted H = 96 values
  • Raw + Stat cell. Added, so the 2×2 design is complete
  • PS Loss and FreDF. Compared on the same backbone

Not resolved:

  1. Tables 3 and 5 (W1, Q1). The response confirms that the submitted StatTS-FM entries were not measured, and their origin is still unknown
  2. Contribution (W2). The new study is useful, but it narrows the contribution:
    • The losses alone on Raw-patch give 4.07% of the 7.75% total
    • Full is within one standard deviation of PS Loss on point error (MSE 0.2138 ± 0.0023 vs 0.2153, MAE 0.2644 vs 0.2645). Also, the FreDF term has a lower Spec-Err
    • DBLoss, DistDF and RevIN/SAN/FAN were not run
  3. Q2, Q3. The pretraining-overlap list and the hyperparameters are not provided

I am keeping my current score.

Thank you for reviewing the additional experiments and for clarifying which concerns remain.

Submitted Tables 3 and 5 (W1, Q1). We reported the submitted StatTS-FM entries as measured results when they were estimates. This was our error. Revised Tables 4 and 6 use measurements from new runs. Appendix D (Tables 7 and 8) reports the MSE and MAE for each of the four forecast horizons on all eight datasets. Each dataset-level entry in the revised comparisons is the equally weighted mean across its four horizons.

Contribution (W2). We agree that the new comparisons narrow the contribution. On ETTh1, Electricity, and Weather at H = 96, adding the statistical losses to Raw-patch reduced macro MSE by 4.07%. Full reduced it by 7.75% relative to the same baseline. The losses therefore provide a substantial benefit even with ordinary patch inputs.

The clearest finding concerns the relationship between point accuracy and statistical fidelity. In all nine paired comparisons (three datasets, three training repetitions each), the structured model trained without statistical losses had lower MSE and MAE than Raw-patch, yet larger marginal, spectral, and lag-correlation errors. Adding the statistical losses to that same structured model improved all five measures in every pair.

We also agree that Full and PS Loss had similar point errors, especially macro MAE (0.2644 and 0.2645, respectively). The FreDF frequency term achieved lower macro spectral error than Full (0.1753 versus 0.1797). These results show that the choice of objective matters for which forecast properties are preserved. We did not evaluate DBLoss or DistDF, or compare the drift module with RevIN, SAN, or FAN.

Checkpoint and pretraining overlap (Q2). The supplementary benchmark code pins google/timesfm-3.0-pytorch to revision 43046b85ec22d584a13f8098c2ed39c889e129c2. We cannot provide a verified list of benchmark test periods included in that checkpoint’s pretraining. The controlled comparisons use a shared pretrained initialization, so their conclusions concern the effect of adaptation choices under that initialization.

Hyperparameters and training batch size (Q3). We checked the configuration generator and loss implementation in the supplementary code. The Full benchmark defaults, in the paper’s notation, are:

ParameterValue
Loss weights (sep, marg, freq, corr, cal)0.01, 0.10, 0.05, 0.10, 0.01
Projection directions K, PACF weight η, calibration decay κ32, 1.0, 1.0
EMA update rate β0.01 on the current batch
Spectral bins Q, pooling width w16, 8
Base scale floor σ_min, correlation lag cap D_corr0.0001, 24
Training batch size16 for Electricity, Traffic, PEMS03, and PEMS08. 32 for the other benchmark datasets.

The effective separation weight is 0.01 because the implementation multiplies the outer decomposition weight (0.1) by the inner separation weight (0.1). The code also allows dataset-specific validation selections to override the marginal, frequency, and correlation weights. The available records establish the defaults above, while the final overrides, if any, remain unverified for individual reported runs.

Reviewer_AWscOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes StatTS-FM, a framework for supervised time-series foundation-model adaptation that embeds statistical structure as an explicit inductive bias throughout the forecasting pipeline. Addressing the fundamental limitation that standard pointwise error metrics (e.g., MSE/MAE) fail to preserve critical trajectory properties, the method explicitly constrains the forecast mean across its empirical value distribution, spectral composition, and temporal lag dependence. The framework integrates three main mechanisms: a statistical input representation decomposing history into trend, seasonal, and remainder patches alongside ACF/PACF lag summary tokens; a drift-conditioned adjustment module that shifts predictive location and scale based on historical mean, variance, and spectral deviations relative to training EMA references; and a multi-part objective combining Gaussian negative log-likelihood (NLL) with sliced-Wasserstein marginal , log spectral power, and lag correlation losses applied directly to the forecast mean. Evaluated by fine-tuning a frozen TimesFM-3 backbone using LoRA adapters, the paper establishes theoretical generalization bounds and demonstrates empirical gains in point forecasting accuracy and statistical fidelity across twelve benchmark datasets.

Critical strengths
  1. The paper effectively addresses the gap between pointwise accuracy (MSE/MAE) and structural fidelity in time-series foundation model fine-tuning. The integration of trend/seasonal/remainder decomposition, ACF/PACF lag features, and historical drift adjustment relative to EMA references, provides an explicit inductive bias for adapting frozen foundation model backbones.
  2. Across 12 datasets, the proposed method achieves superior point-forecasting performance compared to both zero-shot foundation models and supervised baselines. Notably, in controlled same-backbone experiments on TimesFM-3, StatTS-FM achieves a 7.88% reduction in macro MSE over standard raw-patch adaptation while simultaneously improving value distribution (marginal Wasserstein), spectral composition, and lag correlation errors.
Critical weaknesses
  1. Although the title, abstract, and introduction position StatTS-FM as a general framework for supervised foundation model adaptation, all empirical evaluations are strictly limited to a single backbone: TimesFM-3. As the authors acknowledge in Section 6, transfer to other backbones remains an open question. Without demonstrating performance across distinct foundation model architectures (e.g., Chronos, Moirai, or Timer), it is unproven whether the framework's benefits generalize universally or if they depend on TimesFM-3's specific patch/decoder structure.
  2. In Section 2, the authors cite several specialized time-series loss functions designed to enforce structural and frequency constraints (e.g., FreDF, DBLoss, DistDF, and Patch-wise Structural Loss). However, in Section 5.2 and Table 10, the proposed loss formulation is only compared against standard NLL/MSE. The lack of controlled experiments comparing StatTS-FM against these existing state-of-the-art loss functions under the same backbone and patch configuration leaves it unclear whether the proposed statistical supervision offers a genuine advantage over established structural loss formulations.
  3. The manuscript is difficult to follow, and the introduction lacks a structured breakdown of the framework's core contributions. It is unclear from the narrative which specific module (input representation, drift MLPs, or statistical losses) primarily resolves non-stationary statistical drift versus added parameter capacity. Additionally, the text relies on ambiguous jargon (e.g., "readout") and dense multi-module notation (involving six loss terms, EMA reference statistics, and lag tokens) without establishing clear formal definitions upfront.
Questions for authors
  1. Given that StatTS-FM is introduced as a general framework for adapting time-series foundation models, why did the authors not provided empirical results adapting several other time series foundation model architectures (e.g., Chronos, Moirai, or Timer) to demonstrate that the framework's effectiveness generalizes beyond TimesFM-3? Because, each of the foundational model kind of fundamentally different and there are recent new foundation model like UniShape which actually add statistical components to patch embeddings?
  2. How does the proposed combination of statistical losses compare against existing state-of-the-art structural and frequency loss functions cited in Section 2 (e.g., FreDF, DBLoss, or Patch-wise Structural Loss) when evaluated under the exact same TimesFM-3 backbone and input representation? And it is not clear why those loss functions cannot do similar work both theoretically and empirically when incorporated to a time series foundation model, as these existing loss functions already deals with statistical properties of time series in some way.
  3. Table 10 evaluates Raw-patch, w/o stat. losses, and full StatTS-FM. Could the authors isolate the individual performance contribution of the Statistical Input Representation (component decomposition + lag tokens) versus the Drift-Conditioned Adjustment module under standard NLL/MSE supervision?
  4. Could the authors provide a clear formal definition and architectural schematic of the "readout" mechanism mentioned in Section 3.2, and clarify how the training EMA references for drift calculation are configured across diverse multi-dataset fine-tuning regimes?
  5. What is the possibility of developing a standalone new foundation models using the framework proposed by the authors of this paper? What are the limitation around it? Because, if current standard training paradigm of time series foundation model is not good enough to capture statistical inductive bias then better foundation models should be trained from scratch, in that case we do not need adaptation.
3 comments

We thank the reviewer for recognizing the distinction between point accuracy and statistical fidelity, and for identifying where the contribution, architecture, and scope needed clarification.

Q1. Why only TimesFM-3, and how does the method relate to UniShape?

Our study addresses supervised adaptation when pretrained forecasting weights are already available. We use a shared TimesFM-3 checkpoint to compare adaptation choices while holding pretrained initialization fixed. This design supports a controlled study within one backbone; it does not demonstrate effectiveness across different foundation-model architectures. We have restricted the empirical claims accordingly. Transfer to Chronos, Moirai, Timer, and other backbones remains untested.

As specified in Section 3.2, we replace TimesFM-3's native input and output modules and adapt its frozen Transformer stack through Q/V LoRA. Although trajectory losses can be applied to other forecast outputs, transferring the complete framework requires compatible token, masking, and output interfaces, followed by validation on each backbone.

Related Work now acknowledges UniShape's use of local subsequence means and standard deviations in multiscale shape tokens for classification pretraining. Statistical input features therefore have prior precedent. Our study concerns forecasting adaptation and the relationship between historical statistical representations, supervision of the adjusted mean, and the resulting accuracy–fidelity trade-off.

Q2. Can existing structural losses provide similar empirical and theoretical benefits?

The additional controlled results show that established objectives also provide useful empirical supervision. We evaluated PS Loss and the FreDF frequency term within the same structured architecture, retaining Gaussian NLL and both auxiliary regularizers while replacing the three statistical penalties. This comparison covers ETTh1, Electricity, and Weather at L=H=96L=H=96, with three paired repetitions per variant and dataset.

Macro MSE is 0.2138 for Full, 0.2153 for PS Loss, and 0.2161 for the FreDF frequency term. Both alternatives improve all five metrics over no-stat in every paired run. Full and PS Loss have nearly identical macro MAE, 0.2644 versus 0.2645; FreDF has lower macro Spec-Err, 0.1753 versus 0.1797. PS Loss also has lower ETTh1 MAE and Corr-Err. The FreDF row tests its frequency term under the common NLL objective, rather than its complete original formulation. DBLoss and DistDF have not been evaluated under this protocol.

Our penalties target projected within-window value distributions, normalized log-power spectra, and selected ACF/PACF lags. Section 4 analyzes the projected location and spread controlled by the marginal penalty and generalization under stated dependence and moment assumptions. This analysis does not exclude analogous benefits or analyses for other objectives, nor does it establish an empirical ranking of losses. We have clarified the contribution around the evaluated adaptation design and its measured trade-offs.

Q3 and Weakness 3. Individual module contributions, capacity, and presentation.

The revised Introduction and Method distinguish three roles: statistical tokens expose historical component and lag structure; drift adjustment modifies predictive location and scale using deviations from training references; and statistical losses supervise the final adjusted mean. The objective comprises one likelihood term, three fidelity penalties, and two auxiliary regularizers.

Full versus no-stat holds the complete architecture, Gaussian NLL, and separation/adjustment regularizers fixed while adding the three statistical penalties. It therefore evaluates statistical supervision at fixed architecture. The new Raw + Stat control shows that the penalties also improve ordinary patch adaptation, reducing macro MSE from 0.2318 to 0.2224; Full reaches 0.2138.

These controls do not isolate statistical inputs and drift separately under pure NLL/MSE supervision. The existing component removals retain applicable statistical losses, while removing decomposition or drift also changes architecture or capacity. The requested individual attribution therefore remains unresolved. Appendix J explicitly states this distinction, and we do not attribute the entire Full-versus-Raw improvement to one module.

Q4. Formal readout definition and EMA references.

Section 3.2 and Figure 2 define the readout as the mapping from Transformer output states to base Gaussian mean and scale. For variable cc, let hˉcU∈Rd\bar h_c^U\in\mathbb R^d average valid output states belonging to component U∈{T,S,R}U\in\{T,S,R\}, and let hˉc\bar h_c average all valid states, including lag tokens. Each head, indexed by ν∈{T,S,R,σ}\nu\in\{T,S,R,\sigma\}, has separate parameters shared across variables:

hν(z)=WνRMSNormν⁡(z)+bν.h_{\nu}(z)=W_{\nu}\operatorname{RMSNorm_{\nu}}(z)+b_{\nu}.

Here, Wν∈RH×dW_{\nu}\in\mathbb{R}^{H\times d} and bν∈RHb_{\nu}\in\mathbb{R}^{H}.

The component heads and scale head produce

\qquad \sigma^0_{:,c}=\operatorname{softplus}(h_\sigma(\bar h_c))+\sigma_{\min}.$$ The history-conditioned adjustment then gives $\mu=\mu^0+b$ and $\sigma=\sigma^0\odot\exp(a)$, where $b$ and $a$ are the location and log-scale corrections, with $a$ clipped elementwise to $[-5,5]$. Section 3.3 defines per-variable references for training-window mean, within-window variance, and normalized $Q$-bin spectrum, with shapes $C$, $C$, and $C\times Q$. The first training minibatch initializes them. Later batches compute drift against the stored state before the update $$\mathcal R_c\leftarrow(1-\beta)\mathcal R_c+\beta\bar s_c,$$ where $\bar s_c$ averages that batch's window statistics. In particular, the variance reference averages within-window variances. Validation and inference freeze these references; checkpoint restoration restores them with the model parameters. This defines the algorithm for a fixed variable space. Sharing references across datasets requires compatible variable identities and standardization. We cannot yet provide a verified account of the actual reference allocation in a joint multi-dataset fine-tuning regime or the selected $\beta$ and $Q$. Appendix G.2 records this limitation; the algorithmic clarification therefore does not fully resolve the requested run-configuration question. **Q5. Could the framework be used to pretrain a new foundation model from scratch?** The statistical representations and future-trajectory losses could also be used in pretraining. Such a study would require a heterogeneous corpus, choices about weighting domains and statistics across scales and sampling frequencies, and reference management compatible with those domains. Matched data and compute budgets would be needed to attribute gains to the statistical design. Statistically informed pretraining and downstream adaptation can be complementary. Pretraining learns transferable behavior across domains, while adaptation uses the target domain's observed data and forecasting objective. Our results do not establish that pretraining is intrinsically unable to capture statistical structure. Whether a particular pretraining design eliminates the benefit of downstream adaptation remains an empirical question. Our experiments address the setting in which pretrained weights are already available; we have not conducted pretraining from scratch. The Conclusion and Appendix J, “Scope of Adaptation and Statistical Pretraining,” now discuss these distinctions.

Dear Reviewer AWsc,

Thank you again for your review. In our response, we clarified how the prediction heads map the backbone’s hidden states to forecasts. Table 2 of the revised manuscript also compares the same architecture with and without the statistical losses across three training repetitions.

If these clarifications and results address your concerns, would you be willing to reconsider your overall rating? We would be happy to discuss any questions that remain.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.