acceptodds
Under review as a conference paper at ICLR 2027

A Unified Scaling Law for Time Series Foundation Models

Abstract

We develop a **Unified Scaling Law** and a **Unified Theory of Time Series Learning** to understand how model capacity and historical information support forecasting. Across different lookback lengths and forecast horizons, we analyze 18,768 experimental cells from 21 checkpoints on 23 dataset-frequency tasks spanning six domains. Our empirical methodology integrates local resource relations into a parsimonious, fitted five-parameter law: capacity gains increase with history, context gains diminish toward saturation, and horizon effects enter as a common shift. Fitted without Toto 2.0, the law predicts its horizon-averaged capacity-scaling curves with mean absolute percentage errors of 1.09% and 1.50% at input lengths 2048 and 4096. To understand how history supports prediction, our learning theory uses Gaussian regression to analyze rule identification and predictive capability. We hypothesize that full-shot models learn by *accumulating information in weights*, while frozen time series foundation models (TSFMs) use history by *extracting information through activations*. Matched-history comparisons establish the predictive value of additional history. Controlled parameter exchanges and activation interventions provide evidence that history-derived rule information can be retained, reused across queries, and used to recover a contribution to long-context prediction. Together, these findings inform capacity scaling, context allocation, and the development of models that retain and apply historical rules.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series, Time-Series Foundation Models, Scaling Laws, Test-Time Scaling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XuxbOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an inference-time scaling law for time series foundation models, and a theory of time series learning. The former relates a normalized MASE error to context length, target horizon, and parameter count, while the latter argues that specialized models learn by storing information about the target task in the weights, while foundation models do so by storing that information in the activations as a function of the input context. Experimental results show the proposed scaling law can explain the relationship between the considered variables to an extent, and it compares favorably to some more complex alternatives. The paper supports claims about the learning theory with experiments that show task information can be partially transfered by acting on model weights or activations, and that task-specific tuning can improve performance at fixed input lengths.

Critical strengths
  • The work studies the compelling problem of understanding inference-time empirical relationships connecting time series foundation models performance with their parameter, input and horizon scale. Advancing scientific understanding in this direction can help practicioners pick among pre-trained models. Overall, the structure of the proposed scaling laws appears in line with the general understanding that additional context helps performance, longer target horizons are harder to predict, and increasing model size can be beneficial. Previous work does not seem to have directly focused on the specific type of scaling law proposed in this paper.
Critical weaknesses
  • The proposed scaling law, while reasonable in its structure, is not accompanied by sufficiently convincing empirical evidence to support the claims of a general scaling law. In particular, the predicted error does not exhibit a strong linear correlation to the true error when evaluated out-of-sample (Figure 3), and the law appears to systematically underestimate error on the Toto evaluation for shorter input lengths (Figure 7). Moreover, extrapolation does not seem to be sufficiently tested: its unclear whether the law extrapolates well to held-out input lengths or horizons, and extrapolation to held-out models or parameter counts is only partially tested through the Toto-specific study. As such, it is unclear whether the scaling law can reliably guide general decision making. The authors claim to use an Occam's Razor approach to select a parsimonious law parametrization, yet results in Table 5, suggest that the contribution of the capacity parameter is marginal and possibly detrimental, hence it is unclear if a parsimonious law should include it. For instance, in Figure 5b, the law only predicts a marginal change in the preferable input length across parameter counts that differ by orders of magnitude. Finally, the claim that the proposed law can support decision making for model building seems at least partially unsubstantiated, as the law deals with inference time relationships, and completely neglects the influence of training budget in terms of compute and data.
  • I do not believe the evidence provided in this work supports the claims of development of a unified theory of time series learning. First, the theory itself appears mostly incremental, as the intuition that foundation models may infer the task from the input context is already discussed in prior works, and the principle that task-specific models embed the forecasting rule in their weights is the fundamental way in which they are intended to work. The link to Gaussian regression is unclear, as the paper does not demonstrate a connection between that and how the studied models solve the forecasting task, hence I am unsure what it provides beyond a conceptual intuition. In regards to experimental evidence: swapping task-specific model parameters (Sec. 6.3) amounts essentially to swapping models, so the accuracy results after the swap are expected. The experiment with activation patching (Sec. 6.4) is suggestive, yet the breadth of the evaluation is narrow and the experiment seems to only support the claim that activations may contain information about the target task inferred from context, which is a plausibly expected result and not sufficiently strong to support claims of a unified learning theory.
Questions for authors
  • Could the authors explain why the law including the capacity dependency was preferred, given the parsimonious approach?
  • How does the law account for "parameter count" not having a homogeneous meaning across different models?
  • Does the law mix error on both univariate and multivariate tasks? If so, how does it account for the discrepancy in the amount of information for a multivariate vs univariate task at the same input length?
  • Could the authors explain how the law can support model building choices given the lack of relationship to training time effects (e.g., data budget, maximum pre-training length and horizon length)?
  • Can the authors explain the purpose of the Gaussian regression discussion in Sec. 6.1?
  • Can the authors explain how the experiments in Sec. 6 support the proposed unified learning theory being more than an intuition which is in line with intuitions present in prior works?

Minor questions:

  • At line 916 you mention "an aggregated long-context coordinate can include datasets with shorter available histories". Does this mean that error at longer context may potentially include samples that had shorter actual context?
  • Is there a reason for using the ratio of MASE as score, given the MASE is already normalized by the naive error? Would the results change if a probabilistic score like CRPS was used?
  • Why were in Figure 2 the coefficients not re-fitted for the alternative laws?
  • Appendix C.6 does not seem to report any actual result for the horizon-interaction extension.
Reviewer_5ttmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper fits a five-parameter law, log⁡R=c+αlog⁡H−γlog⁡L−βlog⁡Nlog⁡L+q(log⁡L)2\log R = c + \alpha\log H - \gamma\log L - \beta\log N\log L + q(\log L)^2, to the Seasonal-Naive-normalized MASE of 21 released TSFM checkpoints on 23 GIFT-Eval tasks, varying the input length LL (128–8192) and the horizon HH. The law is validated on public GIFT-Eval results and with a leave-Toto-2.0-out test, where it predicts the horizon-averaged capacity curves with 1.09% and 1.50% MAPE.

A second part proposes a "Unified Theory of Time Series Learning". It consists of a Gaussian regression model of how history identifies a rule, together with experiments contrasting full-shot training with frozen TSFMs: matched-history comparisons, parameter exchange on synthetic processes, and activation patching in TimesFM 2.5 and Chronos 2.

Critical strengths
  1. Large, well-documented evaluation. The paper evaluates 18,768 cells on shared-origin grids over LL and HH, normalizes errors by Seasonal Naive, and uses dataset-level bootstrap and dataset-held-out validation. The resulting response panel is useful in its own right.
  2. The leave-family-out test is the right kind of check. The Toto 2.0 slope predicted without any Toto data is close to the observed one.
  3. The synthetic rule-identification setup is clean. Under both rules the final 96 observations are identical, and the design includes an oracle reference as well as opposite-rule, random, and shuffled-history controls.
Critical weaknesses
  1. The capacity term is small and adds no predictive value. With β=0.001846\beta = 0.001846, a tenfold increase in parameters lowers MASE by about 3% (2.6% at L=512L = 512, 3.8% at L=8192L = 8192), and shifts L⋆L_\star by only about 10% (from about 3.8k at 10M to 4.6k at 1B). Table 5 shows that dropping capacity entirely (β=0\beta = 0) gives essentially the same MAPE:

    • Controlled: 9.96% vs. 10.13% for the full law.
    • Public: 13.01% vs. 12.82%.
    • Toto: 11.84% vs. 11.53%.

    The paired intervals for these differences include zero. Almost all of the law's predictive content therefore comes from LL and HH. This does not support calling it a scaling law for TSFMs, nor the claim in Section 7.1 that it provides "empirical support for pretraining larger models".

  2. The capacity–history coupling is a consequence of the parameterization, not a finding. Eq. 2 fixes the capacity elasticity at −βlog⁡L-\beta\log L with zero intercept. Because log⁡L\log L only spans 4.9–9.0 on the grid, a constant capacity effect would also be fitted as β>0\beta > 0. Several observations point to this:

    • The U+N variant, which frees the intercept, has the same validation MAPE (11.78% vs. 11.77%).
    • The observed medians in Figure 2c look nearly flat, with IQRs wider than the gap between the Unified and additive curves.
    • The capacity slopes come from only four families (K = 8 pooled targets). In the leave-Toto-out fit they come from three: Moirai 1 (which mixes a 1.0 checkpoint with 1.1 checkpoints), Chronos 2, and Chronos-Bolt. These span 14–311M, and the fitted slope is then extrapolated to Toto's 4M–2.5B range.

    The 1.09%/1.50% Toto numbers summarize five horizon-averaged points per curve, so they effectively test a single slope.

  3. The proposed law is not a scaling law in the usual sense.

    • No training variable enters the law. No model is trained. The law uses parameter count NN and two inference-time quantities, LL and HH; training data, compute, and training steps never appear.
    • It cannot be used to plan training. Scaling laws are valuable because they are fitted on small, cheap training runs and then predict the loss of larger or longer runs before those runs are made. This is what makes them a tool for allocating parameters, data, and compute (Kaplan et al., 2020; Hoffmann et al., 2022). The proposed law cannot be used this way: it describes how the evaluation error of already-trained checkpoints varies with context length and horizon.
    • Using public checkpoints is acceptable, but family effects must then be modeled. Observational scaling laws (Ruan et al., 2024) also avoid training, but they explicitly model differences between model families in how efficiently they turn training compute into capability. Here, all 21 checkpoints share a single intercept and are distinguished only by NN, although they differ in pretraining data, recipe, architecture, and possibly overlap with GIFT-Eval.
    • Consequences of the single intercept. The law predicts the same error for TimesFM 1.0 and TimesFM 2.5 (both 200M active parameters) at every (L,H)(L, H). Figure 3a shows predictions spanning a much narrower range than the observed log⁡R\log R, which suggests that family effects carry much of the variance; the paper does not quantify this.
    • Novelty relative to prior work. Shi et al. (2024) and Yao et al. (2025) study scaling with controlled training. Relative to them, the new elements here are the horizon term and the panel of released checkpoints.
    • Overreaching recommendations. The advice to model builders in Section 7.1 ("empirical support for pretraining larger models") goes beyond what this law can support. The work is better framed as an empirical study of context and horizon effects in released TSFMs.
  4. The "Unified Theory" is standard and does not explain the law. Section 6.1 and Appendix D apply conjugate Gaussian regression (posterior variance τ2/(1+Lντ2)\tau^2/(1 + L\nu\tau^2)) and derive an elementary monotonicity result. This model gives an error that is non-increasing in LL and contains no NN, so it can produce neither the fitted minimum in log⁡L\log L nor the β\beta term. The distinction between in-weight and in-context learning is already established in the cited work. The supporting experiments mostly confirm what one would expect:

    • Parameter exchange. Swapping parameters between models trained on opposite rules moves predictions toward the donor rule, as any supervised model would.
    • Activation patching. The effects are modest, measured on synthetic data, and obtained at development-selected sites. For Chronos 2, direct transfer gives 0.28776 vs. a baseline of 0.28774, which is worse than the random control (0.28710; Table 9).
  5. The full-shot comparisons are not matched, and two protocols disagree.

    • Unequal inputs. Figure 4 gives TSFMs length-LL inputs but restricts full-shot models to 96-step inputs. Table 6 shows that a 512-step input alone lowers DLinear's MASE from about 1.04 to 0.89.
    • Inconsistent numbers. For the same setting (input 96, H=96H = 96, history 8192), DLinear scores 0.931 in Figure 4 but 1.040 in Table 6, and PatchTST scores 0.833 vs. 0.914. These gaps are larger than most of the effects interpreted in Section 6.2.
    • The adaptation result is overstated. The reported 68% reduction of the short–long gap after adapting TimesFM is driven mainly by the frozen 96-step configuration, which is worse than Seasonal Naive (MASE 1.043).
  6. Presentation.

    • The text relies heavily on coined terms such as "NHL coordinates", "Grid-first selection", and "carrier-only".
    • Key results are left to the released code. The per-fold results of C.5 are not given, and C.6 reports no numbers at all.
    • The six-decimal coefficients in Eq. 7 imply a precision that the bootstrap interval for β\beta ([0.0009, 0.0032]) does not support.
Questions for authors
  1. What share of the variance in log⁡R\log R do NN, LL, and HH explain, and how does this change when family-specific intercepts are added?
  2. Please report the capacity elasticity estimated separately at each LL level, and the λ\lambda and β\beta estimates of the U+N form, all with confidence intervals.
  3. What are the Toto capacity-curve MAPEs of the capacity-independent and constant forms?
  4. Why do Figure 4 and Table 6 disagree for the same full-shot setting? Do TSFMs still win in Figure 4 when full-shot models receive inputs of at least 512 steps?
  5. How is overlap between each checkpoint's pretraining data and the 23 tasks handled?
  6. Toto 2.0 is the only family with five sizes. Can the law be fitted on its three smallest checkpoints (4M–313M) and used to predict the 1B and 2.5B results? Small-to-large extrapolation of this kind is what would justify calling the result a scaling law.
Reviewer_1evVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes scaling law for time series foundation models, relating forecasting error to model capacity, input length, and forecast horizon. The law is fitted using evaluations across models and datasets, and tested on public benchmark results and a held-out model family. The paper also introduces a theory of how models use historical information, combining Gaussian regression analysis with experiments examining how rule information is retained in model weights and extracted through activations.

Critical strengths

Main strength of this work is in the breadth of the experiments across models, datasets, input lengths, and forecast horizons. Experiments on history use also include reasonable controls. Holding queries fixed and comparing matched-rule, opposite-rule, and shuffled-history interventions provides useful evidence that the tested models can extract and reuse historical rule information.

Critical weaknesses

My main concern is that the predictive strength of capacity, is not convincingly demonstrated. In Table 5, removing capacity dependence gives very similar errors, and the reported intervals for the differences include zero. I also find the “Unified Theory of Time Series Learning” framing too broad. Gaussian analysis provides qualitative intuition, but its connection to neural model capacity and the empirical scaling law remains unclear. The experiments support specific findings about the use of historical context, but do not seem sufficient to establish such a general theory.

Questions for authors

Table 5 does not establish improvement from including capacity. Can you add the difference in MAPE between the unified and capacity-independent laws separately by input length on held-out Toto?

Is Gaussian analysis implying the proposed scaling law, or does it only provide qualitative intuition? Please clarify and separate out the results established under the Gaussian assumptions from the claims about neural models. If the connection is only qualitative, could the “unified theory” framing be narrowed?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.