acceptodds
Under review as a conference paper at ICLR 2027

The Price of Generality in Time Series Foundation Models: GRAIN for Yearly Forecasting

Abstract

Foundation models are typically trained as generalists, optimised for average predictive performance across broad and heterogeneous tasks. Motivated by the classical James-Stein phenomenon, we develop a theory that explains why such models can remain suboptimal on specific domains despite strong aggregate performance. We interpret pretraining as empirical Bayes-risk minimisation under a corpus-induced task prior, and define the *price of generality* as the excess risk incurred on a target domain when using a rule optimised for a broader population. While the argument applies broadly to foundation models, it has practical relevance mostly for cases with a limited amount of information in the prompt. Consequently, we study yearly forecasting as a concrete application of this theory. Yearly series are practically important but systematically underrepresented in existing pretraining corpora, and their short histories make predictions more sensitive to prior misspecification. Following our developed theory, we introduce GRAIN, a specialist prior-data fitted network trained on simulations from a posterior-informed prior estimated from real yearly series. On the yearly subset of the GIFT-Eval, GRAIN outperforms generalist foundation models and statistical baselines without data leakage, supporting the benefit of aligning the training prior with the target task population.

Primary area:
learning on time series and dynamical systems
Keywords:
Time series forecasting, Foundation models, Bayesian decision theory, Yearly forecasting, James-Stein

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_4GuFOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates why general-purpose time series foundation models may remain suboptimal in specialized forecasting domains, interpreting this limitation through Bayesian decision theory and the James–Stein phenomenon as a price of generality arising from prior mismatch. Motivated by this perspective, the authors propose GRAIN, a specialist model for yearly forecasting that constructs a posterior-informed task prior from Bayesian LGT models fitted to real yearly series and trains a prior-data fitted network on the resulting synthetic data. Experiments on the yearly subset of GIFT-Eval demonstrate competitive MASE and CRPS performance against generalist foundation models and statistical baselines, although the empirical evidence does not fully establish that the improvements are specifically driven by the proposed prior-mismatch mechanism.

Critical strengths
  1. Important research question with an insightful theoretical perspective. The paper challenges the implicit assumption that strong aggregate performance of general-purpose foundation models necessarily translates into optimal performance across individual domains. The Bayesian decision-theoretic interpretation, particularly the analysis of posterior expert weights and predictive disagreement, provides a useful perspective on why prior mismatch may be especially consequential for short-history forecasting.

  2. Statistically motivated specialist prior construction. GRAIN constructs its synthetic training distribution using parameter posteriors from Bayesian LGT models fitted to real yearly series, rather than relying entirely on a manually specified simulation prior. Combining this posterior-informed prior with the PFN paradigm provides a reasonable approach to incorporating domain-specific inductive biases while enabling efficient neural inference.

  3. Competitive empirical results. GRAIN achieves the best normalized MASE and CRPS among models not flagged for potential test-data leakage on the GIFT-Eval yearly benchmark. Additional comparisons with Bayesian LGT and classical statistical forecasting methods illustrate the potential accuracy–inference efficiency trade-off of the proposed approach.

Critical weaknesses
  1. The theoretical novelty beyond established Bayesian decision theory remains insufficiently clarified. The paper defines the price of generality as the target-domain excess risk of a generalist rule relative to a specialist oracle. However, its non-negativity follows directly from the definition of the target-optimal oracle, while the observation that aggregate risk improvements need not translate into improvements for every subpopulation is already well established through the James–Stein phenomenon and Bayesian decision theory. Although the two-expert posterior-risk decomposition provides useful intuition, it relies on idealized assumptions, including a correctly specified specialist model and squared-error loss. The paper should more clearly distinguish established statistical principles from its genuinely new theoretical contributions and explain what additional predictions the framework makes about practical neural foundation models.

  2. Insufficient controlled evidence for the central prior-mismatch hypothesis. The main argument is that generalist pretraining induces a task prior that can be poorly aligned with a target domain, particularly when short histories provide insufficient evidence for identifying the appropriate forecasting regime. However, the empirical evaluation primarily establishes that GRAIN performs well on yearly forecasting benchmarks and does not directly manipulate the training prior, target-task mixture weight, or context length. Consequently, the reported gains cannot be uniquely attributed to the proposed mechanism rather than differences in architecture, model capacity, training objectives, or synthetic data generation. Controlled experiments using the same PFN architecture under generalist, specialist, and mixed priors, with varying context lengths, would provide much stronger evidence.

  3. The unique contribution of the posterior-informed prior is not sufficiently isolated. GRAIN generates synthetic training data from parameter posteriors of Bayesian LGT models fitted to real yearly series, but the paper lacks direct comparisons against the original LGT prior, simpler empirical parameter distributions, or alternative simulation priors. Since LGT already provides strong forecasting performance in the yearly domain, it remains unclear whether GRAIN's effectiveness is specifically attributable to posterior-informed prior estimation or primarily to the domain-specific inductive biases inherited from LGT. Furthermore, the contributions of dual-scale patching and trend smoothness regularization are not sufficiently isolated through ablation studies.

  4. Limited evaluation scope and insufficient statistical uncertainty analysis. The primary generalization evidence relies on the yearly subset of a single benchmark, GIFT-Eval. While GRAIN ranks first among the unflagged models, its improvement over the strongest competitors is relatively small on some metrics, particularly normalized CRPS (approximately 0.754 versus 0.758). The paper does not provide sufficient across-series confidence intervals, statistical significance tests, or training-seed variability to establish the robustness of these differences. Additional evaluations on independent yearly datasets, varied context lengths, and forecasting horizons would strengthen the generality of the conclusions.

  5. The interpretation of training and evaluation data provenance requires care. The main M3-to-M4 evaluation appropriately separates the series used for prior estimation from the M4 evaluation series. However, the supplementary M3 evaluation uses the same dataset family employed to construct the training prior and therefore should be interpreted as an in-domain approximation and efficiency analysis rather than independent evidence of generalization. Additionally, benchmark flags indicating potential test-data leakage for certain competing models should be distinguished from independently verified leakage when interpreting the reported rankings.

Questions for authors

Q1. Can the authors directly validate the predicted price of generality under controlled prior mismatch? In particular, can the same forecasting architecture be trained with different mixtures of yearly and non-yearly task priors, while varying the observed context length? This would test whether increasing prior mismatch and reducing historical information lead to the excess risk predicted by the theoretical analysis.

Q2. How much of GRAIN's performance comes specifically from posterior-informed prior estimation? Could the authors compare the current approach with models trained using the original LGT prior, an empirical distribution of fitted parameters, and alternative synthetic time series priors? Such comparisons would help distinguish the contribution of posterior-informed prior construction from the already strong inductive bias of LGT.

Q3. Are the reported improvements statistically robust and transferable beyond M4 yearly forecasting? Could the authors provide paired bootstrap confidence intervals across evaluation series, multiple training seeds, and additional results on independent yearly datasets or different history lengths? Given the small CRPS gap relative to the strongest unflagged baseline, these analyses are important for assessing the robustness and broader significance of the empirical gains.

Reviewer_MVTpOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper identifies an important and well-recognized issue in time-series foundation modeling and forecasting, namely that such foundation models do well in aggregate across different time-series forecasting tasks, but can be suboptimal in specific cases. Using yearly forecasting as a motivating example, and relying on theoretical ideas from empirical Bayes estimation and the James-Stein phenomenon, the authors propose (as an illustrative example of their approach) a specialist prior-data fitted network and evaluate its prediction and calibration performance on a well-known benchmark dataset (Gift-Eval) and compare it to other well-known time-series forecasting baselines.

Critical strengths
  • It addresses an important problem in time-series foundation models: although the focus on yearly forecasting could be viewed as somewhat narrow, the authors do a good job of motivating this particular forecasting task as being relevant to time-series foundation models in general

  • The writing is clear

Critical weaknesses
  • A main weakness of the paper is the novelty of the contribution. While I very much appreciate the discussion of the James-Stein and empirical Bayesian interpretations in Section 3, this discussion takes up pages 3 to 7 (i.e., most of) the paper, leaving only the last 2 pages for the discussion of the proposed methodology and experiments. This would be fine if pages 3 to 7 had new theoretical results, but from my reading it seems that the results here are quite general and broadly known, and while I appreciate the discussion of these results in the context of time-series foundation models, it seems difficult to argue that this is a novel contribution at the level we might wish to see for an ICLR paper. To address this issue (which, to be clear, would not be possible in author response, but potentially in a future revision of the paper), the authors could perhaps upweight the empirical aspect of the paper, e.g., by including a second aspect (other than yearly forecasting) where the proposed general theory would be applicable and where a specialist foundation model could be proposed and evaluated. A complementary option would be to strengthen the theoretical results, building on Proposition 1 for example. The characterization of the posterior weights w_T(y) could be interesting for example in terms of mathematically characterizing how these weights could depend on the series length n, characteristics of the prior, and other factors.

  • Another weakness is the experimental results, for a few reasons:

  1. the baselines appear to be missing relevant and modern SOA foundation models, specifically: Toto 2.0, TiRex-2, Tempo-PFN. Its important to include these for a full assessment of the proposed method.
  2. experiments are limited to a single table with aggregated results across multiple time-series (albeit with many models). A more detailed analysis here would be useful. For example, were there any systematic patterns observed for the types of time-series where GRAIN was outperformed by other models
  3. no error bars or sensitivity results are reported, e.g., how sensitive is the GRAIN methodology to the specific choice of prior parametrization used in the experiments?
Questions for authors

(Note to authors: these are not questions that could materially affect my assessment as a reviewer, but instead are just suggestions for future revision of the work based on my notes - I did not know where else to put this information in the review)

  • climate data (e.g.,events that occur on annual scales, such as strength of El Nino or start of monsoon seasons) might also be worth mentioning at the top of page 2 as a good motivating example for yearly forecasting

  • the Related work seems a little bit dated (for time-series foundation models): no papers from 2026 listed for example

  • also for Related work, papers on Bayesian in-context-learning, might be interesting to include in the discussion (even though they come from a different perspective), e.g., Raventos et al NeurIPS 2023

  • if the paper doesn't get into ICLR, the authors might consider submission to the International Journal of Forecasting, perhaps with expanded experimental results: my guess is that readers of that journal might find the topic of this paper to be of significant interest.

Reviewer_uKyXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the "price of generality" in general-purpose time-series foundation models—specifically addressing how models optimized across heterogeneous task populations can remain suboptimal when deployed on specific, specialized domains. Grounded in a Bayesian decision-theoretic framework, the authors introduce GRAIN, a domain-specialist model tailored specifically for low-frequency (yearly) time-series forecasting. Experimental results demonstrate that GRAIN achieves competitive forecasting accuracy and computational efficiency relative to generalist baseline models.

Critical strengths
  1. Well-Motivated Problem Framing: The paper targets an important limitation of generalist time-series foundation models: aggregate performance metrics across broad benchmarks can obscure poor performance on specific, underrepresented domains—especially those characterized by short context lengths like yearly series.

  2. Theoretical Grounding: The paper provides formal mathematical derivations within a Bayesian decision-theoretic framework to justify the design of a domain-specialist model, providing solid theoretical support for the approach.

Critical weaknesses
  1. Limited Empirical Evaluation Scope: The experimental validation is insufficient to demonstrate a generalized advantage in yearly time-series forecasting. Beyond the specific dataset utilized to estimate the Bayesian prior, external evaluation is restricted solely to the M4 Yearly dataset (the only yearly evaluation task present in GIFT-Eval). Broader evaluation across additional low-frequency benchmarks is necessary to confirm general applicability.

  2. Confounded Components and Lack of Ablation: The individual contributions of the proposed synthetic data generation pipeline and model architecture are not clearly isolated:

    • Synthetic Data Impact: There are no controlled experiments comparing training on the proposed synthetic data versus real historical data or alternative data synthesis priors while holding the model architecture constant.
    • Architectural Ablation: The paper lacks architectural ablations (e.g., assessing the triple-representation fusion module shown in Figure 2), making it difficult to discern whether empirical gains stem from the synthetic prior or architectural choices.
Questions for authors
  1. Synthetic Data and Architectural Isolation: Does GRAIN maintain its empirical performance advantage across additional low-frequency/yearly datasets beyond M4? Furthermore, under a fixed model architecture, how does the proposed synthetic data generator perform against (i) training purely on real historical data, and (ii) alternative synthetic data generation priors?

  2. Actionability for Generalist Foundation Models: How can the "price of generality" framework or the proposed synthetic data generator be leveraged to improve universal generalist time-series foundation models? For instance, does augmenting the pre-training corpora of generalist models (e.g., Moirai, Chronos) with the proposed synthetic data resolve their underperformance on yearly forecasting tasks without degrading overall generality?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.