acceptodds
Under review as a conference paper at ICLR 2027

Are Universal Time Series Foundation Models Based on a Category Error?

Abstract

Time series foundation models (TSFMs) achieve strong forecasts across diverse domains with little or no task-specific training. Yet what makes this transfer work, and why can adding data or longer histories sometimes fail to help? We investigate a possible *category error*: treating a shared numerical format as sufficient grounds for shared predictive structure and information. Time series are a container for observations from different processes; the same history can imply different futures when their generating rules remain unresolved. Under squared loss, averaging those rules suppresses conflicting components while retaining common structure, whereas informative context can make the rules distinguishable. We develop this concern into **four questions** about task symmetries, rule information, source mixing, and lightweight reuse of forecasting ability. Controlled interventions verify predicted costs of invalid sharing and show when additional information restores suppressed components. Small-scale pretraining establishes that the value of auxiliary data depends on its source and allocation. An exploratory distillation study finds that a tiny, history-conditioned operator model acquires useful predictive ability from few TSFM examples, with better performance when its training includes the target source. These findings give concrete grounds for reconsidering the assumptions behind universal forecasting and motivate models, losses, training procedures, and data construction that account for what can be shared and what must be distinguished.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series, Time-Series Foundation Models, Learning Theory

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TrfuOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines whether time-series foundation models transfer across domains because the underlying processes share predictive structure, or merely because the observations share a numerical format. The authors formalise the concern as a category error: a time series specifies an indexing convention and a value space, but not the law that governs its continuation, and when indistinguishable histories admit opposing futures, a predictor trained under squared loss targets the conditional mean, which averages the unresolved rules, retaining the component they share while suppressing the component they dispute. They give the claim testable form through controlled interventions: synthetic tasks in which the history law is held fixed while the continuation rule varies; a twelve-source, small-scale pretraining panel that contrasts source compositions and exposure budgets; and a distillation study in which a 4,590-parameter operator model is trained on the forecasts of six pretrained foundation models. The resulting conclusion is deliberately conditional: the value of sharing depends jointly on the task and on the information available to the model, so universality has to be assessed relative to a task rather than asserted over a data format.

Critical strengths

The paper turns an unclear debate into measurable quantities, and section 5.3 backs it with a matched-history three-arm design: the compatible and hidden-conflict arms share an identical history distribution and differ only in the auxiliary source's continuation rule, isolating a ~2,300× gap that can be attributed to rule conflict alone; the informative arm then injects source identity into the first 16 history values and shows the gap closing.

Critical weaknesses
  1. The central concept 'category error' is never defined: it appears several times with no criterion for deciding whether a given model commits it. Besides, terms like parameter sharing/distributional compatibility/predictive sufficiency appear in the paper but none is defined. Would be better adding a short concept table covering every term.
  2. All propositions are established under squared loss and conditional-mean targets. Several recent TSFMs are trained with quantile or distributional objectives, for which Eqs. 4–8 do not apply.
Questions for authors
  1. The central demonstration in section 5.3 is that when the prefix carries information that can reveal the hidden rule, error drops from 0.53 to 0.27, close to the theoretical floor reached when the rule is known, whereas a prefix of the same length with no information produces no improvement at all. But this effect appears only in a gating model that was handed the event times and waveform periods in advance: a generic Transformer of the same size and training budget shows essentially nothing, with an interaction of −0.00018 and a confidence interval that crosses zero, and shifting the event onsets by eight steps pulls the effect down from −0.2648 to −0.0096. The authors should therefore say which of two claims they are making: that information present in the data will be used by a model, or that it will be used only when the model is given the right structural assumptions. The wording of the main text suggests the first, but the evidence supports only the second, and the two have quite different practical implications, the first says that better information interfaces are worth building, the second says that an interface is not enough unless the structure is handed over as well.

  2. All four theoretical results rest on one premise: squared loss, with the conditional mean as the optimal prediction. But the paper is about time-series foundation models, and a good number of them — Chronos, for instance — are trained with quantile or distributional losses. The loss comparison in Table 2 shows how much this premise matters: with the same data, the same information and the same training budget, switching from squared to absolute loss moves the predicted amplitude from a shrunken 0.52 to close to full scale, improving absolute error while making squared error worse. In other words, changing the loss changes which component survives. The authors should therefore state whether their conclusion is "under squared loss, universality has these limits" or "this holds for all TSFMs". If it is the former, the title and abstract should carry that qualifier; if it is the latter, they need to explain how the quantile and distributional cases follow from the squared-loss results, and no such argument appears in the paper.

Reviewer_gQ6rOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper examines when pooling time-series data helps or hurts forecasting. It analyzes task symmetries, conflicting continuation rules, and the value of additional information, then tests these ideas with synthetic experiments. Experiments on natural data vary auxiliary sources and training allocation. A separate distillation study trains a small mixture of forecasting operators on predictions from six pretrained models.

Critical strengths

The paper asks a reasonable question: what predictive structure is actually shared across time-series tasks? It distinguishes numerical form from predictive structure and sufficient information, and analyzes the cost of invalid sharing. The connection between rule identifiability and forecast averaging is useful, as is the analysis of how auxiliary-data value depends on the target and training. The four research questions are easy to follow, and the experiments are organized around them.

Critical weaknesses

I am not fully convinced that the analysis answers the question raised by the motivation. Missing predictive information and averaging under squared loss are limitations that also apply to other forecasters; they do not by themselves show a problem with sharing a foundation model across tasks. The paper acknowledges that a shared model can represent different rules, but the natural-data experiments do not establish whether harmful pooling comes from conflating those rules or from estimation and optimization effects. The distillation study has a similar gap: showing that a small operator mixture can approximate useful teacher behavior does not explain when sharing is inappropriate. Without simple operator-mixture baselines and a student trained directly on observed futures, it is also hard to tell what teacher supervision contributes. The conclusions about existing foundation models therefore need a clearer connection to the evidence.

Questions for authors

Which findings specifically concern sharing across tasks, beyond the general limits imposed by missing information and squared loss? Can the natural-data experiments distinguish conflicting rules from training effects, and how does the distillation result help answer the original question about universal models? Comparing distillation with simple operator mixtures and direct training on observed futures at the same data budget, with results shown for each teacher, would make its role clearer.

Reviewer_4Fh3OpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2(+1)
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

The paper argues the idea that Time Series Foundation Models face “category error”: time series as a container lack sufficient information for inferring predictive structure. They investigate four questions: (A) what the cost of transformations that do not preserve the prediction task, (B) how the optimal forecast behaves under indistinguishable rules and how to restore information, (C) how mixing a target and auxiliary training set impacts model target performance, and (D) whether a lightweight operator mixture model can recover TSFM performance. The authors conclude that task structure and rule information should be assessed for universal time series forecasters.

Critical strengths
  • The main strength of the paper is in the formalization of their four questions through squared-loss decompositions. The authors empirically validate the formalizations through both synthetic and real data experiments spanning both tiny model pretraining and TSFM finetuning.

While their squared loss decompositions are sound and attractive they’re too general and abstract to indicate a shift in TSFMs.

Critical weaknesses
  • A crucial weakness of the paper is its clarity and coherence:
    • The paper tries to fit four separate research questions in the main paper leaving most of the experimental explanations in the Appendix. Line 323 they introduce the Question B setup as “A noisy trajectory reaches a boundary and either transmits or reflects its velocity.” with no other context identifying the functional form or clarifying what velocity means.
    • The paper’s thesis and definition of “category error” changes throughout the paper
    • The paper never answers its title: “Are Universal Time Series Foundation Models Based on a Categorical Error?”
      • The authors state that their questions are presented “without presupposing an unconditional answer to the title.” (L63-65)
      • Neither the discussion nor the conclusion state how a possible finding for Questions A-D would give a yes, no or a conditional answer to the title.
    • Question D asks whether a lightweight adaptive operator model can be trained from a TSFM teacher model, but only has a weak link to category error and is not well fleshed out.
      • L290-291 says the comparison gives “the filter interpretation an operational meaning: the forecasting ability explicit operators can carry and the resources required.” There is no falsifiable claim here.
  • There is limited relevance to current TSFM training
    • The theory of Questions A and B are set up as exact squared loss identities and the experiments are built to hold by construction. They just show that a trained model reaches a Bayes optimal model rather than a real TSFM incurring a cost as a result.
      • The experiment for Question A only showed that a standard MLP can learn a conditional mean.
      • The hidden conflict experiment in Question B fine-tunes TimesFM-2.5 and Chronos-2 on the transmission/reflection task and finds again that they converge to the conditional mean as expected. However, this is not something that can be directly translated to conflicts in the pretraining data.
    • Question C experiments show that pairings are asymmetric where Electricity + ETT2 is +19.47% while Loop is -1.92% and the inverse with ETT2 + Electricity is 20.28%. No explanation is given to why this is what was found. Additionally, because Equation 10 is an exact identity for any trained forecaster it is unclear what finding could have contradicted it.

As currently written, the paper is difficult to follow and their main question goes unanswered. Their experiments were setup to pass by construction and their findings do not considerably benefit the TSFM field.

Questions for authors

See Weaknesses

Comments on the paper

1 comment

The author has responded to everyone's comments. Please reply when it is convenient for you.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.