- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper examines whether time-series foundation models transfer across domains because the underlying processes share predictive structure, or merely because the observations share a numerical format. The authors formalise the concern as a category error: a time series specifies an indexing convention and a value space, but not the law that governs its continuation, and when indistinguishable histories admit opposing futures, a predictor trained under squared loss targets the conditional mean, which averages the unresolved rules, retaining the component they share while suppressing the component they dispute. They give the claim testable form through controlled interventions: synthetic tasks in which the history law is held fixed while the continuation rule varies; a twelve-source, small-scale pretraining panel that contrasts source compositions and exposure budgets; and a distillation study in which a 4,590-parameter operator model is trained on the forecasts of six pretrained foundation models. The resulting conclusion is deliberately conditional: the value of sharing depends jointly on the task and on the information available to the model, so universality has to be assessed relative to a task rather than asserted over a data format.
Critical strengths
The paper turns an unclear debate into measurable quantities, and section 5.3 backs it with a matched-history three-arm design: the compatible and hidden-conflict arms share an identical history distribution and differ only in the auxiliary source's continuation rule, isolating a ~2,300× gap that can be attributed to rule conflict alone; the informative arm then injects source identity into the first 16 history values and shows the gap closing.
Critical weaknesses
- The central concept 'category error' is never defined: it appears several times with no criterion for deciding whether a given model commits it. Besides, terms like parameter sharing/distributional compatibility/predictive sufficiency appear in the paper but none is defined. Would be better adding a short concept table covering every term.
- All propositions are established under squared loss and conditional-mean targets. Several recent TSFMs are trained with quantile or distributional objectives, for which Eqs. 4–8 do not apply.
Questions for authors
-
The central demonstration in section 5.3 is that when the prefix carries information that can reveal the hidden rule, error drops from 0.53 to 0.27, close to the theoretical floor reached when the rule is known, whereas a prefix of the same length with no information produces no improvement at all. But this effect appears only in a gating model that was handed the event times and waveform periods in advance: a generic Transformer of the same size and training budget shows essentially nothing, with an interaction of −0.00018 and a confidence interval that crosses zero, and shifting the event onsets by eight steps pulls the effect down from −0.2648 to −0.0096. The authors should therefore say which of two claims they are making: that information present in the data will be used by a model, or that it will be used only when the model is given the right structural assumptions. The wording of the main text suggests the first, but the evidence supports only the second, and the two have quite different practical implications, the first says that better information interfaces are worth building, the second says that an interface is not enough unless the structure is handed over as well.
-
All four theoretical results rest on one premise: squared loss, with the conditional mean as the optimal prediction. But the paper is about time-series foundation models, and a good number of them — Chronos, for instance — are trained with quantile or distributional losses. The loss comparison in Table 2 shows how much this premise matters: with the same data, the same information and the same training budget, switching from squared to absolute loss moves the predicted amplitude from a shrunken 0.52 to close to full scale, improving absolute error while making squared error worse. In other words, changing the loss changes which component survives. The authors should therefore state whether their conclusion is "under squared loss, universality has these limits" or "this holds for all TSFMs". If it is the former, the title and abstract should carry that qualifier; if it is the latter, they need to explain how the quantile and distributional cases follow from the squared-loss results, and no such argument appears in the paper.
The author has responded to everyone's comments. Please reply when it is convenient for you.