acceptodds
Under review as a conference paper at ICLR 2027

TSDATAMODEL: LEARNING FORECASTING UTILITY FOR EXTERNAL CONTEXT RETRIEVAL WITH FROZEN TIME-SERIES FOUNDATION MODELS

Abstract

Time-series foundation models can forecast new domains with frozen weights, but short observed histories may leave their predictions ambiguous. An external archive can supply additional histories, yet conventional retrieval asks which history looks similar to a target query rather than which one actually improves its forecast. We propose TSDataModel, which learns this missing quantity–forecasting utility. For a labeled fitting query, utility is the reduction in forecasting loss obtained by prepending a source history to the query. We estimate this utility with global, reference-based, and parametric scorers. Auto-TSDM chooses among them using a chronologically held-out selection set and freezes the choice before testing; test futures are never used for retrieval or model selection, and the forecasting backbone is never updated. We evaluate six target datasets with TimesFM 2.5-200M and TimeMoE-50M, a disjoint archive built from three source datasets, five random seeds, and length-matched context controls. Auto-TSDM improves over short-history target-only forecasting in 11 of 12 target–backbone settings, reducing mean MAE by 9.3% on average, and outperforms the strongest non-utility retrieval result in mean MAE in 11 of 12 settings. These results position forecasting-utility retrieval as a practical weight-update-free alternative when longer target histories or weight access are unavailable, rather than as a universal replacement for either.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Time Series Forecasting, Time-Series Foundation Models, Cross-Domain Retrieval, Few-Shot Learning, Data Valuation, Datamodeling, Context Augmentation.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8wWSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper looks at frozen time-series foundation models (TimesFM 2.5 and TimeMoE-50M) used with a very short target history (12 steps). It asks whether prepending windows from an external archive (ETTh2, Exchange, METR-LA, 900 windows) can help. Instead of retrieving by similarity, the authors label source-query pairs with “forecasting utility”, meaning the drop in MAE when the source is prepended, using 256 labeled fitting queries. They fit five scorers on these labels (two query-independent global scores, a reference-based kNN average, and linear/MLP regressors) and pick one on a chronological selection set (Auto-TSDM). On six targets, Auto-TSDM beats Target-Only-12 in 11 of 12 settings and beats similarity, DTW, MMD, TS2Vec and a naive RAF baseline in most settings. Target-Only-48 and LoRA remain better in 9/12 and 7/12 settings, which the paper reports openly.

Critical strengths

Strength 1. The evaluation protocol is careful. Fit, selection and test splits are chronological, test futures are never used for selection, and the selection procedure is fixed in advance. It also includes Target-48, LoRA, and direct predictors trained on the same labels. The paper also reports where the method loses (Target-48, LoRA, TimeMoE-Weather at +23.8% over Target-12).

Strength 2. The basic idea of scoring context by its measured effect on a fixed forecaster, and not by how similar it looks, is sensible and easy to deploy with black-box forecasters. On their own terms, the numbers in Table 1 support “utility labels beat unsupervised similarity at picking prefixes”.

Critical weaknesses

Weakness 1. My main concern is that the paper’s own controls point to a different mechanism than the story in the introduction. The motivation is that “temporal shapes such as peaks, recoveries, oscillations” recur across domains. But GP/GR, which use the same three sources for every query, are selected in 41/60 runs. Shuffling the time order inside source windows improves the MLP in 9/12 conditions (Table 9). Exchange-rate windows alone are the best archive for Solar on TimesFM (Table 3). Retrieving the target channel’s own past by cosine similarity is worse than Target-12 in most settings. To me this does not look like retrieval of useful temporal analogies. It looks like finding a fixed prefix that shifts the backbone’s internal normalization/scale estimate for a 12-point window (about half of the Solar test histories are constant, per C.4). Then “retrieval” is really a search over a discrete input prompt. The fair baselines for that are a tuned constant or mean prefix, a few synthetic prefixes, or a learned soft prefix. ARM’s “no source” and “constant slots” variants (Table 9) already match or beat Auto on Solar, PEMS04 and Weather for both backbones, which supports this reading. The numbers are fine, but the cross-domain temporal context framing is not well supported.

Weakness 2. The comparison against similarity baselines is not matched on supervision. Auto-TSDM uses 384 labeled target queries plus about 39.7k backbone calls to build utility labels. The similarity methods use no target labels at all. When supervision is matched (Table 2), a ridge regression or DLinear on the same 12-point histories is close or better on several targets (for example PEMS04, where Ridge gets 0.252 against Auto’s 0.270 on TimesFM, and Metro-NYC, where XGBoost gets 0.781 against Auto’s 0.991). The 12-step setting itself is also very narrow. Target-48 on Metro-NYC is 0.31 against 0.99 for Auto, so in practice a user would get much more from a slightly longer history. I would like a clearer argument for when someone really has only 12 target observations but also has 384 labeled future windows.

Minor issue : some Auto-TSDM standard deviations are large compared with the gains (PEMS-BAY 0.706±0.048 against 0.715 for Target-12), so a few of the “11 of 12” counts rest on small differences. Sections 5.3 and 5.4 are dense with hedged counts and are hard to read.

Questions for authors
  1. Can you add a prefix baseline that does not use the archive, for example the best fixed constant, a mean or linear-trend prefix, or a prefix tuned by search on the fitting set? If GP/GR do no better than this, what is retrieval adding?

  2. Given that time-shuffled sources do as well or better, how much of the gain is explained by the change in the backbone’s instance normalization statistics? Could you report results where the prefix is rescaled to match the query’s mean and std?

  3. How does the gap to Target-Only-12 change with target history 24 or 36? I suspect it disappears quickly. If so, the paper should say that directly.

Reviewer_VgyZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a method for retrieval augmentation in TSFMs focussed on utility. During fitting sample selection is aligned via utility, the decrease in loss resulting from appending the retrieved series to the context input. Following this, four different selection models are chosen based on a validation set prior to testing, of these variations there are statistical and learned options. During testing source windows are ranked by the selected TSDataModel and the top K are selected to be prepended to TSFM context.

Critical strengths

Reformulation of retrieval augmentation for TSFMs: an interesting and novel approach to retrieval based on forecasting utility. Thorough baselines: an extensive choice of baselines throughout the main paper and appendix demonstrating the empirical improvements of the method. Practical Insights: Demonstrating that global rankings are often selected over learned routers such as MLPs provides practical advice for future research and industry practitioners. Additionally, the shuffling experiments in appendix C provide useful insight into limitations of causal mechanisms in current TSFMs.

Critical weaknesses

Limited evaluation setting: The current set of experiments are run across a limited scope where context and horizon are always 12 time-steps, this fails to display the generalisation of the method.

Lacking experimental analysis: The experimental analysis is brief despite the large amount of experimental results, it would be useful to know why the performance of the proposed retrieval method differs between foundation models, especially where it leads to an improvement on datasets with one model and a failure on the other. Additionally, it would be useful to analyse why the method occasionally outperforms a 48 time-step context window on Time-MoE.

Limited algorithmic novelty: The concept of utility-based data valuation is well established, applying standard regressors to predict utility values from time-series features is conceptually incremental.

Questions for authors

How does the performance improvement of this retrieval change with varied lookback lengths and a longer forecast horizon? Why does the proposed method outperform actual context lookback on Solar data for TimeMoE? Why does the performance improvement given by retrieval vary by FM backbone?

Reviewer_hv2bOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents TSDataModel, a method for retrieving external time-series histories to improve forecasting with frozen foundation models. The main idea is to select historical sequences based on how much they reduce forecasting error rather than how similar they are to the target sequence. The authors define forecasting utility as the change in prediction loss after adding a source history to the input. They consider five utility estimators, including global scoring, reference-based estimation, and linear and MLP predictors. Auto-TSDM selects among these estimators using a held-out validation set. The method is evaluated on six datasets with TimesFM 2.5-200M and TimeMoE-50M. It improves over target-only forecasting in 11 of 12 settings, with an average MAE reduction of 9.3%, and generally outperforms similarity-based retrieval methods. The paper also examines the effects of retrieval size, source archive composition, and different utility estimators.

Critical strengths
  1. The paper addresses a relevant problem in retrieval-augmented time-series forecasting. Using forecasting utility rather than similarity to select external histories is a reasonable approach, as similar sequences do not necessarily improve prediction accuracy.
  2. The method does not require updating the forecasting backbone, making it applicable to frozen or black-box time-series foundation models. The use of a separate validation set for estimator selection also helps avoid test-label leakage.
  3. The experiments cover six datasets and two forecasting models, with comparisons against several retrieval baselines. The authors report results across five random seeds and include additional analyses of retrieval budgets, archive composition, computational costs, and different utility estimators.
Critical weaknesses
  1. Several design choices are not sufficiently justified. In particular, the method estimates forecasting utility for individual source histories but directly uses these scores to select and concatenate multiple histories. This does not account for possible interactions or conflicts between retrieved sequences.
  2. The mechanism behind the performance improvements remains unclear. The reported reduction in forecasting error does not necessarily show that the model benefits from meaningful temporal patterns in the retrieved histories. In Appendix C.4, the time-shuffled MLP achieves better results in 9 of 12 settings, which raises questions about the role of temporal structure. Although this experiment does not isolate the effect of temporal order, further controlled experiments are needed to explain where the improvements come from.
  3. The methodological novelty is somewhat limited. Most utility estimators are based on existing techniques, including statistical scoring, similarity-based estimation, linear regression, and MLPs. Auto-TSDM selects among them using validation performance. Moreover, simple global estimators are selected in 41 of 60 configurations, while parametric estimators outperform global methods in only 4 of 12 settings. This raises questions about the necessity of the more complex utility learning components.
Questions for authors
  1. The method estimates utility for individual source histories but retrieves and concatenates multiple histories. What justifies this design?
  2. In Appendix C.4, the time-shuffled MLP performs better in 9 of 12 settings. Could the improvements come from changes in input statistics or other effects of adding prefixes rather than useful temporal information?

Other concerns are discussed in the Weaknesses section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.