acceptodds
Under review as a conference paper at ICLR 2027

Loss-Guided Pretraining Data Selection for Time-Series Foundation Models

Abstract

Time series foundation models (TSFMs) are pretrained on heterogeneous collections containing billions of observations, yet their training windows are typically sampled without estimating whether they provide useful learning signal. We introduce a static data-selection framework that scores each window with a reference forecaster and retains an intermediate interval within every source dataset. Specifically, we connect forecasting loss to optimization difficulty by showing that normalized squared loss controls the per-sample gradient norm under a local Jacobian condition. We then define a reference loss score and apply dataset-stratified selection to preserve the diversity of samples. Across various TSFM architectures, our method outperforms random selection by an absolute margin and even improves both relative MASE and CRPS over full-data pretraining by retaining fewer candidate pretraining windows. Further analyses show strong cross-scale and cross-architecture score correlations, indicating that a small reference model can often select data for larger targets, provided that the reference and target share compatible difficulty orderings.

Primary area:
learning on time series and dynamical systems
Keywords:
data selection, time series, foundation models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_L29tOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper studies data selection for pretraining time-series foundation models. The proposed method uses a frozen reference model to score candidate forecasting windows by their loss and retains the intermediate-loss windows within each source dataset. Pretraining experiments across Chronos-Bolt, Moirai, and TimesFM show that retaining 80% of the windows can outperform both random selection and full-data training under the same compute budget. Additional analyses examine the impact of the selected loss region, dataset stratification, and transferring scores across reference model sizes and architectures.

Critical strengths
  • The paper tackles a useful practical question with a simple, intuitive heuristic that can easily plug into existing pretraining pipelines. A nice practical advantage is that for a fixed candidate pool and reference model, the offline scoring only needs to be done once, and the cached scores can be reused across different retention ratios and compatible target models. Pretraining three distinct model families from scratch also gives solid evidence that the approach works across different target architectures.

  • The experimental setup is well-structured, comparing against full-data training and size-matched random baselines while thoroughly ablating the loss region and dataset stratification components. I also really appreciate that the authors transparently report negative results—such as showing that using TimesFM as a reference model fails to improve Moirai's pretraining performance.

Critical weaknesses
  • I think it is still unclear how reliable the reported improvements are. At 80% retention, aggregate MASE and CRPS improve by about 0.7% to 2.6% over full-data training, but the paper doesn't mention how many seeds were run or how much variance there is across runs. The gains also look less consistent when more data is pruned, for instance, at 40% retention, Moirai actually performs worse than random selection on both metrics. Having repeated runs alongside a per-task or per-dataset breakdown of the main GIFT-Eval results would make it much easier to tell whether these improvements are statistically solid and hold up across datasets.

  • It also seems like an open question whether this method beats simpler, practical alternatives for data selection. Right now, the main comparisons are limited to random selection, full-data training, or ablations of the authors' own filtering rule. Since the core idea closely follows existing loss-based pruning methods, it would be nice to see clearer evidence that a pretrained TSFM reference model is actually necessary. For example, how well would the exact same stratified selection pipeline work if windows were simply scored by a normalized seasonal-naive loss, or by an existing time-series quality rater like TSRating?

Questions for authors
  • Figure 4 reports rank correlations over the global candidate pool, whereas the actual selection algorithm relies on within-source percentiles. Do the reference models also agree on within-source rankings and on which windows actually get kept or dropped? I think this might help clarify whether the reported correlations explain the transfer results.

  • Could you specify how the "native masked loss" is formulated per window for each of the reference models? Additionally, since your theoretical analysis assumes MSE, which parts of that interpretation still hold for the quantile and negative log-likelihood losses used in practice?

5 comments

Thanks for the careful review and the insightful questions.

R3W1

(1) Multiple runs and variance. The main-text results use one training seed because each pretraining run incurs substantial GPU cost. We now report three runs with different seeds, comparing Full with Random and Ours at 80% retention. The table below directly reproduces the mean and standard deviation results. Ours has lower mean MASE and CRPS than both baselines for all target architectures, supporting the reliability of the improvements across runs.

Target modelMetricFull (100%)Random (80%)Ours (80%)
Chronos-Bolt BaseMASE0.795±0.0040.795\pm0.0040.802±0.0050.802\pm0.0050.782±0.006\mathbf{0.782}\pm0.006
Chronos-Bolt BaseCRPS0.552±0.0040.552\pm0.0040.553±0.0040.553\pm0.0040.542±0.002\mathbf{0.542}\pm0.002
Moirai-BaseMASE1.022±0.0051.022\pm0.0051.029±0.0061.029\pm0.0061.009±0.008\mathbf{1.009}\pm0.008
Moirai-BaseCRPS0.704±0.0060.704\pm0.0060.709±0.0070.709\pm0.0070.688±0.008\mathbf{0.688}\pm0.008
TimesFM 2.5MASE0.821±0.0070.821\pm0.0070.832±0.0040.832\pm0.0040.804±0.003\mathbf{0.804}\pm0.003
TimesFM 2.5CRPS0.565±0.0060.565\pm0.0060.572±0.0040.572\pm0.0040.554±0.001\mathbf{0.554}\pm0.001
TimerMASE0.984±0.0070.984\pm0.0070.988±0.0030.988\pm0.0030.919±0.009\mathbf{0.919}\pm0.009
TimerCRPS0.793±0.0080.793\pm0.0080.787±0.0080.787\pm0.0080.742±0.002\mathbf{0.742}\pm0.002

(2) Consistent gains. Moirai’s result at 40% retention is because a narrower central interval can reduce data diversity, while random selection gains an advantage through broader pattern coverage. Ayed and Hayou [M1] theoretically identify generalization degradation induced by score-based pruning under strong compression in their analyzed settings. Moreover, we emphasize that our main contribution is outperforming full-data pretraining by retaining 80% of the candidate pretraining windows.

[M1] Fadhel Ayed and Soufiane Hayou. Data Pruning and Neural Scaling Laws: Fundamental Limitations of Score-Based Algorithms. Transactions on Machine Learning Research, 2023.

(3) Per-task breakdown. We additionally report the results on the long-horizon and short-horizon forecasting tasks in GIFT-Eval. Table below reports this breakdown and Ours improves both metrics over Full and Random on long-horizon and short-horizon tasks.

Target modelHorizonFull (100%) / MASEFull (100%) / CRPSRandom (80%) / MASERandom (80%) / CRPSOurs (80%) / MASEOurs (80%) / CRPS
Chronos-Bolt BaseShort0.7340.5410.7420.5380.7230.533
Chronos-Bolt BaseLong0.8890.5730.8890.5700.8680.562
Moirai-BaseShort0.9460.6960.9470.6910.9260.680
Moirai-BaseLong1.1330.7011.2360.7601.1270.678
TimesFM 2.5Short0.7490.5430.7700.5600.7450.539
TimesFM 2.5Long0.9200.5860.9330.5990.8900.571
TimerShort0.8580.7620.8640.7680.7930.721
TimerLong1.1890.8231.2840.8531.1160.769

R3W2

We add the normalized seasonal-naive loss alternative using the exact same stratified selection pipeline at 80% retention. Table below shows that RFL improves both MASE and CRPS across all architectures, which is a clearer evidence that a pretrained TSFM reference model is useful for data selection.

MethodChronos/ MASEChronos/ CRPSMoirai/ MASEMoirai/ CRPSTimesFM/ MASETimesFM/ CRPSTimer/ MASETimer/ CRPS
Seasonal0.7980.5601.0310.7030.8320.5690.9380.778
Ours0.7770.5421.0050.6800.8040.5540.9090.742
Full0.7960.5551.0170.6980.8130.5580.9780.788

Moreover, TSRating targets task-specific training or fine-tuning through LLM pairwise judgments and a meta-learned rater, and selects a subset. In contrast, we focus on pretraining time-series foundation models. Scaling TSRating’s LLM annotation and meta-training to our more than 100 source datasets would introduce substantial computational resource cost and time cost, whereas RFL requires only one frozen forward pass of a lightweight model.

R3Q1

The reference models also agree on within-source rankings and on which windows actually get kept or dropped. We additionally report the Pearson correlation between source-wise percentile vectors as shown in Figure 4. TimesFM and Chronos-Bolt agree strongly (0.879), while Moirai agrees weakly with TimesFM (0.182). We also compare retained-window overlap at 80% retention using source-wise sorting. Among the windows retained by TimesFM, 92.04% are also retained by Chronos-Bolt, whereas Moirai’s overlap with TimesFM is consistent with that of random selection (80%). Thus, the transfer results are consistent with both within-source rankings and selected-window agreement.

R3Q2

(1) Per-window formulation of the native masked loss. Appendix B now specifies all four objectives. Chronos-Bolt uses the normalized pinball loss. Timer uses squared errors and TimesFM combines point MSE with the sum of the pinball loss. Moirai uses the negative log-likelihood of each observed future value.

(2) Interpretation for quantile and negative log-likelihood losses. Appendix B extends the gradient analysis to both objectives.

Quantile loss. The gradient depends on the sign of the residual and on the quantile level, and the corresponding norm bound contains no residual-magnitude term (Appendix B.1).

Negative log-likelihood. The gradient is a responsibility-weighted combination of the component likelihood gradients (Appendix B.2). With a shared fixed variance, Gaussian NLL is an affine transformation of MSE, so the MSE interpretation holds exactly. With a predicted scale or mixture components, the update additionally reflects predictive uncertainty and distributional fit, which goes beyond the MSE interpretation.

Dear Reviewer L29t, as the discussion is coming to an end, we would like to check whether your concerns have been adequately addressed. We look forward to hearing from you. Best regards

Reviewer_ZVZKOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a loss-guided data selection framework for pretraining time-series foundation models. A frozen reference forecaster assigns each candidate window a reference loss score, and the method retains intermediate-difficulty samples within each source dataset to remove overly easy or potentially noisy examples while preserving data diversity. Experiments on Chronos-Bolt, Moirai, and TimesFM show that the selected subsets can outperform random selection and, at high retention ratios, even full-data pretraining.

Critical strengths
  1. Simple and practical data-selection strategy. RFL requires only a frozen reference model and one offline scoring pass, making it easy to integrate into existing TSFM pretraining pipelines.

  2. Strong empirical effectiveness across different TSFM architectures. The selected subsets consistently outperform random sampling and can even surpass full-data pretraining.

  3. Useful analysis of reference-model transferability. The paper studies both cross-scale and cross-architecture transfer, showing when a small reference model can reliably guide data selection.

Critical weaknesses
  1. Reference-model choice remains under-specified. Performance depends on whether the reference and target induce similar difficulty rankings, but the paper does not provide a principled way to choose a suitable reference model before training the target.

  2. The central-RFL heuristic is still somewhat empirical. The theory links loss to potential gradient magnitude, but does not directly prove that intermediate-loss samples are always the most useful for optimization.

  3. Data selection is based on a single scalar score. RFL may not fully capture other useful properties such as diversity, rarity, seasonality, or long-range temporal structure.

  4. Additional offline scoring is still required. Every candidate window must be passed through a reference model once, which can become expensive for very large pretraining corpora despite being cheaper than full retraining.


If the authors can provide a convincing explanation and additional evidence addressing these concerns, I would be willing to reconsider and potentially revise my score.

Questions for authors

Please see the Critical Weaknesses section.

5 comments

Thanks for the careful reading and constructive suggestions.

R2W1

We provide a principled way based on loss compatibility to choose a suitable reference model, as in Appendix F. For same-architecture, a small same-family reference is a practical choice to accelerate scoring. For cross-architecture, point and quantile loss based models can align, whereas Moirai NLL aligns weakly with others. To illustrate, we add the point MSE based model Timer in Table 2 and update Figure 4 accordingly. ChronosBolt/Timer/TimesFM all use point MSE or quantile loss which measures forecast discrepancies, and they show strong rank agreement of 0.828–0.936 as shown in Figure 4. Moirai’s NLL additionally depends on predictive uncertainty and mixture-distribution, and its correlations with other models are low.

Target ModelReference ModelRetainedMASECRPS
TimerTimer80%0.9090.742
TimerChronos-Bolt Mini80%0.910‾\underline{0.910}0.745‾\underline{0.745}
TimerMoirai-Small80%1.0580.774
Timer— (Full data)100%0.9780.788

R2W2

The RFL-region ablation in Section 4.4.1 supports the central-RFL heuristic by testing intermediate-loss samples against both tails and random selection. We do not claim a theoretical proof that intermediate loss is always optimal, instead we provide a theoretical explanation and motivational discussion.

Section 3.2 uses a first-order Taylor expansion (Equation 4) to describe how an individual sample update changes the overall loss. The same expansion describes how an overall update changes that sample’s loss.

=-\eta\langle\bar{\mathbf{g}},\mathbf{g}_i\rangle+o(\eta), \tag{R2.1}$$ $$\ell_i(\boldsymbol{\theta}-\eta\bar{\mathbf{g}})-\ell_i(\boldsymbol{\theta}) =-\eta\langle\mathbf{g}_i,\bar{\mathbf{g}}\rangle+o(\eta). \tag{R2.2}$$ Thus, gradient alignment connects a sample’s contribution to training with how it benefits from the overall update. Theorem 3.1 further relates MSE to gradient magnitude under the stated local-Jacobian condition. Together, these observations motivate the following interpretation. **(1) Low-loss samples.** A window with low loss throughout training is easy to predict. Learning intermediate-difficulty patterns may generalize to simpler windows sharing those patterns. Equation R2.2 allows an omitted window’s loss to decrease if its gradient aligns with the aggregate gradient of the retained subset. Moreover, the theorem’s upper bound associates a small loss with a small gradient, limiting its contribution in Equation R2.1. We test the generalizability by training Chronos-Bolt for one epoch on our central 80% and the full candidate pool. We evaluate the mean loss on low-loss (bottom-10%) windows for both models. Results show training on the central interval alone reduces their mean loss by 90.62%, with 99.48% of the low-loss windows improving. This provides empirical evidence of transfer from intermediate-difficulty training data to easier windows, supporting the reduction of redundant low-loss samples. | Training subset | Loss decrease for low-loss samples | Improved | | -------------------------------------- | ---------------------------------- | -------- | | Central 80% (exclude low-loss samples) | 90.62% | 99.48% | | Full 100% (include low-loss samples) | 91.97% | 99.65% | **(2) High-loss samples.** High loss may reflect abrupt changes or poorly predictable futures, as illustrated in Figure 5. Even when these windows induce large gradients, their updates may not be useful. We analyzed the Chronos-Bolt training-loss trajectory data. Among the high-loss samples, by epoch 6 of training, 83.3% of the samples were still in the highest decile, and 24.5% of them actually increased in loss. Equation R2.2 further explains that negative gradient alignment causes a first-order increase in the sample’s loss under a full-gradient update, while its own update opposes overall loss reduction in Equation R2.1. Besides, Figure 3(a) reports the validation loss when training on high-loss data (Top at 40% retention), and its performance is the worst. This explains the reduction of high-loss samples.

R2W3

Our framework actually addresses some of these properties through dataset-stratified selection and the predictive knowledge of the reference model.

(1) Diversity. Dataset-stratified selection explicitly preserves source coverage and data diversity by retaining the central interval within each source. In Figure 3(c), stratified selection, while preserving data diversity, consistently improves MASE and CRPS over global selection.

(2) Rarity. We prioritize the usefulness of a window rather than rarity. For example, the high-RFL tail mixes rare but harmful noise, and our central-RFL outperforms full-data training after removing these samples as shown in Table 3. These results support concentrating training on informative, learnable windows. Meanwhile, dataset-stratified selection preserves coverage of rare sources.

(3) Seasonality. Seasonal regularities affect RFL through their contribution to forecasting. We additionally use seasonal-naive loss as a seasonality-oriented scoring baseline, since its forecasts repeat a seasonal cycle. With the same stratified selection pipeline and 80% retention, RFL achieves lower MASE and CRPS as shown in table below.

MethodChronos/ MASEChronos/ CRPSMoirai/ MASEMoirai/ CRPSTimesFM/ MASETimesFM/ CRPSTimer/ MASETimer/ CRPS
Seasonal0.7980.5601.0310.7030.8320.5690.9380.778
Ours0.7770.5421.0050.6800.8040.5540.9090.742
Full0.7960.5551.0170.6980.8130.5580.9780.788

(4) Long-range temporal structure. TSFMs are pretrained to forecast using long contexts and our experimental pipeline uses a context length of 2048. RFL evaluates the long-range forecasting loss and therefore implicitly reflects the long-range temporal dependencies learned by the reference model. We additionally evaluate on long-horizon GIFT-Eval tasks and our method maintains its advantage.

MethodChronos/ MASEChronos/ CRPSMoirai/ MASEMoirai/ CRPSTimesFM/ MASETimesFM/ CRPSTimer/ MASETimer/ CRPS
Random0.8890.5701.2360.7600.9330.5991.2840.853
Ours0.8680.5621.1270.6780.8900.5711.1160.769
Full0.8890.5731.1330.7010.9200.5861.1890.823

Moreover, a single scalar keeps scoring to one frozen forward pass with one scalar per window, which is efficient at pretraining scale. Nevertheless, our framework is also extensible. Additional attributes can be combined with RFL during selection, and multi-attribute scoring is a promising direction for future work.

R2W4

Our scoring process can be reduced to nearly 13 minutes through lightweight reference models and parallel scoring.

(1) Lightweight reference models. The reference-model analysis shows that small references can produce competitive training subsets while reducing the additional offline scoring cost. As reported, Chronos-Bolt Mini scores all candidate windows in only 40 minutes on one A100 GPU. The resulting score cache can also be reused for subsequent training runs under different retention ratios and with different target models, amortizing this one-time cost.

(2) Parallel scoring. Moreover, through parallel scoring, our scoring process can be reduced significantly. The single-inference scoring times we currently report all use a single GPU, whereas most target training uses more GPUs. For example, in our experiments, Chronos-Bolt Base uses 3 GPUs for training. Since candidate windows are scored independently and scoring is performed before training, training resources can be fully utilized by partitioning the candidate pool across multiple GPUs and placing the frozen reference model on each device. This means the Chronos-Bolt Mini can complete scoring in about 40/3≈1340/3\approx13 minutes, which is significantly cheaper than training.

Dear Reviewer ZVZK, as the discussion is coming to an end, we would like to check whether your concerns have been adequately addressed. We look forward to hearing from you. Best regards

Reviewer_pz7NOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a method for selecting and reducing the number of samples used to pretrain time-series foundation models. Using the prediction loss of a pretrained reference model, the method removes samples that are either too easy or too difficult. Because the pretraining corpus consists of multiple datasets, the selection procedure is applied separately to each dataset to preserve sample diversity. Experiments with several time-series foundation models show that retaining only 80% of the data can improve forecasting performance compared with both random selection at the same retention rate and pretraining on the full dataset.

Critical strengths
  • The proposed method is efficient because it requires only a single inference pass by the reference model for each sample and the storage of one scalar value; the reference information does not need to be updated during training. Even when the pretraining data are reduced to 80%, the method outperforms both training on the full dataset and training on a randomly selected subset of the same size.

  • The paper shows that applying the selection method separately to each constituent dataset contributes more to forecasting performance than pruning the combined data without distinguishing among datasets.

  • The experiments compare performance across multiple foundation models. They also examine cases in which selection remains effective with a smaller reference model and analyze transfer across different architectures, providing useful practical guidance for choosing a reference model.

Critical weaknesses
  • A similar approach was studied by Marion et al. (2023), so the methodological novelty is relatively limited. The paper does not sufficiently demonstrate technical innovations or new insights beyond applying the approach to time-series data.

  • The theoretical analysis establishes a relationship between MSE and gradient magnitude, but it does not justify why removing both low-loss and high-loss samples should benefit pretraining.

  • Although performance improves after data reduction, some gains are small. Because the paper does not report variability across multiple runs, it remains unclear whether the proposed method yields consistent improvements.

Questions for authors
  • Data selected using TimesFM improves Chronos but degrades Moirai. Do the authors have any insight into which properties of the architecture or loss function determine whether data selected by a reference model will transfer effectively to the target model? Such guidance would be useful when choosing a reference model.

  • In Table 2, when Moirai-Base is used as both the reference model and the training target, the benefit of retraining relative to the already pretrained reference model is unclear. Reporting the reference model’s forecasting performance under the same 91 evaluation settings would help clarify how the newly trained model compares with the reference model itself.

5 comments

Thanks for the helpful comments and detailed suggestions.

R1W1

We respectfully disagree that the methodological novelty is relatively limited. Marion et al. rank text sequences using reference-model perplexity and train language models from scratch. We focus on TSFM pretraining, which is different from Marion et al. Specifically, we theoretically show that, in the time-series setting, normalized MSE controls gradient magnitude under an explicit local Jacobian condition, which Marion et al. does not provide. Besides, Marion et al. apply corpus-level percentiles to a CommonCrawl corpus, whereas we compute percentiles independently within each source dataset. This preserves source coverage and dataset diversity, as shown in the global-selection ablation (Figure 3(c)). Moreover, time series reference models exhibit strong rank agreement across scales and cross-architecture transferability, whereas Marion et al. report improved pruning with larger reference models in their language-model experiments. We add this in Appendix A.2 to explicitly demonstrate technical innovations and distinguish additional insights.

R1W2

The RFL-region ablation in Section 4.4.1 compares Bottom, Top, Central, and Random, demonstrating the effectiveness of removing low-loss and high-loss samples. We do not claim a theoretical proof that intermediate loss is always optimal, instead we provide a theoretical explanation and motivational discussion.

Section 3.2 uses a first-order Taylor expansion (Equation 4) to describe how an individual sample update changes the overall loss. The same expansion also describes how an overall gradient update changes that sample’s loss.

=-\eta\langle\bar{\mathbf{g}},\mathbf{g}_i\rangle+o(\eta), \tag{R1.1}$$ $$\ell_i(\boldsymbol{\theta}-\eta\bar{\mathbf{g}})-\ell_i(\boldsymbol{\theta}) =-\eta\langle\mathbf{g}_i,\bar{\mathbf{g}}\rangle+o(\eta). \tag{R1.2}$$ Thus, gradient alignment connects a sample’s contribution to training with how it benefits from the overall update. Theorem 3.1 further relates MSE to gradient magnitude under the stated local-Jacobian condition. Together, these observations motivate the following interpretation. **(1) Low-loss samples.** A window with low loss throughout training is easy to predict. Learning intermediate-difficulty patterns may generalize to simpler windows sharing those patterns. Equation R1.2 allows an omitted window’s loss to decrease if its gradient aligns with the aggregate gradient of the retained subset. Moreover, the theorem’s upper bound associates a small loss with a small gradient, limiting its contribution in Equation R1.1. We test the generalizability by training Chronos-Bolt for one epoch on our central 80% and the full candidate pool. We evaluate the mean loss on low-loss (bottom-10%) windows for both models. Results show training on the central interval alone reduces their mean loss by 90.62%, with 99.48% of the low-loss windows improving. This provides empirical evidence of transfer from intermediate-difficulty training data to easier windows, supporting the reduction of redundant low-loss samples. | Training subset | Loss decrease for low-loss samples | Improved | | -------------------------------------- | ---------------------------------- | -------- | | Central (exclude low-loss samples) | 90.62% | 99.48% | | Full (include low-loss samples) | 91.97% | 99.65% | **(2) High-loss samples.** High loss may reflect abrupt changes or poorly predictable futures, as illustrated in Figure 5. Even when these windows induce large gradients, their updates may not be useful. We analyzed the Chronos-Bolt training-loss trajectory data. Among the high-loss samples, by epoch 6 of training, 83.3% of the samples were still in the highest decile, and 24.5% of them actually increased in loss. Equation R1.2 further explains that negative gradient alignment causes a first-order increase in the sample’s loss under a full-gradient update, while its own update opposes overall loss reduction in Equation R1.1. Besides, Figure 3(a) reports the validation loss when training on high-loss data (Top at 40% retention), and its performance is the worst. This explains the reduction of high-loss samples.

R1W3

The main-text results use one training seed because each pretraining run incurs substantial GPU cost. We now report variability across three runs using different random seeds. Table below compares full-data pretraining with Random and Ours at 80% retention, reporting the mean and standard deviation. Ours achieves lower mean MASE and CRPS than both baselines across all four target architectures. The improvements over Random and Full hold in every seed for both metrics, supporting the consistent improvements.

Target modelMetricFull (100%)Random (80%)Ours (80%)
Chronos-Bolt BaseMASE0.795±0.0040.795\pm0.0040.802±0.0050.802\pm0.0050.782±0.006\mathbf{0.782}\pm0.006
Chronos-Bolt BaseCRPS0.552±0.0040.552\pm0.0040.553±0.0040.553\pm0.0040.542±0.002\mathbf{0.542}\pm0.002
Moirai-BaseMASE1.022±0.0051.022\pm0.0051.029±0.0061.029\pm0.0061.009±0.008\mathbf{1.009}\pm0.008
Moirai-BaseCRPS0.704±0.0060.704\pm0.0060.709±0.0070.709\pm0.0070.688±0.008\mathbf{0.688}\pm0.008
TimesFM 2.5MASE0.821±0.0070.821\pm0.0070.832±0.0040.832\pm0.0040.804±0.003\mathbf{0.804}\pm0.003
TimesFM 2.5CRPS0.565±0.0060.565\pm0.0060.572±0.0040.572\pm0.0040.554±0.001\mathbf{0.554}\pm0.001
TimerMASE0.984±0.0070.984\pm0.0070.988±0.0030.988\pm0.0030.919±0.009\mathbf{0.919}\pm0.009
TimerCRPS0.793±0.0080.793\pm0.0080.787±0.0080.787\pm0.0080.742±0.002\mathbf{0.742}\pm0.002

R1Q1

The loss function is a useful guide to transfer across architectures. We add the point MSE based model Timer in Table 2 and update Figure 4 accordingly. Specifically, ChronosBolt/Timer/TimesFM all use point MSE or quantile loss which measures forecast discrepancies, and they show strong rank agreement of 0.828–0.936 as shown in Figure 4. This agreement helps explain why TimesFM-selected and Chronos-selected data transfer effectively. Moirai’s NLL additionally depends on predictive uncertainty and mixture-distribution, and its correlations with other models are low. This difference helps explain why TimesFM degrades Moirai. We add the above discussion to Appendix F.

Target ModelReference ModelRetainedMASECRPS
TimerTimer80%0.9090.742
TimerChronos-Bolt Mini80%0.910‾\underline{0.910}0.745‾\underline{0.745}
TimerMoirai-Small80%1.0580.774
Timer— (Full data)100%0.9780.788

R1Q2

We report the reference model’s forecasting performance under the same 91 evaluation settings in table below. We use the following three models which serve as both the reference model and the training target. The ’Reference’ columns report the reference model itself, while ’ours’ reports the target trained on the selected subset. Our method achieves lower MASE and CRPS across different architectures. This directly demonstrates the benefit of training on the selected data relative to using the already pretrained reference model for forecasting.

MethodMoirai/ ReferenceMoirai/ OursTimer/ ReferenceTimer/ OursTimesFM/ ReferenceTimesFM/ Ours
MASE1.0271.0050.9920.9090.8470.804
CRPS0.7090.6800.8020.7420.5780.554
AuthorsOpenReview ↗

We look forward to hearing from you.

Dear Reviewer pz7N, as the discussion is coming to an end, we would like to check whether your concerns have been adequately addressed. We look forward to hearing from you. Best regards

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.