acceptodds
Under review as a conference paper at ICLR 2027

TRACS: Time-Series Rebalancing and Coverage-Aware Selection for Pretraining

Abstract

As time-series foundation models (TSFMs) mature, performance increasingly depends on the quality and composition of their pretraining data. Most existing data selection methods rely on training feedback or external supervision, which makes it difficult to determine the selected corpus entirely before pretraining. We present TRACS (Time-Series Rebalancing and Coverage-Aware Selection), a zero-feedback method for selecting time-series pretraining data without either source of supervision. TRACS uses GMM-based soft clustering to organize candidate series by temporal pattern, then assigns sampling quotas based on the coverage required within each cluster rather than on cluster size. The selected subset and its pattern coverage can therefore be determined before pretraining begins. Under the stated assumptions, we prove coverage of all reachable patterns, finite-step termination of the missing-pattern repair procedure, and a bounded representation error with respect to the rebalanced target distribution. We evaluate TRACS on MOMENT-1 Small, PatchTST, and Timer under the same 10B-token pretraining budget. Using only 25% or 50% of the candidate corpus, TRACS attains the lowest aggregate MASE and CRPS among the evaluated pretraining baselines on 20 stable zero-shot forecasting tasks from GIFT-Eval. Relative to pretraining on the full corpus under the same token budget, TRACS reduces MASE by 2.2%–4.0% and provides an explicit guarantee on pattern coverage that the baseline methods do not offer.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Foundation Models, Pretraining Data Selection, Diversity Maximization, Coverage Constraints

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_wP7COpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TRACS selects time-series pretraining data using statistical features, GMM clustering, balanced cluster quotas, within-cluster sampling, and a final repair step for missing binary feature patterns. It provides conditional coverage and distributional bounds. Under a fixed 10B-token budget, selected subsets improve aggregate forecasting metrics across three architectures.

Critical strengths
  1. The method makes an explicit, checkable distinction between spreading samples through feature space and retaining every reachable discrete pattern. The repair procedure preserves subset size and gives a concrete coverage guarantee under the stated feasibility condition.
  2. The experiments control the processed-token budget and cover three architectures with five seeds. Table 2 shows that selected subsets can outperform full-corpus training under this control. The per-task results and bootstrap analyses also make the variability of the gains visible.
Critical weaknesses
  1. The closest baseline is substantially modified. BLAST-style replaces the original features with F1–F7 and removes grid mixup, while TRACS uses a richer 27-dimensional clustering representation (Section 5.1 and Appendix E). The paper acknowledges this, but without the original baseline or a matched-representation comparison, the advantage of the proposed selection procedure over the closest existing approach remains uncertain.
  2. The evidence that pattern repair improves forecasting is weaker than the interpretation in Section 5.3. The No Swap subset misses nine patterns, so repair replaces nine series in a subset of roughly 300,000, yet Table 3 attributes a MASE reduction from 1.0892 to 1.0500 to this step. Such an effect is possible, but the corresponding task-bootstrap interval includes zero, as do all component-ablation intervals in Table 15. These results do not yet distinguish a reliable benefit from training variability or unusually influential replacement series.
Questions for authors
  1. Does the advantage remain against the original BLAST implementation, and when both selection pipelines use the same input features?
  2. How many training windows or tokens do the nine replacement series contribute? Can paired runs and a control with nine random replacements establish that recovering the missing patterns explains the reported forecasting gain?
2 comments

Thank you for your careful reading of the paper and your thoughtful consideration of the baseline comparisons and the experimental evidence for pattern repair. To address these issues, we have added baseline and ablation experiments. We respond to your comments point by point below.

W1 / Q1 BLAST baseline adaptation and data-selection comparisons

Regarding the difference in the BLAST configuration that you pointed out, we have added a comparison using its original statistical indicators and selection procedure. Here, we respond to your comments in three parts, explaining how the baseline is configured, how the representations of the two methods enter the comparison, and what results the additional experiments obtained.

(1)

The added comparison retains BLAST’s original statistical features, encoding, UMAP projection, and grid selection configuration, disables only Grid Mixup, and is still labeled BLAST-style in the table. We also added Training Diet and LP-Mirror to further supplement the baseline comparisons. However, we still insist on excluding Grid Mixup because it is itself an independent data-augmentation method that constructs new training samples by mixing existing sequences. If it were retained only for BLAST, it would be difficult to distinguish whether the forecasting differences come from data selection or additional data augmentation. Therefore, we hope to better observe the data-selection effects of the respective frameworks when neither uses Mixup.

(2)

TRACS uses a 27-dimensional statistical representation that we designed to characterize temporal structure more fully during the preceding clustering stage, while the final pattern definitions and coverage checks still use a seven-bit encoding. This experiment retains the respective input representations and selection procedures of the two methods and uses the same candidate corpus, retention ratio, and training budget. We compare the forecasting performance of the respective complete selection frameworks, which also includes the joint effects of the input representation and the selection procedure.

(3)

We expanded the candidate corpus from approximately 600,000 to approximately 2,170,000 sequences and also increased the number of training seeds from 5 to 7. The additional comparisons cover five architectures, MOMENT, PatchTST, Timer, Chronos2-small, and Toto2-22M, with evaluation across 97 downstream evaluation configurations. The results show that, after restoring BLAST’s original statistical indicators, TRACS still achieves lower forecasting error for every reported model and retention ratio. This shows that TRACS retains its forecasting advantage when BLAST uses its own configuration and the evaluation scope is expanded, with the detailed results shown in the table below.

Forecasting error comparison. Lower geometrically aggregated MASE is better; we report the mean ±\pm standard deviation across 7 seeds.

RetainedMethodMOMENTPatchTSTTimerChronos2-smallToto2-22M
100%Full1.891±0.1241.891 \pm 0.1241.823±0.1321.823 \pm 0.1321.809±0.1461.809 \pm 0.1461.611±0.1081.611 \pm 0.1081.699±0.0661.699 \pm 0.066
25%Random1.783±0.0231.783 \pm 0.0231.814±0.1031.814 \pm 0.1031.885±0.1101.885 \pm 0.1101.529±0.0061.529 \pm 0.0061.695±0.0061.695 \pm 0.006
25%Stratified1.755±0.0721.755 \pm 0.0722.278±0.5102.278 \pm 0.5101.943±0.3191.943 \pm 0.3191.519±0.0731.519 \pm 0.0731.684±0.0151.684 \pm 0.015
25%BLAST-style1.804±0.1331.804 \pm 0.1331.778±0.2091.778 \pm 0.2091.899±0.1911.899 \pm 0.1911.514±0.0871.514 \pm 0.0871.685±0.0131.685 \pm 0.013
25%Training Diet1.729±0.0591.729 \pm 0.0591.736±0.0701.736 \pm 0.0701.787±0.2461.787 \pm 0.2461.513±0.0651.513 \pm 0.0651.673±0.030‾\underline{1.673 \pm 0.030}
25%LP-Mirror1.623±0.016\mathbf{1.623 \pm 0.016}1.609±0.043\mathbf{1.609 \pm 0.043}1.604±0.026\mathbf{1.604 \pm 0.026}1.471±0.010\mathbf{1.471 \pm 0.010}1.762±0.0181.762 \pm 0.018
25%TRACS1.661±0.026‾\underline{1.661 \pm 0.026}1.723±0.398‾\underline{1.723 \pm 0.398}1.749±0.198‾\underline{1.749 \pm 0.198}1.508±0.046‾\underline{1.508 \pm 0.046}1.661±0.042\mathbf{1.661 \pm 0.042}
50%Random1.838±0.0271.838 \pm 0.0271.879±0.0771.879 \pm 0.0771.861±0.094‾\underline{1.861 \pm 0.094}1.525±0.006‾\underline{1.525 \pm 0.006}1.696±0.007‾\underline{1.696 \pm 0.007}
50%Stratified1.897±0.0871.897 \pm 0.0871.957±0.1511.957 \pm 0.1511.911±0.1671.911 \pm 0.1671.653±0.0781.653 \pm 0.0781.759±0.0371.759 \pm 0.037
50%BLAST-style1.887±0.0671.887 \pm 0.0671.912±0.1671.912 \pm 0.1671.896±0.1721.896 \pm 0.1721.630±0.0601.630 \pm 0.0601.756±0.0391.756 \pm 0.039
50%Training Diet1.952±0.1221.952 \pm 0.1221.886±0.0821.886 \pm 0.0821.880±0.1581.880 \pm 0.1581.653±0.0771.653 \pm 0.0771.774±0.0531.774 \pm 0.053
50%LP-Mirror1.677±0.055\mathbf{1.677 \pm 0.055}1.856±0.096‾\underline{1.856 \pm 0.096}1.899±0.1561.899 \pm 0.1561.465±0.041\mathbf{1.465 \pm 0.041}1.745±0.0361.745 \pm 0.036
50%TRACS1.809±0.056‾\underline{1.809 \pm 0.056}1.840±0.125\mathbf{1.840 \pm 0.125}1.832±0.062\mathbf{1.832 \pm 0.062}1.620±0.0671.620 \pm 0.0671.688±0.044\mathbf{1.688 \pm 0.044}

W2 / Q2 Forecasting effects of pattern repair and ablation evidence

Following your suggestion, we added two sets of comparisons. One set compares the full configuration with No Swap using a larger dataset, a broader evaluation scope, and more random seeds; the other adds Random Swap with the same number of replacements to further compare repair guided by missing patterns with random replacement.

(1)

We added paired ablations of the full configuration and No Swap on a larger candidate corpus and more downstream evaluation configurations and increased the number of random seeds. The results show that, after adding repair, overall forecasting error is lower and performance is better on most evaluation configurations, with the statistical comparison also supporting this difference. This improvement can still be observed after expanding the scale of the experiments and increasing the number of repeated runs, providing more substantial evidence for the practical role of repair than the original manuscript provided.

(2)

We also added Random Swap as you suggested, keeping its number of random replacements the same as that of pattern repair. In this comparison, the full configuration still has lower overall forecasting error and performs better on most evaluation configurations, with the corresponding statistical comparison also supporting this difference. In other words, with the same number of replacements, repair guided by missing patterns achieves better forecasting performance than random replacement, which further supports our reasoning for designing the repair step this way.

These two sets of results give us a stronger basis for explaining that repair also has an effect on downstream forecasting and that targeted replacement is more effective than random replacement. We will add these results to Section 5.3, with the detailed ablation performance and statistical comparisons shown in the table below.

VariantMASE ↓\downarrowChange vs. Origin (%)Bootstrap interval (Origin −- variant)W/T/L
Origin (TRACS-50%)1.832±0.0621.832 \pm 0.0620.00----
K=1000K=10001.867±0.1931.867 \pm 0.193+1.92[−2.392,,0.024][-2.392,\\,0.024]58/0/39
K=500K=5001.893±0.0901.893 \pm 0.090+3.35[−1.142,,0.059][-1.142,\\,0.059]67/0/30
K=3000K=30001.851±0.1031.851 \pm 0.103+1.04[−1.796,,−0.091][-1.796,\\,-0.091]57/0/40
Proportional quota1.897±0.1471.897 \pm 0.147+3.52[−1.570,,−0.148][-1.570,\\,-0.148]76/0/21
Random within-cluster selection1.876±0.1181.876 \pm 0.118+2.43[−0.488,,−0.055][-0.488,\\,-0.055]63/0/34
No Swap1.931±0.5931.931 \pm 0.593+5.38[−2.025,,−0.117][-2.025,\\,-0.117]74/0/23
Random Swap1.884±0.7921.884 \pm 0.792+2.84[−1.351,,−0.067][-1.351,\\,-0.067]72/0/25
No entropy guidance1.917±0.1381.917 \pm 0.138+4.67[−0.613,,−0.024][-0.613,\\,-0.024]79/0/18

The revised PDF and supplementary material have been uploaded. In the annotated PDF, green margin labels marked wP7C identify revisions addressing your comments, with the relevant weakness and question numbers shown in each label. Shared revisions may carry labels for more than one reviewer. We welcome further discussion of the baseline comparisons and repair evidence and would be glad to clarify any remaining points.

Reviewer_HuYCOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces TRACS, a data selection method for TSFMs without training feedback or external supervision. TRACS first diagnoses whether each time series is suitable for STL decomposition, and then applies a four-stage framework consisting of feature extraction, Gaussian mixture clustering, coverage-aware quota allocation, and pattern repair. Experiments on 20 forecasting tasks show that TRACS achieves competitive performance using only 25%-50% of the candidate corpus across three architectures: MOMENT-1 Small, PatchTST, and Timer.

Critical strengths
  1. Compared with other data selection methods for TSFMs, the proposed approach does not rely on training feedback or external supervision. This is particularly important with expanding large-scale time-series corpora and the high cost of pretraining.
  2. The method is well organized and decomposes the problem into several clear components. Compared with simple stratified sampling or mixture optimization methods that rely on training feedback, this idea of determining the subset before pretraining while providing verifiable coverage guarantees is reasonably novel.
  3. The experiments are conducted on three heterogeneous architectures, showing the consistent effectiveness of TRACEs.
Critical weaknesses
  1. The baselines are not sufficiently strong, which weakens the experimental results. The paper emphasizes a zero-feedback setting and therefore excludes methods such as Training Diet and LP-Mirror, which rely on training feedback or guidance sets. This is understandable given the problem formulation. However, in the experiments, the main baselines are naïve and simple, such as Random, Stratified, and a modified BLAST-style baseline. As a result, the experimental comparisons are somewhat limited and do not fully demonstrate the strength of the proposed method against stronger alternatives.
  2. TRACS consists of multiple stages, including STL-based routing, 27-dimensional feature engineering, incremental PCA, coreset GMM clustering, and two separate sampling strategies for wide and narrow clusters. Although the authors emphasize that the selection process is less costly than training, this is only qualitative. The paper lacks quantitative reporting on the actual wall-clock time and the associated CPU/GPU resource overhead, which would be important for assessing the practical efficiency of the method.
  3. Although the authors emphasize that the subset can be determined before pretraining, the method still relies on several heuristic design choices and manually selected hyperparameters. For example, the STL applicability thresholds in Stage 0 (e.g., missing rate, CV, and ACF floor) and the use of global medians to binarize F1-F6 are all empirically specified, yet the paper does not provide sensitivity analyses to show the robustness of the results to these choices. In addition, as shown in Table 2, the better retention ratio (25% vs. 50%) varies depending on the architecture and evaluation metric, with no clear prior criterion for selecting between them.
  4. The presentation of the paper needs improvement. For example, the main paper provides only a very brief description of four stages, while many important design details and implementation choices are deferred to the appendix. In addition, several components are presented without sufficient explanation of the underlying design rationale. As a result, the method can come across as somewhat thin in the main text, with the contribution appearing to be more of an engineering pipeline than a well-justified methodological design.
Questions for authors

See Weaknesses.

8 comments

Thank you for recognizing the zero-feedback setting and the organization of the method. We have added experiments and related explanations regarding the baseline comparisons, practical costs, and design rationale you raised, and respond to each point below.

W1 Representativeness of the baselines and comparisons with stronger methods

Following your suggestion, we have added stronger baselines, strengthening the comparisons mainly in two ways. On the one hand, we added new methods and restored BLAST's original selection configuration; on the other hand, we expanded the scope of the data, models, and repeated experiments to examine whether these comparison results still hold.

(1)

We added Training Diet and LP-Mirror, as well as a comparison that retains BLAST's original statistical features, encoding, UMAP projection, and grid selection configuration while disabling only Grid Mixup; this comparison is still labeled BLAST-style in the table. This both introduces new selection methods and retains BLAST's own statistical indicators and selection procedure, allowing a more direct comparison of the effectiveness of their respective complete frameworks.

(2)

Compared with the original experiments, we expanded the candidate corpus from approximately 600,000 to approximately 2,170,000 sequences and increased the number of training seeds from 5 to 7. The added comparisons cover five architectures, MOMENT, PatchTST, Timer, Chronos2-small, and Toto2-22M, evaluated on 97 downstream evaluation configurations. The results show that TRACS has lower forecasting errors than BLAST-style and Training Diet across all reported models and retention ratios, while TRACS and LP-Mirror each have advantages under different settings. Therefore, after adding these strong baselines and broadening the comparisons, TRACS remains competitive, with further support for its advantages over BLAST-style and Training Diet. The detailed results are shown in the table below.

Comparison of forecasting errors. Lower geometrically aggregated MASE is better, and we report the mean ±\pm standard deviation across 7 seeds.

RetainedMethodMOMENTPatchTSTTimerChronos2-smallToto2-22M
100%Full1.891±0.1241.891 \pm 0.1241.823±0.1321.823 \pm 0.1321.809±0.1461.809 \pm 0.1461.611±0.1081.611 \pm 0.1081.699±0.0661.699 \pm 0.066
25%Random1.783±0.0231.783 \pm 0.0231.814±0.1031.814 \pm 0.1031.885±0.1101.885 \pm 0.1101.529±0.0061.529 \pm 0.0061.695±0.0061.695 \pm 0.006
25%Stratified1.755±0.0721.755 \pm 0.0722.278±0.5102.278 \pm 0.5101.943±0.3191.943 \pm 0.3191.519±0.0731.519 \pm 0.0731.684±0.0151.684 \pm 0.015
25%BLAST-style1.804±0.1331.804 \pm 0.1331.778±0.2091.778 \pm 0.2091.899±0.1911.899 \pm 0.1911.514±0.0871.514 \pm 0.0871.685±0.0131.685 \pm 0.013
25%Training Diet1.729±0.0591.729 \pm 0.0591.736±0.0701.736 \pm 0.0701.787±0.2461.787 \pm 0.2461.513±0.0651.513 \pm 0.0651.673±0.030‾\underline{1.673 \pm 0.030}
25%LP-Mirror1.623±0.016\mathbf{1.623 \pm 0.016}1.609±0.043\mathbf{1.609 \pm 0.043}1.604±0.026\mathbf{1.604 \pm 0.026}1.471±0.010\mathbf{1.471 \pm 0.010}1.762±0.0181.762 \pm 0.018
25%TRACS1.661±0.026‾\underline{1.661 \pm 0.026}1.723±0.398‾\underline{1.723 \pm 0.398}1.749±0.198‾\underline{1.749 \pm 0.198}1.508±0.046‾\underline{1.508 \pm 0.046}1.661±0.042\mathbf{1.661 \pm 0.042}
50%Random1.838±0.0271.838 \pm 0.0271.879±0.0771.879 \pm 0.0771.861±0.094‾\underline{1.861 \pm 0.094}1.525±0.006‾\underline{1.525 \pm 0.006}1.696±0.007‾\underline{1.696 \pm 0.007}
50%Stratified1.897±0.0871.897 \pm 0.0871.957±0.1511.957 \pm 0.1511.911±0.1671.911 \pm 0.1671.653±0.0781.653 \pm 0.0781.759±0.0371.759 \pm 0.037
50%BLAST-style1.887±0.0671.887 \pm 0.0671.912±0.1671.912 \pm 0.1671.896±0.1721.896 \pm 0.1721.630±0.0601.630 \pm 0.0601.756±0.0391.756 \pm 0.039
50%Training Diet1.952±0.1221.952 \pm 0.1221.886±0.0821.886 \pm 0.0821.880±0.1581.880 \pm 0.1581.653±0.0771.653 \pm 0.0771.774±0.0531.774 \pm 0.053
50%LP-Mirror1.677±0.055\mathbf{1.677 \pm 0.055}1.856±0.096‾\underline{1.856 \pm 0.096}1.899±0.1561.899 \pm 0.1561.465±0.041\mathbf{1.465 \pm 0.041}1.745±0.0361.745 \pm 0.036
50%TRACS1.809±0.056‾\underline{1.809 \pm 0.056}1.840±0.125\mathbf{1.840 \pm 0.125}1.832±0.062\mathbf{1.832 \pm 0.062}1.620±0.0671.620 \pm 0.0671.688±0.044\mathbf{1.688 \pm 0.044}

W2 Actual runtime and resource costs of the selection process

After you raised the issue of practical costs, we also collected two sets of timing measurements, namely the runtime of the selection algorithm itself and the time for a single model training run. We hope to present these two parts clearly so that readers can assess the practical value of data selection methods alongside their forecasting performance. The related results and timing explanations will also be added to the revised manuscript.

(1)

At both the 25% and 50% retention ratios, TRACS's selection algorithm takes less time than BLAST-style, Training Diet, and LP-Mirror. Together with the preceding results, TRACS achieved both lower forecasting errors and a shorter runtime for the selection algorithm itself than BLAST-style and Training Diet in this set of experiments. This is the practical advantage we hope to further illustrate through these timing measurements, with detailed results provided in the table below.

Selection algorithm runtime and training time for reference, in minutes.

Method25% retained (min)50% retained (min)
Random0.00140.0015
Stratified0.03080.0342
BLAST-style46.0945.03
Training Diet108.8289.39
LP-Mirror48.3261.53
TRACS37.6241.31
Full0.000.00

(2)

We also compiled the time for a single training run for the five architectures as a reference for the selection algorithm runtime. Training times differ across models, and listing the two sets of times separately allows readers to compare how much time the selection algorithm and a single training run each require for the model they actually use.

Single-run training time, mean ±\pm standard deviation

ArchitectureMean ±\pm std (min/job)Recorded jobs
MOMENT111.31±29.06111.31 \pm 29.0691
PatchTST77.85±1.6377.85 \pm 1.6391
Timer53.35±13.5853.35 \pm 13.5891
Chronos2-small122.85±1.48122.85 \pm 1.4891
Toto2-22M125.28±4.00125.28 \pm 4.0091

W3 Heuristic rules, hyperparameters, and retention ratios

We carefully reviewed your comments on these empirical settings. There are three main points to clarify here, namely the basis for pattern encoding, the reasons for setting the STL routing thresholds this way, and how the retention ratio is determined before pretraining.

(1)

Median-based encoding follows the grouping approach in Section 5.2 of TIME. We decided in advance to use median-based splits, with the specific split values calculated from the candidate corpus, without needing to repeatedly choose split points based on the downstream model's forecasting performance. Appendix G of TIME also discusses how overly fine partitions can easily lead to sparse joint groups. Therefore, we use relatively coarse binary encoding to check for missing patterns while retaining continuous features for actual sample selection, accommodating both pattern verification and finer distinctions among samples.

(2)

We set the thresholds in STL routing according to the respective issues they need to address. For the ACF threshold, the NIST statistical handbook provides approximate reference bounds for autocorrelation under the white-noise assumption, and we adopt its approximate form of 2/L2/\sqrt{L} to account for sampling fluctuations at different sequence lengths. We use 0.15 as the minimum correlation strength, a value also used by Lotze et al. (2008) in screening public health time series for seasonality. The upper limits of 50% for the missing-data rate and 5% for the sampling-interval CV are empirical tolerances we set in advance; the former is intended to control the proportion of imputed data to avoid excessive dependence of feature estimation on imputation results, while the latter is intended to identify sequences with insufficiently regular sampling intervals and route them to the fallback procedure. Kwok et al. (2023) analyzed the effects of the missing-data proportion and gap distribution on STL trend extraction, and Rehfeld et al. (2011) analyzed the effects of irregular sampling on correlation estimation, so these studies can serve as references for the above processing choices. All these settings were determined before sample selection and were not repeatedly adjusted based on downstream forecasting results.

(3)

Regarding the retention ratio, TRACS defines the problem as data selection given a target subset size mm. mm is the data budget specified before selection begins, and we focus on how to select more useful samples at this data scale without using downstream forecasting results to adjust the retention ratio. In practice, users can set mm in advance according to the corpus size they can accommodate and their storage and data processing conditions. The 25% and 50% ratios are two budgets we fixed before starting the experiments to observe selection performance at different degrees of data reduction. Different models perform differently under the two budgets, but users can still first determine the data budget they can afford and then complete sample selection. We will report the results for each model under these two budgets separately to make the comparison conditions clearer.

Reviewer_uxhoOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TRACS, a model-feedback-free data selection method for time-series pretraining. The method first extracts statistical features from time series, organizes the data using GMM clustering, and allocates balanced sampling quotas across clusters. It then employs coverage-aware sampling and a swap-based repair mechanism to preserve diverse temporal patterns. Experiments on MOMENT, PatchTST, and Timer show that TRACS improves zero-shot forecasting performance under the same pretraining token budget.

Critical strengths
  1. The paper addresses data selection for time-series foundation model pretraining and attempts to select representative training data without relying on model feedback.
  2. The method is clearly described and provides explicit pattern coverage guarantees under certain assumptions.
  3. The experiments cover three different model architectures and include multiple baseline comparisons and ablation studies under a fixed training token budget.
Critical weaknesses
  1. The method mainly combines existing techniques, including statistical feature extraction, GMM clustering, Sobol sampling, and sample replacement. Its core novelty is relatively limited.
  2. The coverage guarantee is based on seven-bit binary patterns derived from handcrafted statistical features. However, it remains unclear whether these predefined patterns adequately capture the temporal diversity relevant to downstream forecasting.
  3. The theoretical analysis mainly guarantees pattern coverage and distributional approximation in the feature space, without directly establishing a connection to downstream forecasting performance. Therefore, the practical significance of these theoretical results remains unclear.
  4. The BLAST-style baseline modifies the feature representation of the original BLAST method and removes Grid Mixup, making it difficult to assess the actual improvement of TRACS over the original method.
  5. The experiments are limited to univariate time series and zero-shot forecasting, without evaluation on other time-series tasks or larger foundation models.
Questions for authors
  1. How can the authors demonstrate that the seven-bit binary statistical patterns adequately represent the temporal diversity relevant to downstream forecasting?
  2. Could the authors provide empirical analyses to examine the relationship between the theoretical pattern coverage and distributional approximation bounds and downstream forecasting performance?
  3. Could the authors include a direct comparison with the original BLAST method to better validate the effectiveness of the proposed data selection strategy?
  4. Given that some experimental comparisons and ablation results lack statistical significance, could the authors further validate the independent contributions of each component?
7 comments

Thank you for recognizing the research problem, the setting without model feedback, and the presentation of the method in this paper. Regarding your questions about the methodological contribution, theoretical basis, and experimental comparisons, we have added experiments and respond to each point below.

W1 Contributions of the Joint Design and Mathematical Proofs

We have carefully read your comment on novelty and also hope to explain more clearly what we have mainly done in this work. There are two main contributions here, namely the joint design of selection and pattern repair under a fixed budget, and the mathematical proofs developed for this design.

(1)

When designing TRACS, we originally considered how to retain both representative samples and temporal patterns of interest under a fixed budget. If continuous selection is performed only according to quotas, some discrete patterns may be omitted; however, replacements made to recover these patterns will in turn affect the original quotas and continuous distribution. Therefore, we design quota allocation, continuous selection, and pattern repair together, and make the completeness of reachable patterns a requirement that the final selected subset must satisfy. The novelty of TRACS lies in designing selection and repair around the interactions among these requirements and analyzing whether they can be satisfied simultaneously.

(2)

We also use mathematical modeling to explain why this design is feasible and what effects repair will have. We prove that, when the budget is feasible, repair can recover all reachable patterns with only as many replacements as the number of initially missing patterns, while preserving the subset size and introducing no new omissions. For the quota drift and perturbations to the empirical distribution in the six-dimensional continuous feature space caused by replacements, we provide explicit upper bounds, giving a rigorous mathematical basis for how coverage can be achieved and what costs repair entails. These results cannot be obtained directly from the existing properties of GMM clustering or Sobol sampling and require proofs for the entire selection and repair procedure.

W2 and Q1 Seven-Bit Patterns and Temporal Diversity Relevant to Forecasting

In light of your comment, we would like to clarify the two considerations behind our use of seven-bit encoding. First, these attributes are already supported by analyses related to forecasting, and second, we also retain continuous features during selection so that pattern checks and fine-grained selection can work together.

(1)

Our attribute selection and pattern encoding draw on the structural analysis framework of TIME. TIME uses seven types of attributes, including trend, seasonality, residual dependence, spectral complexity, and stationarity, binarizes the first six continuous features using global medians, and directly retains the stationarity indicator to form a seven-bit code for pattern assessment. Its attribute-based grouping analysis shows that model performance differs across groups. These existing results provide an empirical basis for our selection of the relevant attributes and for using groups to characterize temporal differences related to forecasting. TIME Sections 5.2 and 6.3 and Table 5

(2)

We follow this framework and make targeted adaptations, using combinations of these attributes to check whether structural categories of interest are omitted after the corpus is reduced. At the same time, TRACS still uses continuous-feature retrieval and a 27-dimensional statistical representation to organize and distinguish samples more finely. Thus, the seven-bit encoding is mainly used to check which patterns are retained, while the continuous features continue to distinguish finer differences between samples. We hope that, through this combination, we can apply structural attributes supported by existing forecasting analyses to pretraining data selection while retaining the continuous feature information needed for selection.

W3 and Q2 Practical Significance of the Theoretical Analysis and Its Empirical Support

Regarding the practical significance of the theory, we originally hoped to use mathematical modeling to better guide the design of the selection procedure. In light of your comment, we respond to this question from two aspects, namely how the theory helps the design and what the additional ablations show.

(1)

We conduct these theoretical analyses to clarify the conditions under which coverage can be achieved under a fixed budget, how omissions can be repaired, and how much repair affects the original selected subset. For the deviation between the final subset and the rebalanced target distribution in the six-dimensional continuous feature space, we provide upper bounds from three aspects, namely quantization error, differences in representative weights, and repair perturbations. This allows us, when designing selection and repair, to analyze whether the coverage requirements are feasible and what effects completing coverage has, rather than only considering whether the procedure can be executed.

(2)

We added Timer ablation experiments under the same training token budget. The results show that, after the full configuration recovers the missing patterns, its forecasting performance improves compared with No Swap; although proportional quotas and random within-cluster selection also cover the same full set of patterns, their forecasting performance still falls short of the full configuration. This indicates that, in addition to retaining all patterns, how the budget is allocated and which specific samples are selected also affect forecasting performance. Therefore, we design quota allocation, continuous selection, and pattern repair together, with the theoretical analysis providing a basis for this procedure and the ablation results further supporting the practical role of these designs in forecasting. The relevant results are shown in Table 2 below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.