acceptodds
Under review as a conference paper at ICLR 2027

Investigating the effects of patch size in Time Series Foundation Models

Abstract

Time series analysis is a core task across science, industry, and decision-making, and transformer-based foundation models have recently emerged as a promising unified approach. A critical step in designing transformer-based Time Series Foundation Models (TSFMs) is tokenization. Most existing TSFMs use patch-based tokenization, making the patch size a central design choice that controls temporal resolution, token sequence length, and computational cost. In this work, we systematically study the role of patch size and architecture choice in time series foundation models, two decisions that are typically studied independently but, as we show, are deeply coupled. We compare encoder-only and decoder-only forecasting models trained on the same data mixture and with the same optimization budget, varying the patch size within each model class. The two architectures respond differently: across 97 GIFT-Eval tasks, encoder-only models improve as patch sizes decrease, whereas decoder-only models perform best at an intermediate patch size, reflecting a dataset-level preference that shifts towards larger patches as the forecasting horizon grows. On UCR-91 classification with frozen representations, smaller patches again help the encoder, while the decoder is hardly affected. Encoder-only models also benefit more from longer context windows, though gains saturate beyond 2048 context length. The two architectures occupy complementary positions on the accuracy–compute trade-off: decoder-only models are more efficient, while encoder-only models achieve higher accuracy. These findings suggest that patch size and architecture should not be chosen in isolation, but should both follow from the target forecasting horizon and deployment constraints.

Primary area:
learning on time series and dynamical systems
Keywords:
Time series, foundation models, tokenization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vgtaOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This empirical study examines the interaction between patch size and encoder-only versus decoder-only TSFM training recipes. The authors train both model families from scratch at approximately matched parameter scales, using a common pretraining data mixture, number of optimization steps, and basic hyperparameters. Patch size is varied within each family, and context length is additionally varied for the encoder. Evaluation includes the 97 GIFT-Eval forecasting tasks, linear classification probes on frozen representations across UCR-91, training FLOPs, and CPU inference latency. In the main setting of approximately 260M parameters and 100k training steps, encoder forecasting improves as patches become smaller, whereas the decoder obtains its best aggregate forecasting results around P=64. The decoder's task-level preferred patch size varies across forecast-horizon groups. Smaller patches also benefit encoder classification. The appendix provides two important qualifications: the forecasting optimum for an approximately 35M-parameter encoder shifts to P=16, and comparisons of intermediate checkpoints with matched patch-token budgets favor coarser patches trained for longer, although those comparisons are confounded by differences in learning-rate annealing.

Critical strengths
  1. A reasonably systematic controlled study of a practical design question. The two families share the data mixture and number of optimization steps. The encoder's internal masking span is also fixed at 128 time points, avoiding a change in the masking task itself when patch size changes. Aggregate GIFT-Eval scores, task-level win rates, and context-length sweeps provide complementary evidence. For example, at C=2048, the P=8 encoder obtains the best MASE on 57/97 tasks and the best CRPS on 64/97 tasks (p. 6). This is useful evidence for design choices within the evaluated model families.
  2. Evidence across tasks, model scales, and budgets. Frozen linear probes on UCR-91 provide observations on another task, and the approximately 35M-parameter models reveal scale dependence. Appendix A.6 reports that matched-token checkpoint comparisons favor coarser patches. The authors also specify the conditions of the single-CPU latency measurements and acknowledge that the large speedups at long horizons are not accompanied by accuracy evaluations. This transparency strengthens the credibility of the empirical contribution.
Critical weaknesses
  1. The findings are specific to training recipes and budget definitions. The model families differ in tasks, normalization, the arcsinh transformation, positional encoding, heads, and training quantiles (pp. 3–5; Appendix A). Table A.1 uses 99 encoder quantiles versus 9 decoder quantiles; evaluating both on nine does not remove this objective difference. Equal optimization steps also do not imply equal FLOPs. In Table A.4, 2P@100k beats P@50k in every matched-token comparison, but the 50k checkpoints have not completed annealing, so this does not establish greater compute efficiency for coarser patches. The abstract and design recommendations should tie conclusions to the tested recipe, scale, and budget, rather than imply general architectural properties or compute-optimal scaling laws.
  2. Horizon trends and boundary configurations need stronger validation. Table A.2 (p. 14) reports 1/2/3/3/3 seeds for decoder P=16/32/64/128/256, with additional seeds allocated according to initial performance. A non-converged small-decoder run at P=16 was excluded (p. 16), leaving important boundary settings insufficiently replicated. The short/medium/long groups contain 55/21/21 tasks, so their win rates may reflect dataset composition as well as horizon. Establishing that larger patches become preferable as a task’s horizon grows requires paired analysis within shared datasets and frequencies. Current evidence supports an association across groups without fully attributing it to horizon.
Questions for authors
  1. Scope clarification. Would the authors limit the main findings to approximately 260M parameters, the current data mixture, and the fixed-step recipe? Are fully annealed results with matched FLOPs or token budgets available? Otherwise, can compute-optimal patch size remain an open question rather than an inference from intermediate checkpoints?
  2. Analysis of existing results. Within datasets and frequencies evaluated at multiple horizons, how do patch rankings change? Please provide seed-level results for key decoder configurations. Do trends persist when stability statistics include failed runs or comparisons use matched seed sets?
  3. Efficiency verification. Figure 5 uses one warm-up and one timed iteration (Appendix A.5). Is its low-latency Pareto frontier stable under repeated measurements with reported distributions? Please retain the qualification that Figure 6’s up-to-30× speedup applies to the specified CPU and generation strategies, without corresponding accuracy measurements.
1 comment

We thank the reviewer for their thorough analysis of the paper and insightful comments which allowed us to significantly improve the paper. We have provided a general overview of the modifications in the general comment and we provide answers to the points raised below.

W1 + Q1.

  • Architectures: The model families indeed differ in several architectural choices, but they are each based on SOTA models from their families: encoder-only models on PatchTST-FM and decoder-only models on TimesFM and Toto. The difference in the number of quantiles predicted isn’t significant as shown in the PatchTST-FM paper (Appendix F.1.2, [1]) and is only a remnant of the original architecture.
  • Token budget: We now provide results for fully annealed P@50k and we have dedicated Section 3.3 to the comparison of equal token budget configurations. In all cases, for the same token budget it is better to train longer with a larger patch size than to use a finer one trained for fewer steps.

W2 + Q2.

  • Configurations: We now provide 5 seeds for all the 260M models, which are the main object of the paper. Regarding the small 35M configurations, we think that the non-converged run actually strengthens the results: it shows that the boundary value P=16 is unstable. (Appendix C), hinting to use larger patch sizes to get good forecasting with decoder architectures.
  • Horizon trends: We now dedicate Section 3.6 to the analysis of the two architectures’ performances based on horizon on controlled synthetic data.

We further provide more fine-grained results based on the GIFT-Eval forecasting lengths in Appendix A.5 where we have restricted every term to the 21 dataset–frequency pairs that GIFT-Eval scores at all three terms. On these 21 datasets, the encoder continues to favor fine patches. For the decoder, the MASE gap between P=16 and the intermediate patch sizes widens as the horizon grows, showing that the pattern persists with the dataset and frequency composition held fixed (Fig. A.3).

Q3. We agree that the latency results are limited in scope. We have moved it to the appendix since it is not the main focus of the paper. Nonetheless, we have conducted further experiments in a more controlled environment that is described in Appendix A.6, with more warm-up iterations, more timed iterations and in a clearly specified environment. Qualitatively, the results are the same for CPUs. We now discuss them more in depth and make it clear that they are only applicable to low throughput CPU regimes. We also provide results obtained on an A40 GPU, on which the encoder-only models are faster than the decoder-only models due to the accelerations.

[1] Y. Wen, W. M. Gifford, C. Reddy, L. M. Nguyen, J. Kalagnanam, and A. A. Julius. Revisiting the generic transformer: Deconstructing a strong baseline for time series foundation models, Neurips 2026

Reviewer_Y2FoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents an empirical study of how temporal patch size interacts with architecture choice (encoder-only vs. decoder-only) in patch-based time-series foundation models. The main findings include: (1) Encoder-only models improve monotonically as patche size decrease. (2) Decoder-only models follow a U-shaped curve, leading the authors to argue that the aggregate optimum partly reflects the benchmark's horizon mix. (3) The encoder's preference transfers to classification, while the decoder is nearly insensitive.

Critical strengths
  1. The paper correctly identifies that published TSFM patch-size recommendations confound corpus, scale, objective and tokenization, and then removes these confounds by carefully designed controlled experiments.
  2. The paper's conclusions offer actionable guidance, facilitating improved architecture selection and patch size hyperparameter tuning.
Critical weaknesses
  1. The authors reach convincing conclusions via controlled experiments, but these conclusions remain empirical. The absence of a mechanism-level discussion means they may be constrained by the scope of the experiments conducted.
  2. The conclusion that the aggregate optimum is a compromise in decoder-only expermients is inferred rather than shown. The argument jumps from win-rate counts to the shape and location of the aggregate curve, yet win rate and the aggregate metric are different statistical objects. Moreover, in figure 3, the paper's own numbers partly undercut a monotone story: on short-term tasks CRPS is best at P=64 (20 of 55 tasks), not at P=16, so the claim that "the preferred patch size increases with the horizon" holds for MASE but not for CRPS at the short end.
Questions for authors

Why do the encoder-only and decoder-only architectures behave so differently, and what mechanism explains the divergence in their patch-size preferences?

1 comment

We thank the reviewer for their comments, based on which we have added more material to the paper and improved it. We have provided a general overview of the modifications in the general comment and we provide answers to the points raised below.

W1 + Q. We agree that the original manuscript did not sufficiently show why the conclusion holds. We now dedicate Section 3.6 to the interpretation of the difference between the mechanisms in which we provide our hypothesis and experiments supporting it. We have added controlled experiments on 10,000 held-out KernelSynth series to test whether autoregressive feedback explains the different patch sizes preferences. In short, the U-shape curve for the decoder architectures can be explained from the tradeoff between the increasing task complexity for larger patch size, and the error rollout that compounds with smaller patch sizes, while the encoder architecture only sees the task complexity. (Figs. 6–7; details in Appendix D).

W2. We agree that win rates and the aggregate metrics are different statistics and on the short end, the reviewer is right that the short-term CRPS winner is P=64. We now report the aggregate metrics separately for each forecasting term (Appendix A.5), which allows us to reach the trade-off conclusion based on aggregate metrics alone. Furthermore, the analysis conducted in Section 3.6 strengthens the claims. This pattern also holds when we evaluate the same 21 dataset pairs at all three horizons, where (P=16) falls further behind the intermediate patch sizes as the horizon increases (Fig. A.3).

Reviewer_mQZgOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work presents an empirical study on the effect of patch size for time-series foundation models. Many TSFMs use patching to tokenize a fixed number of timesteps, thereby splitting the input time-series into a sequence of contiguous temporal motifs i.e. patches. This is an important design axis that has bearing on the computational cost, the representations learned by the model, and the interpretability. The authors make two central claims, under matched training data and compute budget:

Claim 1 is that encoder-only models perform better with short patch size, while decoder-only models perform better with intermediate patch size, and with large patch size for long-horizon forecasting tasks. The experiments for this claim are based on GIFT-Eval.

Claim 2 is that this observation carries over to classification tasks with frozen representations, i.e. encoder-only models benefit from short patch size (and longer context windows, saturating at 2048 timesteps). The experiments for this claim are based on UCR-91.

The authors also frame their study in terms of the accuracy-compute tradeoff between encoder-only and decoder-only architectures. The encoder-only model the authors train is based on PatchTST-FM, and the decoder-only model the authors train is based on TimesFM and Toto.

Critical strengths
  • The authors present a thorough body of previous and related work, and consider the relevant architectures and models, including: PatchTST-FM, Chronos-2, TimesFM, Moirai 2.0, and Toto.

  • The training set is well-constructed, to combine GIFT-Eval Pretrain with two synthetic generators: KernelSynth and RandomPatterns (the latter is introduced by the authors).

  • The experimental details and setup are described clearly and concisely in Section 2 (with additional details in Appendix), making it easy to read and understand what the authors did for their experiments.

Critical weaknesses
  1. Lack of error bars, quantification. While the existing empirical results do seem to back up the main claims, they are in my view quite unpolished. I would like to see error bars in the reporting of these metrics, which are aggregated across several datasets. I would like to see some measures of statistical significance to quantify the numerical comparisons.

  2. Limitation to univariate forecasting. The present study focuses entirely on univariate forecasting, despite the current paradigm and best models being for multivariate tasks. This greatly limits the relevance of the current study.

  3. Limitation to single model architecture per class. I understand that it is expensive to train multiple architectures. However, the authors choose a single architecture for each class (i.e. encoder-only and decoder-only) and it is not obvious to me why these architectures should be representative of their classes. It is also quite possible that the patch-size dependence is largely dependent on specific architectural choices, which could be a confounding factor. A natural way to handle this is through architectural ablation experiments.

  4. Lack of mechanistic insight or grounding in the quality of the forecasts.: I trust that the authors did their due diligence in making sure that the models they trained at various patch sizes are at comparable compute budget and "well-trained". But I would like to see a more clear grounding of the patch-size results in the forecast quality. Specifically I mean that often these TSFMs on long-horizon forecasting tasks produce degenerate forecasts i.e. either mean regression or context parroting. Are the long-horizon forecast results skewed by mean-regressed forecasts that essentially flatline, or by forecasts that just repeat the same pattern? More broadly, I would expect the choice of patch size to be task-dependent. On time-series with fast frequencies, I would expect short patch size to be advantageous, while on time-series with slower frequencies, periodicity, seasonality, I would expect long patch sizes to be sufficient. Overall, it is important to see what the forecasts actually look like, and to interpret when and why the shorter patch sizes are advantageous. This grounds the story much better than solely looking at the aggregated metrics, which can hide a lot of the nuances. The CPU inference latency results seems to be limited in scope. From my understanding, the dominant effect on the encoder-only vs decoder-only speed comparison rests on the authors' claim that decoder-only models tolerate larger patches, since at equal patch size "both families have similar latency". Furthermore, the choice of batch size 1 seems to also be not aligned with practice.

Questions for authors
  1. The authors mention that Moirai, Kairos, and TimeSqueeze use mechanisms to explicitly handle multiple temporal scales, but that recent studies have claimed adaptive patching does not yield consistent performance improvement over fixed patching. Do the authors have any perspective as to why this is the case?

  2. I want to confirm my understanding of the experiments: Halving the patch size i.e. from 16 to 8 should roughy double the training compute. Could the authors confirm again how they make sure to equalize the training budget for the various patch sizes, given this tradeoff, for an apples-to-apples comparison?

3 comments

We thank the reviewer for their assessment of the paper and thorough comments. We have provided a general overview of the modifications in the general comment and we provide answers to the points raised below.

W1 - Lack of error bar: All the results had error bars when configurations allowed for them. To further improve the reproducibility and robustness of our paper, we made sure that the experiments that did not already have 5 seeds (260M configuration) now have 5 seeds with their corresponding error bars per seed and we further provide Wilcoxon tests to quantify the uncertainty in the win rates (Appendix A.5). For decoder only, P=16 is significantly worse than the highest win rate patch size in every forecasting term for both MASE and CRPS. For the encoder, P = 8 wins the most tasks in every term, but P= 16 and P= 32 are statistically indistinguishable from it on medium and longer term tasks.

W2 - limitation to univariate: The present work is indeed limited to univariate forecasting. The present analysis doesn’t transfer to multivariate forecasting in a straightforward manner. While we agree that this would be a very interesting and important analysis that we mention as a possible future work, we believe that the present work is still interesting to help design better TSFM.

W3 - limitation to one model per class:. The two architectures chosen follow two mainstream SOTA architectures for univariate models. The encoder-only model is PatchTST-FM, while the decoder-only model is based on TimesFM and Toto. Conducting a rigorous and complete architectural ablation study would be very interesting, but beyond the scope and computational budget of our paper, we limit our conclusions to the two architectures that we have analyzed.

W4 - lack of mechanistic insights: We agree that the original manuscript did not sufficiently show why the conclusion holds. We now dedicate Section 3.6 to the interpretation of the difference between the mechanisms in which we provide our hypothesis and experiments supporting it. In short, the U-shape curve for the decoder architectures can be explained from the tradeoff between the increasing task complexity for larger patch size, and the error rollout that compounds with smaller patch sizes, while the encoder architecture only sees the task complexity. We further provide a forecast gallery (Appendix E) on time series from GIFT-Eval which provides insight into the quality of the predictions. The CPU latency subsection has been expanded and moved to Appendix A.6, see the general comment.

Q1. We believe there are two main explanations : Local signal complexity does not necessarily translate into a need for finer temporal resolution. Adaptive patching methods typically allocate shorter patches to regions with higher local variability or information density. However, these characteristics are not necessarily predictive of forecasting difficulty or of the benefit of finer tokenization. As discussed by [1], a locally complex region does not necessarily benefit from increased resolution in terms of forecasting accuracy. As we show in this work, the optimal patch size depends on the model architecture and the forecasting horizon. Consequently, allocating finer patches based solely on local signal characteristics may not align with the minimization of the model's forecasting error. A rule that adapts to the input alone can assign the same patches to the same history even when the task changes from predicting the next few points to forecasting much further ahead. This may help explain why adapting to local input structure yields inconsistent gains over a fixed patch size tuned for the forecasting task.

Q2. In the original setting, we have not used equal compute, but equal number of training steps, 100k. We have expanded this to have more configurations trained at equal token budget and discuss it in Section 3.3 and Appendix A.7.

[1] F. Zucchi, Y. Xie, C. Zhang, K. Luo, T. Lampert, and Z. Li. Adaptive patching is harder than it looks for time-series forecasting, 2026.

Reviewer_mQZgOpenReview ↗

Response to rebuttal comment

Thank you for the many additional experiments and especially the section on the mechanistic interpretation of the patch size dependence. It seems like the patching paradigm will remain the standard representation for SOTA TSFMs and this work quantifies the effect of patch size adequately. My concerns have been addressed and I have raised my score accordingly.

Thank you for your response. We sincerely appreciate your constructive feedback, which has helped us improve the paper.

Comments on the paper

1 comment

We thank the reviewers for their insightful comments. Overall, reviewers acknowledge that the patch-size effect is entangled with other changes in existing literature and that our systematic evaluation offers “actionable guidance”. Following their feedbacks, we’ve made the following general modifications to the paper:

  • We have added a subsection (Section 3.3) that discusses the comparative performances of models with an equal training token budget across multiple configurations.
  • We have added a subsection (Section 3.6) that attempts to explain the mechanistic differences between the two architectures, through experiments on controlled synthetic data.
  • We have expanded and moved the subsection that discusses latency experiments to the appendix (Appendix A.6). Nonetheless, we’ve conducted the latency experiments in a more controlled and reproducible manner and performed experiments on GPUs too. The absolute values obtained in the CPU experiments have changed, but the trend has not.
  • We have evaluated the main models analyzed in the paper on the TIME and BOOM benchmarks too and have obtained similar results. We have added these results in the appendix (Appendix B) and mentioned them in the main paper.

We have further addressed each reviewer’s concerns through a dedicated comment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.