acceptodds
Under review as a conference paper at ICLR 2027

One More Token: Patch-Boundary Revisions in Time-Series Foundation Models

Abstract

Fixed-patch time-series foundation models group past observations into tokens before making a forecast. Adding one observation can create a new token and produce a sharp change in the prediction. We study this transition in Chronos-Bolt, TimesFM, and Moirai by extending the same history one observation at a time. Among the sampled local transitions, the token boundary has the largest average revision in 46 of 48 model, dataset, and horizon settings. Changing Moirai’s patch size moves the revision peak to the new boundary. An attention intervention shows that allowing other tokens to read the new token contributes to the change, with additional TimesFM experiments confirming the effect on previously unused series. We then explain how these revisions affect accuracy. An exact squared-error decomposition separates the size of the revision from its direction relative to the model’s existing error. On generated autoregressive series, Moirai’s revision corrects that error, with an expected-loss change 2.47% of the irreducible forecasting variance lower than at nearby non-boundary transitions. Real-data results show that the loss effect varies across models and datasets. Together, these findings connect a discrete change in input representation to both forecast revisions and their accuracy.

Primary area:
learning on time series and dynamical systems
Keywords:
time series foundation models, time series forecasting, patch tokenization, context-length sensitivity, attention interventions

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_5tFDOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper reports a finding observed in time-series models: extending the available history by one earlier observation can lead to a change in the forecast. The author conducts experment on three models to demonstrate the finding.

Critical strengths
  1. The paper’s finding that adding historical observations can change the forecast is interesting.
Critical weaknesses
  1. The paper is poorly organized and presented. Both the introduction and the experiment section cite figures and tables that appear several pages after they are cited, some even in the appendix. The logical structure of section 3 is unclear. This paper is hard to read, and it reads more like an experimental report.
  2. The experiments are not rigorous. Although the paper conducts experiment on three models, some experiments do not report results for all three models, and the author does not clearly explain why. It is unclear whether these experiments are inapplicable to certain models, or whether unfavorable results have been omitted. See Tables 2, 3, and 4.
Questions for authors

The authors should explain the missing experimental results and improve the organization of the paper.

Reviewer_eAy7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies abrupt forecast revisions when extending the context activates a new patch token, using attention interventions and loss analysis to examine their causes and consequences.

Critical strengths
  1. Fixing the forecast cutoff while extending the oldest history provides a controlled way to study representation changes without changing the prediction target.
  2. Experiments across three foundation models, together with patch-size and normalization controls, provide convincing evidence that revisions track patch boundaries.
  3. The attention interventions and loss analysis distinguish forecast instability from accuracy degradation, showing that large revisions can sometimes improve predictions.
Critical weaknesses
  1. The context-expansion setting differs from common fixed-length rolling-window inference, leaving the deployment relevance of the observed boundary effect unclear.
  2. Activating a new token also introduces a sparsely observed patch, so the experiments do not fully separate token-count effects from partial-patch encoding and padding.
  3. The accuracy analysis has limited coverage, and the elementary loss decomposition explains revision outcomes without predicting when a boundary revision will help or hurt.
Questions for authors
  1. Does the effect remain important under fixed-length rolling-window inference when forecasts are compared on shared future timestamps?
  2. Is the discontinuity specific to non-overlapping patches, or does it also occur with overlapping sliding-window patching?
  3. Which controls distinguish the effect of adding a token from the effects of partial-patch occupancy and padding?
  4. Does masking the leading token at nearby non-boundary lengths produce comparable changes, or is attention sensitivity specifically amplified when a token first appears?
  5. Do the accuracy conclusions extend to processes with longer historical dependencies, where the newly added observation can contain genuinely useful predictive information?
1 comment

Thanks for the response, I will keep my score which is already positive.

Reviewer_L5hHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the effect of adding a single additional observation at the beginning of the input sequence to a time series foundation model. In particular, the study focuses on patch-based models, and on the case where the additional observation induces the creation of an additional patch. The paper shows that the forecasting error changes more significantly when the additional observation generates a new patch, compared to when it does not. This is shown in controlled and real-world settings, using bootstrap sampling to compute confidence intervals on the reported values. Through an intervention study, the authors connect this effect to the visibility of the additional patch in the attention mechanism. Ultimately, the effect appears to be model and dataset dependent.

Critical strengths
  • The paper investigates a potentially interesting phenomenon that, to the best of my knowledge, has not been previously characterized for time series foundation models. The effect is investigated across multiple models and datasets, providing reasonably convincing evidence for its existence.
Critical weaknesses
  • The paper is extremely difficult to follow, despite the underlying intuition being relatively simple. The details of the experimental setup constantly vary, while the rationale for each choice, and the question each experiment seeks to answer, are not consistently presented together. For example, Section 3.2 introduces several experimental configurations before their purpose and context is clear. Section 4.1 rapidly moves between experimental settings, without a clear separation of the questions being addressed, their relevance, and the conclusions each experiment supports. This makes it difficult to piece everything together and assess how the different results complement each other, leaving this burden to the reader.
  • The work provides little understanding of what causes the revisions. The attention intervention provides evidence that boundary token visibility contributes to the revision. However, this is mostly additional confirmation for the presence of the effect, which largely disappears when the responsible token is masked from the main pathway the model uses to process information. The intervention does not explain why a patch containing only one valid observation has such a significant impact. Furthermore, the decomposition of the change in squared error into revision energy and error alignment seems mostly a different perspective on previous results, and does not provide further explanation of the effect or a generalizable principle that predicts it. The real-data experiments ultimately show that the impact on accuracy depends on the model-dataset pair, without explaining this dependence or providing actionable insight into how it might be mitigated. Taken together, the paper convincingly establishes a phenomenon, but does not provide a sufficiently deep understanding of the underlying behavior that produces it or a demonstration of relevant practical implications.

Overall, I believe the paper identifies a potentially interesting phenomenon. However, the limited explanatory contribution, combined with the difficulty of following and evaluating the experimental narrative, makes the work read more like a preliminary empirical report rather than a complete research contribution. I would advise the authors to substantially streamline the presentation, clearly separate the rationale and conclusions of individual experiments, and provide insight beyond an empirical characterization of models' sensitivity to boundary patches.

Questions for authors
  • Are the original TimesFM results in Section 4.2 computed using the corrected implementation? If not, why were the original experimental settings not rerun?
  • What additional insight does the energy/alignment decomposition provide beyond showing that the change in expected error depends on its direction and magnitude?
  • Which additional understanding about why the revision occurs does the attention masking intervention provide?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.