- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper empirically studies how calendar covariates should be encoded for zero-shot, covariate-informed Time Series Foundation Models (TSFMs). It compares seven encoding strategies—three conventional target-agnostic (ordinal, one-hot, cyclic) and four target-based (mean target encodings over hour×weekday with variants for holidays)—across four recent TSFMs on nine hourly datasets with pronounced weekend/holiday effects, and uses a SHAP-based attribution method to analyze how models utilize these covariates. The main findings are that (i) encoding choice materially affects accuracy for all models, (ii) target-based encodings yield the largest gains on holidays and are often more robust overall, and (iii) the best encoding is model-dependent, implying that benchmarks with fixed encodings can misrepresent relative model performance.
Critical strengths
- Provides a systematic, architecture-spanning investigation of calendar encoding—an overlooked but impactful design choice in zero-shot TSFMs.
- Introduces a suite of simple, context-only target encodings that are leakage-safe (computed from past context) and pragmatically applicable at inference time.
- Extends a sampling-free SHAP methodology to multiple TSFMs for diagnosing the relative contribution of calendar covariates versus autoregressive signal.
Critical weaknesses
- Target encodings are computed as simple means without regularization/backoff; low-support categories (especially holidays) can yield high-variance encodings
- Only hourly datasets with strong calendar/holiday signals are included; lacks negative controls (datasets where calendar is weak) and other cadences
- Context length is fixed to 8192 based on Chronos-2’s limit; this may not be optimal for other models and constrains generality
- The method assumes each forecast-horizon category combination appears in context; handling of missing combinations (e.g., rare holidays within a 49-week context) is under-specified
- There is a lack of clear figures to illustrate the motivation and the pipeline
Questions for authors
See weakness
3 comments
I thank the authors for their rebuttal. Although the clarification on Appendix A.3.11 is noted, the response reinforces several of my original concerns.
Specifically, the fact that 65/110 holidays require a naive global-mean fallback at highlights the practical fragility of unregularized target encodings under shorter contexts.
Furthermore, declining to include negative controls leaves open the question of whether target encodings cause performance degradation on datasets without strong seasonality.
Coupled with the fact that the 15-minute cadence results are only promised for a future revision, I do not find sufficient grounds to raise my score and will maintain my rating of 2.
Response to Reviewer
We apologise for any misunderstanding. These additions were planned for this revision from the outset, and many new results have now been incorporated. Computing them, however, took considerable time, which caused the delay.
Rebuttal
We thank the reviewer for thoroughly reading our paper.
Weakness 1: no regularization/backoff
We deliberately use simple context means to keep the encoding parameter-free and identical across models, so that differences can be attributed to the models rather than to encoding-specific tuning. We acknowledge that holidays are a low-support category. However, our target-encoding variants already span a range of granularities, from coarse, high-support to fine, low-support holiday categories, and thus effectively cover different degrees of backoff. Even the finest variant with the lowest support (Th,W,H) outperforms all target-agnostic encodings on holidays for three of four models [ref. Table 2]. While the low support count of the Th,W,H encoding is a weakness of the specific encoding, we argue that it is not a weakness of our benchmark, as we deliberately include target encodings from high support count (Th,W (Hol = Sun)) to low support count (Th,W,H).
Weakness 2: only hourly data, no negative controls
We agree that the generalizability of our results is limited by the lack of another resolution. Therefore, we will add another experiment run with 4 of the 9 datasets that are available in 15 min resolution to the revised paper.
Regarding the suggested negative control, we thank the reviewer for this idea. However, the scope is deliberate: we study how the encoding of calendar information affects forecast performances, which presupposes that this information is relevant. Datasets with little calendar influence therefore fall outside this scope.
Weakness 3: context length fixed to 8192
Context length is not fixed to 8192, as Appendix A.3.11 contains the overall results with all models and context length 2048.
Weakness 4: missing combinations in context
We fully acknowledge that our description of unseen calendar combinations in the paper was insufficient and appreciate this comment. For context length 8192, one target encoding (Th,W,H) encounters a forecast-horizon combination that does not exist in the context in 13 of the 110 holidays of the benchmark dataset. With context length 2048 this problem occurs substantially more frequently, with 65/110 holidays containing a combination not present in the context. On non-holidays, this is never the case for both context lengths, and all other target encodings do not suffer from this problem at all. The fallback solution in this scenario is to replace the encoding for the affected day by the context mean. This is a fallback that is possible in every situation. We will integrate a description of this fallback, as well as the support counts and the number of affected windows in the revised paper.
While the differences in support count show in the comparison between target encodings, the Th,W,H encoding still shows comparable or superior performance to the non-target encodings across both context lengths. An ablation study using Th,W (Hol = Sun) as a fallback would likely improve performance. However, this would make the overall encoding more similar to the fallback, complicating the attribution of gains to the explicit Th,W,H encoding. We therefore retain its original form.
Weakness 5: missing figures for motivation and pipeline
We thank the reviewer for this suggestion. We will include a figure illustrating our pipeline in the appendix, as the limited space in the main paper requires us to prioritize figures that we consider more relevant to the central contributions of our work.