acceptodds
Under review as a conference paper at ICLR 2027

TopoPrimer: The Missing Topological Context in Forecasting Models

Abstract

Many leading time series foundation models (TSFMs) forecast individual series based predominantly on their own temporal histories, relying on implicit cross-attention to capture any broader context. It remains largely unexplored whether explicit, domain-wide population structure can improve these models. Specifically, it is unclear if a series' relative position within a population, or the global shape of the population itself, offers a recoverable signal for forecasting. To investigate this, we introduce TopoPrimer, a framework that injects the global topological structure of a series population as an explicit prior into forecasting backbones. TopoPrimer captures this structure through two frozen encodings, precomputed once per domain: a spectral relational coordinate that locates each series within the population, and a persistent-homology fingerprint that describes the population's global shape. These descriptors are injected either per-token into models trained from scratch, or via a lightweight adapter into frozen, pre-trained foundation models. We find a strict division of labor: the relational coordinate varies across series and drives accuracy on frozen backbones when history is abundant, whereas the topological fingerprint is constant within a population and relies entirely on cross-cohort structural diversity to provide predictive signal. TopoPrimer improves accuracy on three of four public benchmarks, reducing MSE by 7.4% on ECL. Notably, on the ECL dataset, the relational coordinate outperforms full fine-tuning of Chronos while training fewer than 1% of the parameters. On a large-scale internal corpus, the fingerprint reduces peak-demand error by 9 to 14% and improves cold-start MAE by 13% at launch. Because a replication on the public M5 dataset yielded no aggregate cold-start gain, we scope this zero-shot advantage to training distributions spanning structurally distinct populations.

Primary area:
learning on time series and dynamical systems
Keywords:
time series forecasting, time series foundation models, topological data analysis, persistent homology, parameter-efficient adaptation, spectral embeddings, cold-start forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_diASOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TopoPrimer adds two forms of frozen and precomputed population context into time series forecasters, a per-series spectral coordinate and a population level persistent homology fingerprint. The authors evaluate them with from-scratch Transformers and adapters for frozen foundation models. The relational coordinate improves across several public benchmark results. The fingerprint's strongest improvements appear in peak demand and cold start tests on an internal dataset. However, a public M5 cold start replication does not show overall improvement.

Critical strengths

The paper separates the roles of the two descriptors clearly, and explains under what circumstances a population level fingerprint can provide information. It makes the claims more precise than a general “topology helps” argument.

The evaluation includes meaningful controls and unfavorable results. These analyses are helpful in determining when the proposed prior hypotheses are valid and in avoiding overly generalized claims.

Critical weaknesses

Much of the public benchmark advantage seems to come from the adapter, or from the overall advantage in providing cross-series information. Therefore, the reason for a distinctive benefit of the proposed topological representation is modest.

The fingerprint's strongest evidence comes from an unreleased internal dataset, whereas the public cold-start replication is negative. Its applicability to other populations remains uncertain.

Questions for authors
  1. Can the fingerprint's cold start benefit be demonstrated on a public benchmark with several structurally distinct cohorts and multiple training seeds?

  2. Do the relational coordinate gains persist across multiple forecast origins?

6 comments
AuthorsOpenReview ↗

W1, Adapter vs. Rel vs. Fingerprint (1/3)

We thank you for the clear review.

W1: Adapter vs. Rel vs. Fingerprint

W1. "Much of the public benchmark advantage seems to come from the adapter, or from the overall advantage in providing cross-series information. Therefore, the reason for a distinctive benefit of the proposed topological representation is modest."

We agree completely that on single-population public benchmarks, predictive gains come from per-series cross-series coordinates (Rel) rather than cohort-level persistent homology (Fingerprint). Rerunning ECL without non-negativity clipping across 5 seeds illustrates this division of labor cleanly:

Per-Series vs. Per-Cohort Features on Unclamped ECL (Chronos-2 adapter, 96h horizon, 5 seeds)

Injected Structural FeatureGranularityDimTest MSEΔ vs. Plain AdapterSeed Consistency
None (Plain Adapter)-00.1730--
Rel (Default / Population-scaled 8% of N)1 vector per series256 / 260.1660 / 0.1595−4.0% / −7.8%5/5 seeds
Raw Correlation Row (CorrRow control)1 vector per series2560.1612−6.8%5/5 seeds
Persistent Homology Fingerprint (TDA)1 vector per cohort1250.1729−0.1%-
Covariance Eigenvalues (Spectral control)1 vector per cohort1250.1730−0.0%-

Next, to test whether Rel's advantage comes merely from adding an adapter to a frozen backbone or from arbitrary per-series conditioning, we ran five strict controls on unclamped ECL (k=26, 5 seeds) and held-out Weather stations:

Controls for Rel (5 seeds)

Controlled ComparisonEmpirical Result (Δ Error, 95% Bootstrap CI)What This Rules Out
1. Rel Adapter on LoRA (r=2) / Fully Fine-Tuned Chronos-2−7.4% [−9.5, −4.8] / −7.1% [−9.4, −4.5]Gains are not an artifact of freezing the backbone.
2. Parameter-Matched Widened Plain Adapter vs. Plain−1.1% (Frozen) / −0.7% (LoRA) / −0.7% (Full FT)Gains are not due to extra adapter parameters.
3. Rel vs. Shuffled Rel (across 4 backbone setups)−1.8% to −2.4% (all 95% CIs strictly exclude 0)True spectral coordinates add ≈ 2% beyond distinct series vectors.
4. Rel vs. Learned 26-dim Client Embedding−3.0% (Chronos-2) / −2.6% (TimesFM-2.5), 5/5 seeds eachA learned ID of the same size doesn't match Rel (TimesFM-2.5 MSE interval crosses 0).
5. Zero-Shot Transfer to 602 Unseen Weather StationsRel: −4.0% (5/5) vs. Shuffled Rel: +1.7% (0/5)On unseen series, arbitrary IDs hurt while projected Rel transfers.

Takeaway: Rel's gain is not the adapter, the frozen backbone, or extra parameters. About 2% of it comes from the coordinates themselves rather than from giving each series its own vector, and only Rel carries over to series the model has never seen.

Finally, where does the fingerprint beat simpler summaries? On cold start, it beats summary statistics and ID controls both per item on our internal corpus (10/10 seeds) and per domain on public M4 (5/5 seeds). Separately, its persistence statistics rank which internal cohorts will benefit, before any training:

Verified Settings Where Persistent Homology (Fingerprint) Outperforms Non-Topological Controls

SettingControlled ComparisonEmpirical Outcome
1. Per-Item Cold Start (Internal corpus, weeks 0-2, 10 seeds; full table: Table 8 in Post 4 of the General Response)125-dim TDA fingerprint of a paired reference item's early weeks vs. 7 controls computed from the exact same reference data, plus the plain modelEach control has higher MAE than TDA: 125 summary statistics +9.1%, PCA of statistics +7.5%, mean early curve +5.2%, swapped fingerprints +4.6%, and random reference IDs +15.5% (10/10 seeds on every control; every control with a reported 95% CI excludes 0).
2. Public M4 Cold Start (new series after a 1996 cut-off, 5 seeds; full table: Table 7 in Post 3 of the General Response)125-dim TDA fingerprint vs. 125 summary statistics of the same data, and vs. a learned domain IDSummary statistics +2.6%, learned domain ID +15.5% (5/5 seeds each)
3. Screening which cohorts benefit (within-corpus; 19 internal cohorts; S1Lq Q4)Persistence statistic ranks cohorts before any trainingHeld-out cohorts: median ρ = −0.69. Survives selection correction (p = 0.0004) and 13 plain statistics (partial ρ = −0.64).
AuthorsOpenReview ↗

W2 and Q1, Cold-Start Claims and M4 (2/3)

W2: Scoping the Cold-Start Claims

W2. "The fingerprint's strongest evidence comes from an unreleased internal dataset, whereas the public cold-start replication is negative. Its applicability to other populations remains uncertain."

We agree, and have revised the claims to match the evidence:

  1. What we withdraw: Your request for more seeds was right. On our internal corpus, the paper gave every item in a cohort the same fingerprint. Re-tested at 5 seeds, that version's cold-start gain (13.3%) is not significant, and its peak-demand gain (9–15%) does not replicate. We remove both from the abstract and main claims.
  2. What replaces it (our new cold-start claim): the same fingerprint, computed for each new item from a comparable earlier item instead of once per cohort. An ID cannot stand in for it, because the new item is a different item.
    • Against the plain model (10 seeds): 4.8% lower error in the launch week and 3.5% in week 1 (9/10 and 10/10 seeds). The gain fades as the item builds its own history.
    • Against simpler encodings of the same data (10/10 seeds): summary statistics are 9.1% worse, the reference item's average early curve 5.2% worse, and a random ID per reference item 15.5% worse.
    • Robust to TDA settings: other neighbor counts and landscape resolutions keep 53–97% of the gain (full table: Table 8 in Post 4 of the General Response).

Q1: Public Cold Start With Multiple Seeds

Q1. "Can the fingerprint's cold start benefit be demonstrated on a public benchmark with several structurally distinct cohorts and multiple training seeds?"

Yes: on public M4, the fingerprint beats both summary statistics and a learned domain ID on cold start, across 5 seeds. We made M4 Monthly (about 48,000 business and economic series across 6 domains) into a cold-start test by holding out series that start after a cut-off in 1996. The model is the paper's Transformer, with a learned domain ID in every arm, and each arm's settings chosen on a held-out validation cohort from the same grid. M4 has 6 domains and thousands of series entering over time, close to the setting you describe.

Public Cold-Start Replication: M4 Monthly, New Series Starting After 1996 (test sMAPE, 5 seeds)

ModelsMAPEFingerprint better bySeeds Where TDA Wins
+ TDA fingerprint7.19--
+ 125 summary statistics of the same data7.382.6% (interval excludes 0, resampling seeds and series)5/5
Naive last-value forecast (no training)7.676.3%5/5
Plain (learned domain ID only)8.5015.5%5/5

At the 2001 cut-off the fingerprint still beats the learned domain ID but ties summary statistics; at 2010 it ties both. Most other public benchmarks are a single population or a few fixed groups, so a cold-start fingerprint has nothing to vary over.

A second public dataset: Iowa Liquor Sales (public state ordering data, 2012–2019, about 2,300 stores). Iowa has a structure M4 lacks. Each product is ordered across many stores, so a new product's early sales form their own population, and it usually has a comparable earlier product. This lets us compare two designs:

  • The paper's design (one fingerprint per product category): does not help (−0.1% vs. plain, 2/5 seeds), as Appendix M predicts for a vector that can only shift its category.
  • Per-item design (the same fingerprint, computed for each new product from its comparable earlier product instead of once per category): lowers cold-start error by 1.9–2.6% against the plain model in three setups (5/5 seeds, intervals exclude 0). It also edges out summary statistics in 4 of 5 setups (0.3–1.6%, not significant). A simple product label matches or beats it, since the related product usually appears in training.
  • Fingerprint source: the earlier product's first 13 weeks across stores. A store that starts carrying an existing product gets that product's first weeks elsewhere.

On public cold start, the fingerprint beats summary statistics on M4, and the per-item version beats the plain model on Iowa Liquor Sales.

AuthorsOpenReview ↗

Q1 (continued) and Q2, Unseen Stations and Forecast Origins (3/3)

Q1 (continued): Rel on Unseen Series

TopoPrimer's per-series coordinate (Rel) also shows strong cold-start gains on public data. We held 602 Weather stations out of training and validation and projected each unseen station's history onto the frozen training basis. This significantly reduces 30-day block MAE across all 5 seeds:

Rel on 602 Unseen Weather Stations (30-day block MAE, 5 seeds)

Coordinate Assigned to Unseen Station30-Day Block MAE Δ vs. Plain Model95% Bootstrap CISeeds Improved
Rel (projected onto frozen training basis)−4.0%[−5.6%, −2.6%]5/5
Shuffled Rel (permuted across unseen stations)+1.7%-0/5
Rel vs. Shuffled Rel (station-specific signal)−5.6%[−7.3%, −4.0%]5/5

Takeaway: on stations the model never saw, Rel lowers error by 4.0% in every seed, while the same vectors shuffled across stations make it worse (+1.7%). The gain comes from each station's own coordinate, not from giving it just any vector.

Q2: Rel Across Forecast Origins

Q2. "Do the relational coordinate gains persist across multiple forecast origins?"

Yes for the ECL adapters and the Weather from-scratch model: Rel beats the plain model in 7–8 of 8 test months on ECL and at all 4 start dates on Weather. The Weather adapters gain only at shorter context, and on METR-LA the gain is clear only on top of Chronos-2's joint mode. First, we clarify an exposition error in our original text: our ECL foundation-model adapter evaluation already evaluated all 5,261 rolling hourly forecast origins across the entire 8-month test split (321 clients × 5,261 origins). Second, we ran new multi-origin evaluations on Weather (4 start dates spaced 30 days apart across the full 30-day forecast block) and METR-LA (572 rolling hourly origins):

Rel Across Forecast Origins (5 seeds per origin)

Benchmark & Model ArchitectureForecast Origins EvaluatedRel vs. Plain Baseline Across OriginsConsistency Across Time / Seeds
ECL Adapters (Chronos-2 & TimesFM-2.5)All 5,261 rolling hourly origins (Nov 2018 – Jun 2019, split by month)−4.0% [−6.0, −2.0] (Chronos-2) and −3.7% [−6.0, −1.2] (TimesFM-2.5) at default k=256; −7.8% / −7.7% at k=26Better in 7/8 months (MSE) and 8/8 months (MAE); 5/5 seeds
Weather (From-Scratch Transformer)4 start dates spaced 30 days apart (full 30-day block MAE)−3.9% overall (−3.2%, −4.6%, −5.1%, −2.7% at the 4 dates)95% CI excludes 0 at all 4 dates; wins in 19/20 date × seed pairs
Weather Adapters (Chronos-2 & TimesFM-2.5)4 start dates spaced 30 days apart (full 30-day block MAE)No gain at 300-day context (+0.8% / +0.4%); −14% gain at 90-day context (one start date)At 90-day context, wins in 5/5 seeds (see our reply to Reviewer S1Lq, Q3)
METR-LA Traffic (Chronos-2 Adapter)572 rolling hourly origins (15-min MAE)Univariate: −1.6% (p = 0.095, n.s.); On top of Multivariate Joint Mode: −1.5%Wins in 5/5 seeds when paired with joint mode (see our reply to Reviewer a6vu, Q2)

Takeaway: the gains are not an artifact of one forecast origin. On ECL and on the Weather from-scratch model, Rel wins across months, start dates and seeds.

Thank you again. We welcome any further questions.

Reviewer_S1LqOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes TopoPrimer. The idea is to compute two additional features from the training data. 1) Spectral relational coordinates, which is the i-th row of SVD matrix U. 2) Population TDA fingerprint, which describle for a connected component of squeneces, what are their birth-dead patterns when changing threshold of corr. These features are added to a trainable adapter of a frozen time series foundation model. the method is tested on public and internal dataset.

Critical strengths
  1. the proposed features can be calculated offline, which makes it simple to use.
  2. The paper includes useful comparisons with correlation features, series topology, and different adaptation methods.
  3. Some datasets get big gains with using the features.
Critical weaknesses
  1. the author should substainlly rewrite their methodlogy part, with clear motivation, definition of key concepts and reserve key equations in the main text.

  2. The paper's writing and presentation is bad. The key methodology of these two features are written in two very intensive paragraphs, with many important details on the appendix. this makes the paper hard to read and follow.

  3. It is unclear what the main contribution is beyond adding population features to an adapter. Using simple corrrow features already gives close performance to Rel. Also, Section 4.1 acknowledges that most of the improvment comes from the adapter itself. Hence, I am not convinced that the proposed features are justified.

  4. It lacks motivations for the population fingerprint. It basically gives different group a context vector. What is the difference compared with a group-specific embedding? The cold start experiment show positive results on private data, but not an overall gain.

  5. The experiment need more checking. For the public results, only 1 forecast origin per series is used, so it is unclear whether the gains hold at different times. More comprehensive testing is needed.

Questions for authors
  1. Could the authors compare the TDA fingerprint with a learned cohort embedding and simple cohort statistics on the same internal task? This would help distinguish the value of topology from simply providing group identity.

  2. Could Table 1 and the corresponding ECL controls be rerun without the non-negativity clipping? Does the reported 7.4% MSE improvement remain under this evaluation?

  3. Do the main improvements hold over several forecast origins? For Weather, please also compare the adapter variants using the full 30-day forecast block.

  4. The screening statistic was selected from 16 candidates using the same 19 internal cohorts on which its association with forecasting gains was measured. Could this criterion be checked on held-out cohorts? More broadly, can the cold-start result be supported by additional seeds and an independent evaluation where population structure actually varies?

6 comments
AuthorsOpenReview ↗

W1–W3, Section 3 and Adapter Controls (1/5)

Thank you for the careful review.

W1 and W2: Rewriting Section 3

W1 & W2. "The author should substantially rewrite their methodology part... The paper's writing and presentation is bad."

We agree, and we will rewrite Section 3 with a clear motivation and definition for each descriptor and the key equations in the main text, moving implementation details to the appendix.

W3: Is It Just the Adapter?

W3. "It is unclear what the main contribution is beyond adding population features to an adapter. Using simple CorrRow features already gives close performance to Rel. ... most of the improvement comes from the adapter itself."

It isn't just the adapter. CorrRow is close to Rel only on series seen in training. On unseen series and a growing population, CorrRow degrades, while Rel holds up far better. We tested whether Rel merely compensates for a frozen backbone or acts as extra adapter capacity. We held the adapter architecture fixed on unclamped ECL (k=26, 5 seeds). We then varied both the underlying backbone adaptation (frozen, LoRA rank 2, or full end-to-end fine-tuning) and the conditioning input. Rel's gain survives every control: it stays near 7% on fine-tuned backbones, a same-size plain adapter gains only about 1%, and Rel beats both a shuffled Rel and a learned client ID:

Controls for Rel on Unclamped ECL (Δ MSE vs. Plain Adapter on the same backbone, 5 seeds)

Conditioning Input / ControlChronos-2 (Frozen)Chronos-2 (+ LoRA r=2)Chronos-2 (+ Full FT)TimesFM-2.5 (Frozen)
+ Rel (26 dims, 8% of N)−7.8%−7.4%−7.1%−7.7%
+ Shuffled Rel (permuted across clients)−6.0%−5.7%−5.5%−5.5%
+ Parameter-Matched Widened Plain Adapter−1.1%−0.7%−0.7%Not run
Rel vs. Shuffled Rel (client-specific gain)−2.0% [−3.0, −0.9]−1.8% [−2.7, −0.8]−1.8% [−3.4, −0.7]−2.4% [−4.3, −0.8]
Rel vs. Learned 26-dim Client Embedding−3.0% (5/5 seeds)Not runNot run−2.6% (5/5 seeds)
  1. Not a substitute for backbone fine-tuning: Rel still reduces MSE by −7.4% (LoRA) and −7.1% (full FT) across 5/5 seeds (with intervals: Table 4 in Post 2 of the General Response), consistent with fine-tuning still forecasting each series on its own. Moreover, as shown in Section 4.1, frozen Chronos-2 + Rel outperforms full fine-tuning while training only 0.64% of the parameters.
  2. Not extra adapter capacity: Widening the plain adapter to match Rel's exact parameter count yields only −0.7% to −1.1%.
  3. Rel beats arbitrary IDs and ties CorrRow on seen series: Distinct per-client vectors provide baseline conditioning on seen series. Rel beats shuffled Rel by 1.8%–2.4% across every backbone (all 95% CIs exclude zero). It beats a learned 26-dim embedding in 5/5 seeds on both models (Table 4 in Post 2 of the General Response). Against CorrRow we claim parity: Rel is ahead at k=26 (−7.8% vs. −6.8%; see Table 1 in Post 1 of the General Response), but CorrRow is ahead at the rank validation picks.
  4. Generalizes to unseen series where learned IDs and CorrRow fail: On 602 held-out Weather stations never seen in training, projecting onto the frozen Rel basis reduces 30-day block MAE by −4.0% [−5.6%, −2.6%] (5/5 seeds). Shuffled Rel instead worsens error by +1.7% (Table 5 in Post 2 of the General Response). On unseen ECL clients, CorrRow worsens error by +8.0%. For the full held-out table across history lengths, please see Post 1 of our reply to Reviewer a6vu.
AuthorsOpenReview ↗

W4 and Q1, Fingerprint vs. Learned Cohort Embedding (2/5)

W4 and Q1: Fingerprint vs. a Learned Cohort Embedding

W4. "It lacks motivations for the population fingerprint... What is the difference compared with a group-specific embedding? The cold start experiment show positive results on private data, but not an overall gain."

Q1. "Could the authors compare the TDA fingerprint with a learned cohort embedding and simple cohort statistics on the same internal task?"

We agree with your geometric diagnosis of the cohort-shared setting. When a single vector is shared across all series in a static group, its capacity on cold start is, in principle, upper-bounded by a learned group embedding (as we mentioned in Appendix M, which we will move to Section 3). On public M5 (30 cohorts, 5 seeds), a learned group embedding, a random ID, and shuffled fingerprints all match the cohort fingerprint within 0.2%. The bound is on what the model could fit in principle. A learned embedding still has to be fit jointly with the model, and it can be less stable: on public M4 (1996 cut-off), where every model already learns a domain embedding, the fingerprint still lowers cold-start error by 15.5% and beats summary statistics by 2.6% (5/5 seeds; see Table 7 in Post 3 of the General Response).

To answer Q1 directly on our internal corpus, we first tested a model that already includes a learned cohort embedding (3 seeds) and added either the cohort-level TDA fingerprint or dimension-matched cohort summary statistics:

Adding a Cohort Fingerprint to a Model That Already Has a Cohort Embedding (Internal corpus, MAE across 3 seeds)

Series RegimeLearned Cohort Embedding Only+ Cohort TDA Fingerprint+ Cohort Summary Statistics
Mature Series (abundant history)1.4571.331 (−8.6%, 3/3 seeds)1.357 (−6.9%)
Cold-Start Series (cohort-shared vector)1.5391.539 (0.0%)1.551 (+0.8%)

Shared across a cohort, the fingerprint ties the learned embedding on cold start, as you expected. So we computed the same fingerprint at a finer grain, one per new item instead of one per cohort, where an ID cannot stand in for it.

Where the TDA fingerprint provides non-ID signal is at the per-item cold-start level.

  • Setup: When a new item appears across many entities, we pair it with a comparable prior reference item and compute the 125-dim TDA fingerprint of the reference item's first 13 weeks.
  • Why an ID can't do this: Because the target item is a different item from the reference item, a group or reference ID has no shared identity to transfer.
  • Result: On the paper's cold-start evaluation (10 seeds), the per-item fingerprint (computed for each new item, not shared across its cohort) lowers launch-week error by 4.8% against the plain model (9/10 seeds), and it beats every statistical, PCA, and ID control built from the exact same reference data in 10/10 seeds:

Per-Item Cold-Start Benchmark (Internal corpus, weeks 0-2 MAE, 10 seeds; for the full table, please see Table 8 in Post 4 of the General Response)

Representation Feeding the 125 Input Slots (from Reference Item)MAE Δ vs. TDA FingerprintSeeds Where TDA Wins95% Bootstrap CI of MAE Gap
Reference Item Fingerprint (Persistent Homology, TDA)Baseline (1.703)--
Plain Model (no reference-item vector; identical backbone)+2.9%9/10[−0.055, +0.125]
Swapped Fingerprint (mismatched reference item at test time; pooled over cohorts)+4.6%10/10-
Reference Item Mean Early Curve+5.2%10/10[+0.015, +0.164]
PCA / Scale-Normalized / Concatenated Summary Statistics Variants+7.5% to +8.6%10/10 eachAll CIs strictly > 0
125 Summary Statistics of the Same Reference Data+9.1%10/10[+0.086, +0.234]
Fixed Random Embedding per Reference Item (ID control)+15.5%10/10[+0.096, +0.374]

This gain is also robust to TDA hyperparameters (varying neighborhood size and landscape resolution retains 53–97% of the gain across 5 seeds; see Post 4 of the General Response) and requires zero neural training.

AuthorsOpenReview ↗

W5 and Q3, Multiple Forecast Origins (3/5)

W5 and Q3: Multiple Forecast Origins

W5. "For the public results, only 1 forecast origin per series is used, so it is unclear whether the gains hold at different times."

Q3. "Do the main improvements hold over several forecast origins? For Weather, please also compare the adapter variants using the full 30-day forecast block."

Across origins, the gains hold for the ECL adapters and the Weather from-scratch model. They do not hold for the Weather adapters on the full 30-day block at 300-day context, or for univariate METR-LA. For ECL, the paper's text understated the evaluation: the adapters were already scored on all 5,261 rolling hourly forecast origins in the test split (321 clients × 5,261 origins, Nov 2018 to Jun 2019). Thank you for catching this. We will make it explicit in the text. We also ran new multi-origin evaluations on Weather and METR-LA:

Rel Across Forecast Origins (5 seeds per origin)

Benchmark & Model ArchitectureForecast Origins EvaluatedRel vs. Plain Baseline Across Time
ECL (Chronos-2 & TimesFM-2.5 Adapters)All 5,261 rolling hourly origins (8 test months)Wins in 7/8 months (MSE) and 8/8 months (MAE); overall −4.0% [−6.0, −2.0] (Chronos-2) and −3.7% [−6.0, −1.2] (TimesFM-2.5)
Weather (From-Scratch Transformer)4 distinct start dates spaced 30 days apart (full 30-day block)−3.9% overall; wins at all 4 start dates (−3.2%, −4.6%, −5.1%, −2.7%; CI excludes 0 at each) and in 19/20 date-by-seed pairs
Weather (Chronos-2 / TimesFM-2.5 Adapters)4 distinct start dates spaced 30 days apart (full 30-day block)At 300-day context, slightly worse: +0.8% (3/20 pairs) / +0.4% (6/20 pairs), CIs exclude 0. At 90-day context, −14% (one start date; diagnostics below)
METR-LA Traffic (Chronos-2 Adapter)572 rolling hourly originsUnivariate adapter gain not statistically significant overall (−1.6%, p = 0.095); better in 5/5 seeds in joint mode (−1.5%; see our reply to Reviewer a6vu, Q2)

We therefore restrict our Weather claim at the 300-day context to the from-scratch model. To understand why Rel helps the from-scratch model at 300 days while the foundation adapter only benefits at shorter contexts, we ran four diagnostic ablations:

Diagnostic Decomposition of Rel's Signal on Weather

Diagnostic ExperimentEmpirical ResultMechanistic Interpretation
1. From-scratch model given only a 4-way variable-type label (rain / min temp / max temp / solar) on seen stationsRecovers ≈ 80% of Rel's gainOn seen stations with 300d context, most of Rel's value comes from separating the 4 meteorological variable types.
2. Chronos-2 Adapter + Rel with 300-day contextNo gain (+0.8% block MAE)With 300 days of history, a pretrained foundation backbone likely already infers the variable type from the raw trajectory.
3. Chronos-2 Adapter + Rel with 90-day context−14% MAE (5/5 seeds)When historical context is shortened to 90 days, the backbone has less history to infer the variable type from, and Rel helps (our reading).
4. 602 Unseen Weather Stations (From-Scratch, 30-day block): Rel vs. Variable-Type Label vs. Shuffled RelRel: −4.0% (5/5) vs. Variable Label: −1.1% (n.s.) vs. Shuffled: +1.7%For unseen stations, a coarse variable-type label is insufficient (−1.1%); the station's own Rel coordinate helps (−4.0%).

Takeaway (our reading): On seen stations, most of Rel's gain in the from-scratch model is variable-type information. A pretrained backbone can read that from 300 days of history, which would explain why the adapters gain only when context is short. On unseen stations, Rel carries more than the variable type (−4.0% vs. −1.1% for the label alone).

Reviewer_a6vuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TopoPrimer, a framework for incorporating population-level structural information into time series forecasting models. The authors argue that existing forecasting models often rely on individual series histories and may overlook useful relationships across a population of series. TopoPrimer introduces two precomputed descriptors: an SVD-based relational coordinate representing each series' position within the population, and a persistent-homology fingerprint describing the population's global structure. These features are incorporated into Transformers trained from scratch or used through a lightweight residual adapter for frozen time series foundation models. Experiments on four public datasets show that relational coordinates can improve forecasting accuracy, particularly for frozen backbones. The topological fingerprint provides limited benefits on public benchmarks but improves cold-start and peak-demand forecasting on an internal dataset.

Critical strengths
  1. An interesting research perspective. The paper explores whether explicit population-level information can provide useful context beyond individual time series histories. I think this is a meaningful direction, particularly for pretrained forecasting models that process series independently. The distinction between per-series relational coordinates and population-level structural fingerprints is also interesting.

  2. Extensive experiments and careful discussion of limitations. The authors provide a fairly thorough evaluation, including comparisons with LoRA, full fine-tuning, correlation-based features, and per-series topological descriptors. I also appreciate that the paper reports negative results and discusses the conditions under which the proposed descriptors are useful. The analysis is more careful than a simple demonstration of forecasting improvements.

Critical weaknesses
  1. The unique contribution of the proposed topological representations is not sufficiently established. Most of the consistent improvements on public benchmarks come from the SVD-based relational coordinates, which are closely related to conventional low-rank representations and cross-series correlation modeling. Appendix E also shows that simple correlation features achieve very similar results. More importantly, the persistent-homology fingerprint provides little consistent benefit on public datasets. Since this fingerprint is shared by all series within a cohort, I wonder whether its contribution mainly comes from providing cohort-level information rather than capturing something uniquely topological.

  2. The main evidence supporting population-level persistent homology is still limited. The most encouraging results for the topological fingerprint come from cold-start and peak-demand forecasting on a proprietary dataset. These experiments use a single training seed, and the dataset is not available for independent reproduction. Meanwhile, the public M5 experiments show no comparable overall benefit and even some performance degradation for new series.I appreciate that the authors acknowledge these limitations and restrict their claims to structurally diverse populations. However, the current evidence does not fully establish how general or reliable the proposed fingerprint is. This is particularly important because population-level persistent homology appears to be the most distinctive contribution of the paper.

Questions for authors
  1. What unique predictive information does persistent homology provide? Since the fingerprint is constant within each cohort, could the authors compare it with simpler alternatives, such as learned cohort embeddings or basic group-level statistics? This would help clarify whether the improvements really come from topological information.

  2. How much does the SVD-based representation improve upon conventional correlation features? Appendix E shows that CorrRow performs quite similarly to the proposed relational coordinates. Could the authors further explain the specific benefits of the spectral representation beyond dimensionality reduction? A clearer comparison with recent correlation-aware TSFM adaptation methods would also help position the contribution.

  3. Can the positive cold-start results be validated more broadly? The internal dataset shows encouraging results, while M5 does not reproduce the same gains. Is there additional evidence that differences in cohort topology explain this discrepancy? Could simpler cohort features provide similar cold-start benefits?

  4. How stable are the precomputed structural descriptors over time? Both descriptors are computed once from historical data and then frozen. If the relationships among series change over time, would the descriptors need to be updated? How would such updates affect the trained adapter?

8 comments
AuthorsOpenReview ↗

W1 Part 1 and Q2, Rel vs. CorrRow (1/6)

Thank you for the thoughtful review.

W1 (Part 1) and Q2: Rel vs. CorrRow

W1 (Part 1) and Q2 "How much does the SVD-based representation improve upon conventional correlation features? ... CorrRow performs quite similarly... Could the authors further explain the specific benefits ... beyond dimensionality reduction?"

Rel has two benefits beyond dimensionality reduction. It carries over to series the model has never seen, where CorrRow breaks: on held-out ECL clients (full history), Rel lowers error by 4.0% and CorrRow raises it by 8.0% (5/5 seeds). And on seen series, Rel beats CorrRow at 10× lower dimension when its rank is set by a fixed rule (−7.8% vs. −6.8%, 5/5 seeds). Low-rank CorrRow also degrades on new series, so the transfer comes from projecting onto a fixed basis, not from low rank alone.

1. Seen Series: Why Appendix E Showed a Near-Tie

You noted that CorrRow performs similarly to Rel in Appendix E. That near-tie came from the paper's default rank k=256. On ECL (N=321 clients) that spans 80% of the population basis, so each Rel vector is close to unique and acts like a series ID rather than a position in shared structure. Fixing the rank a priori to 8% of N (Weather's 256 / 3,010 ratio, giving k=26) makes Rel the best of the three, at 10× lower dimension than CorrRow:

Seen-Series Comparison on Unclamped ECL (Δ MSE vs. Plain Adapter, 5 seeds)

Per-Series RepresentationDimChronos-2 (Frozen)TimesFM-2.5 (Frozen)
Rel at paper default rank (k=256, 80% of N)256−4.0% (5/5 seeds)−3.6% (5/5 seeds)
Raw Correlation Row (CorrRow; 320-entry row, randomly projected to 256)256−6.8% (5/5 seeds)−6.1% (5/5 seeds)
Rel scaled to population (k=26, 8% of N)26−7.8% (5/5 seeds)−7.7% (5/5 seeds)

Shrinking CorrRow doesn't close the gap. Rebuilt at the same rank, it stays at −6.8% from k=16 to 256, and Rel wins by 0.8–0.9% at k=16 and 32 (5/5 seeds; for the full table, please see Table 2 in Post 1 of the General Response). At the larger rank validation picks (k=128), CorrRow is 1.7% better, so this seen-series edge depends on keeping the rank small.

2. Unseen Series and a Growing Population

Why Rel transfers and CorrRow does not:

  • Rel: SVD of the row-normalized series matrix (Section 3, Eq. 1), giving a fixed basis over the whole population. Projecting a new series onto it leaves existing coordinates unchanged.
  • CorrRow: a row of the N × N correlation matrix. Its size is tied to N, and a new series must be correlated against all N.

On series the model never saw, Rel beats CorrRow at every history length on both datasets. With 300 days of history or more, it also beats the plain adapter in 5/5 seeds, while raw CorrRow makes error worse at every shorter history, and on ECL even with full history. We held out every series once via 5-fold cross-validation (5 seeds per fold), computing each feature from different amounts of history:

Held-Out Unseen Series (5 Folds × 5 Seeds) (Δ Error vs. Plain Adapter; in parentheses, seeds better than plain out of 5)

Dataset (Metric)History Used to Compute FeatureRel (Projected)Raw CorrRowLow-Rank CorrRow
Weather (3,010 stations, Day-30 MAE)30 days (short history)+1.5% (0/5)+3.7% (0/5)+1.7% (1/5)
90 days (1 season)−0.1% (n.s.)+3.7% (0/5)+2.1% (0/5)
300 days (forecast context)−1.3% (5/5)+0.9% (1/5)+1.4% (1/5)
Full training window−1.9% (5/5)−1.6% (5/5)−1.5% (5/5)
ECL (321 clients, 96h MSE)Full training window−4.0% (5/5)+8.0% (0/5)Not run

Rel also handles a growing population. When we added new stations to Weather, recomputed CorrRow degraded a trained adapter by +3.7% on stations it already knew, while a Procrustes-aligned Rel basis cost ≈ 0%.

AuthorsOpenReview ↗

Q2 Part 2, Rel on Chronos-2 Joint Mode (2/6)

Q2 (Part 2): Rel on Top of Chronos-2's Joint Mode

Q2 (continued). "A clearer comparison with recent correlation-aware TSFM adaptation methods would also help position the contribution."

Rel still adds on top of Chronos-2's correlation-aware joint mode on all three datasets. To compare against recent correlation-aware foundation model mechanisms, we evaluated Chronos-2's native multivariate joint prediction mode (which applies cross-series attention across grouped time series). We grouped series either by maximum empirical cross-correlation or by default ID order (control), and trained both the Plain Adapter and the +Rel Adapter (k=256, 5 seeds) on top of joint-mode Chronos-2:

Rel Combined with Correlation-Aware Multivariate Chronos-2 (Joint Mode) (5 seeds)

Benchmark (Metric)Joint Mode Alone vs. Univariate Chronos-2 (Correlation Groups / ID Groups)Adding +Rel on Top of Joint Mode (Correlation Groups / ID Groups)
ECL (96h Unclamped MSE)−1.1% / −0.5%−3.3% / −3.9%
METR-LA Traffic (15-min MAE across 572 hourly origins)−1.6% / −1.7%−1.0% (5/5 seeds) / −1.5% (5/5 seeds)
Weather (Day-30 MAE)+4.1% / +0.6% (degrades)−3.4% (5/5 seeds) / −1.0% (4/5 seeds)

Rel provides complementary gains on top of joint multivariate attention across all three benchmarks. The two work at different scales: joint mode attends within small groups of series over the forecast context, while Rel places each series in the whole population. They help at different times of day, so their gains add up rather than overlap. On METR-LA (572 forecast origins), Rel adds most during rush hours (−2.1% to −2.2% on top of joint mode), while joint mode's own gain is largest overnight (−2.5%):

Diurnal Complementarity on METR-LA (ID-Order Groups, 572 Forecast Origins)

Time-of-Day RegimeJoint Multivariate Mode vs. Univariate+Rel Adapter on Top of Joint Mode
Morning & Evening Rush Hours (Peak)−1.3%−2.2% (Morning) / −2.1% (Evening)
Overnight (Off-Peak)−2.5%−0.9%
AuthorsOpenReview ↗

W1 Part 2 and Q1, Fingerprint vs. Cohort ID (3/6)

W1 (Part 2) and Q1: Is the Fingerprint Just Cohort Information?

W1 (Part 2) and Q1 "What unique predictive information does persistent homology provide? ... could the authors compare it with simpler alternatives, such as learned cohort embeddings or basic group-level statistics?"

You are right: shared across a whole cohort, the fingerprint mostly acts as cohort information. Computed per new item, it carries information that IDs and summary statistics don't (shown on our internal corpus).

  • Why a shared fingerprint mostly acts as an ID: As formalized in Appendix M (which we are promoting to Section 3), a static vector shared across a cohort can at most apply a cohort-wide shift, so in principle a learned cohort embedding can do the same. In practice the embedding has to be learned jointly with the model, and a frozen descriptor can be the more stable signal (row 2 below).
  • Public data: Most public benchmarks offer one population or a few fixed groups, so the fingerprint has little to vary over and matches an ID (row 1 below). On M4, where thousands of series enter over time, it beats both a learned domain ID and summary statistics on cold start at the 1996 cut-off (row 2; see our answer to W2, Part 2, in Post 5).
  • Why per item is different: On our internal corpus, each new item gets the fingerprint of a comparable prior reference item. An ID cannot transfer here, because the target item is distinct from the reference item.

The table compares the fingerprint with learned or random IDs and group summary statistics in all four settings:

Summary of TDA Fingerprint vs. Identity and Statistical Controls

Experimental RegimeTDA vs. Learned / Random IDTDA vs. Group Summary StatisticsMechanistic Conclusion
1. Public M5 Cohort Cold Start (30 cohorts, 5 seeds)Matches learned & random ID (± 0.2%)Matches summary statistics (± 0.2%)When shared across a static cohort, the vector acts as a cohort ID.
2. Public M4 Cold Start (6 domains, 1996 cut-off, 5 seeds)Beats the M4 learned domain ID by 15.5% (5/5)Beats summary statistics by 2.6% (5/5)A frozen descriptor is a more stable signal than a jointly learned domain ID (the ID-only model varies 5–6× more across seeds).
3. Internal Cohort Model with Learned Cohort ID Already Present (3 seeds)Ties on cold start; reduces mature-series MAE by −8.6%Statistics reduce mature-series MAE by −6.9%Acts primarily as a cohort-level structural prior.
4. Internal Per-Item Cold Start (10 seeds; see our answer to W2 in Post 4)Random reference ID is 15.5% worse (10/10 seeds)125 summary stats are 9.1% worse; PCA/normalized stats 7.5–8.6% worse (10/10 seeds)Carries item-specific information that summary statistics and IDs don't.

Persistent homology also has a use that needs no model training: its statistics can screen which cohorts will benefit before any forecasting model is fit (full five-check audit in our reply to Reviewer S1Lq, Q4). Within our internal corpus, the screen holds up out of sample and carries information plain statistics do not. It survives correction for picking the best of 16 candidate summaries, predicts gains on held-out cohorts, and still predicts them after controlling for 13 plain cohort statistics:

Out-of-Sample Validation of the Pre-Training Topological Screening Statistic

Validation Test on Internal Cohorts (N=19)Statistical Result
Multiple-hypothesis correction across all 16 candidate topological summariesp = 0.0004
Split-sample generalization (criterion selected on 10 cohorts, evaluated on 9 held-out cohorts)Median held-out Spearman ρ = −0.69
Partial correlation with forecasting gain after controlling for 13 plain cohort statisticsPartial Spearman ρ = −0.64 (p = 0.002)

Comments on the paper

6 comments
AuthorsOpenReview ↗

Summary of Results, and ECL Without Clipping (1/4)

We thank the Area Chairs and all three reviewers for their feedback.

Throughout our responses, W and Q mark a reviewer's Critical Weaknesses and Questions For Authors, numbered in order.

Two findings run through our responses. First, the per-series coordinate (Rel) drives most of the accuracy gain, and it holds up under every new control, including fine-tuned backbones, learned IDs and unseen series. Second, the fingerprint helps where each new series has a population of its own, in cold start, which we now show with multiple seeds on both internal and public data.

Summary of results in the revised paper:

  • ECL: reported unclamped, Rel lowers MSE by 4.0% at k=256 and 7.8% at k=26 (5/5 seeds).
  • Internal cold start: the single-seed cohort-level claims are withdrawn and replaced by 10-seed per-item results. The fingerprint lowers launch-week error by 4.8% against the plain model (9/10 seeds), and it outperforms every simpler encoding of the same data in 10/10 seeds (summary statistics by 9.1%).
  • Public cold start: on M4, the fingerprint beats summary statistics by 2.6% and a learned domain ID by 15.5% (5/5 seeds). On Iowa Liquor Sales, the per-item fingerprint beats the plain model by 1.9–2.6% (5/5 seeds).
  • New controls: Rel's gain holds on LoRA and fully fine-tuned backbones (−7.4%, −7.1%). It beats a same-size widened adapter, shuffled Rel and a learned ID, carries over to 602 unseen stations (−4.0%, 5/5 seeds), and holds across forecast origins (ECL in 7–8 of 8 months, Weather at all 4 start dates).

ECL Without Clipping, and the Size of Rel

  • What was asked: clipping on ECL and Rel vs. CorrRow and cohort-level controls
  • What we found: Without clipping, Rel still lowers MSE by 4.0% at the paper's rank (k=256) and by 7.8% at k=26, in 5/5 seeds (Table 1). Cohort-level features (the TDA fingerprint, covariance eigenvalues) stay within 0.1% of plain (per-series vs. per-cohort table is in our reply to Reviewer diAS, W1).
  • Scope: Zero-clipping affected only these ECL adapter rows, and no other benchmark in the paper's Table 1.

We also report Rel at k=26 (8% of ECL's 321 clients, Weather's rank-to-population ratio, fixed before any ECL forecast), since at the paper's k=256 the vectors span 80% of clients and act more like IDs.

Table 1. Unclamped ECL (321 clients, Chronos-2 adapter, 96h horizon, 5 seeds)

Per-Series FeatureSizeTest MSEΔ vs. Plain AdapterSeed Consistency
None (Plain Adapter baseline)-0.1730--
Rel (Paper default)2560.1660−4.0%5/5 seeds
Raw Correlation Row (CorrRow control)2560.1612−6.8%5/5 seeds
Rel (Sized to 8% of N)260.1595−7.8%5/5 seeds

At k=26, Rel beats full CorrRow by −1.0% [−1.5, −0.6] on Chronos-2 and −1.7% [−2.5, −1.0] on TimesFM-2.5 (intervals resample whole weeks).

Is this a fair comparison with CorrRow? Yes. We rebuilt CorrRow at the same rank as Rel and retrained both (same run, 5 seeds). CorrRow stays at −6.8% to −6.9% at every rank we tried (16 to 256), so sizing doesn't help it. At the same small rank, Rel wins in 5/5 seeds:

Table 2. Rel vs. CorrRow at Matched Rank (Unclamped ECL, MSE Δ vs. plain adapter, 5 seeds. Shown: k=16 and k=32, the tested ranks on either side of 26.)

Comparisonk=16k=32
Rel−7.5%−7.7%
CorrRow at the same rank−6.8%−6.8%
Rel vs. CorrRow−0.8% (5/5 seeds)−0.9% (5/5 seeds)

Why k=256 underperforms on ECL (N=321 clients):

  • At k=256, Rel acts like an ID. Each client's vector is nearly unique: shuffling it costs nothing at k=256 but about 2% at k=26 (see Table 4 in Post 2).
  • The loss comes from clients that drift. About 80% of the 256→26 gain comes from the fifth of clients whose usage level shifted most after training.
  • 26 was fixed in advance, not tuned. Picking k by validation instead gives k=128, which still beats 256 (−5.3% vs. −4.0%, Table 3). At that rank, CorrRow is 1.7% better than Rel.

The revision reports both k=256 and k=26 and recommends keeping k well below N.

Table 3. Rel at Different Ranks on Unclamped ECL (5 seeds)

Spectral Rank kSelection CriterionShare of Population (k/N)Chronos-2 (Δ MSE vs. Plain)TimesFM-2.5 (Δ MSE vs. Plain)
256Paper default79.8%−4.0%−3.6%
128Validation-loss minimum39.9%−5.3%−4.8%
26Fixed a priori at 8% of N (Weather ratio)8.1%−7.8%−7.7%

Takeaway: On ECL, a smaller Rel beats 256 on both backbones. The rule-set k=26 roughly doubles the gain.

AuthorsOpenReview ↗

Controls for Rel, and Unseen Series (2/4)

Controls for Rel

Reviewers asked whether Rel's improvements come primarily from (a) adding a trainable adapter to a frozen backbone, (b) extra parameter capacity, or (c) giving each series an arbitrary distinct identifier. We tested all three directly on unclamped ECL. None of the three explains the gain. Rel helps as much on fine-tuned backbones. A same-size plain adapter gains far less. Rel beats shuffled Rel and a learned ID.

Table 4. Controls for Rel on ECL (Unclamped MSE Δ, k=26, 5 seeds, 95% CI in brackets, resampling seeds and clients)

Controlled ComparisonChronos-2 (Frozen)Chronos-2 (+ LoRA r=2)Chronos-2 (+ Full FT)TimesFM-2.5 (Frozen)
Rel vs. plain adapter−7.8% [−10.0, −5.3]−7.4% [−9.5, −4.8]−7.1% [−9.4, −4.5]−7.7% [−10.0, −5.1]
Widened plain adapter (same parameter count)−1.1%−0.7%−0.7%Not run
Rel vs. shuffled Rel (across clients)−2.0% [−3.0, −0.9]−1.8% [−2.7, −0.8]−1.8% [−3.4, −0.7]−2.4% [−4.3, −0.8]
Rel vs. learned 26-dim client embedding−3.0% [−8.1, −0.3]Not runNot run−2.6% [−7.9, +0.3]

Key Takeaways from Table 4:

  1. Gains are not an artifact of freezing the backbone: Rel yields nearly identical improvements when Chronos-2 is adapted via LoRA (−7.4%) or fully fine-tuned end-to-end (−7.1%), consistent with fine-tuning still forecasting each series on its own.
  2. Gains are not due to adapter capacity: Widening the plain adapter to match Rel's exact parameter count yields at most −0.7% to −1.1%.
  3. Rel's content matters, not just a unique ID: Giving each client another client's Rel vector (shuffled) keeps a unique ID per client and already gives most of the gain. Rel does better by 1.8% to 2.4% on every backbone (all 95% CIs exclude zero). It also beats a learned 26-dim client embedding (−3.0% Chronos-2, −2.6% TimesFM-2.5) in 5/5 seeds on both. The MAE intervals exclude zero on both models. On MSE, a few high-variance clients widen the TimesFM-2.5 upper bound to +0.3%.

Rel on Unseen Series and Short Histories

Reviewers asked whether Rel works for unseen series, it does. We held out 602 of Weather's 3,010 stations, leaving enough to build a stable basis. Rel reduces their error, while a shuffled Rel makes it worse. On ECL, holding out every client once and using its full history gives the same picture (Rel −4.0%, 5/5 seeds; CorrRow +8.0%). Rel also helps more when the backbone sees less history.

  • Why Rel can handle new series: A learned client embedding has no trained weights for an unseen series, and a raw correlation row (CorrRow) has a dimension fixed to the training population. Rel instead embeds any newly arriving series by projecting its history onto the frozen k-dimensional training basis.
  • How we tested it: We trained our from-scratch Transformer on Weather while strictly holding out 602 weather stations from both training and validation, and evaluated across the full 30-day forecast block.

Table 5. Zero-Shot Transfer to Held-Out Weather Stations (n = 602 unseen stations, 30-day block MAE, 5 seeds)

Coordinate Provided at Test Time30-Day Block MAE Δ vs. Plain Model95% Bootstrap CISeeds Improved
Rel (projected onto frozen training basis)−4.0%[−5.6%, −2.6%]5/5
Shuffled Rel (permuted across unseen stations)+1.7%-0/5
Rel vs. Shuffled Rel (station-specific signal)−5.6%[−7.3%, −4.0%]5/5

On unseen stations, shuffled Rel makes error worse, while Rel beats it by 5.6% in 5/5 seeds. Rel also helps more as the context gets shorter. On ECL the gain grows steadily. On Weather it peaks at 90 days.

Table 6. Rel With Less Context (Chronos-2 Adapter) (5 seeds each; n/5 = seeds where Rel beats plain)

Context the backbone seesECL MSE Δ (96h ahead)Weather MAE Δ (day 30 ahead)
Long / Paper default (96h ECL / 300d Weather)−4.0% (5/5)−1.4% (4/5)
Medium history (48h ECL / 90d Weather)−9.2% (5/5)−14% (5/5)
Short history (24h ECL / 30d Weather)−11.8% (5/5)−1.2% (3/5)
AuthorsOpenReview ↗

The Fingerprint on Public Cold Start (M4, Iowa Liquor Sales) (3/4)

The Fingerprint on Public Cold Start

All three reviewers asked whether the fingerprint does more than a cohort ID on public data. It does on M4: on new series, the fingerprint beats a learned domain ID by 15.5% and summary statistics of the same data by 2.6%, in 5/5 seeds (Table 7). Most public benchmarks are a single population or a few fixed groups, so a shared fingerprint has little to vary over and acts like an ID, as we see on ECL, M5 and Weather (consistent with Appendix M). M4 is different: thousands of series enter over time across 6 domains. We turned M4 Monthly (about 48,000 business and economic series) into a cold-start test by holding out every series that starts after 1996. The model is the paper's Transformer with a learned domain ID in every arm, and each arm's settings were chosen on a held-out validation cohort.

Table 7. Public Cold-Start Replication: M4 Monthly, New Series Starting After 1996 (test sMAPE, 5 seeds)

ModelsMAPEFingerprint better bySeeds Where TDA Wins
+ TDA fingerprint7.19--
+ 125 summary statistics of the same data7.382.6% (interval excludes 0, resampling seeds and series)5/5
Naive last-value forecast (no training)7.676.3%5/5
Plain (learned domain ID only)8.5015.5%5/5
  • Design and stability: M4 uses the paper's original design (one fingerprint per domain), so it also shows the cohort-level fingerprint can help when the learned ID is unstable: the learned-ID model varies 5–6× more across seeds than the fingerprint model.
  • Other cut-offs: At the 2001 cut-off the fingerprint still beats the learned domain ID but ties summary statistics; at 2010 it ties both.

A second public dataset: Iowa Liquor Sales (public state ordering data, 2012–2019, about 2,300 stores). Iowa has a structure M4 lacks. Each product is ordered across many stores, so a new product's early sales form their own population, and it usually has a comparable earlier product. This lets us compare two designs:

  • The paper's design (one fingerprint per product category): does not help (−0.1% vs. plain, 2/5 seeds), as Appendix M predicts for a vector that can only shift its category.
  • Per-item design (the same fingerprint, computed for each new product from its comparable earlier product instead of once per category): lowers cold-start error by 1.9–2.6% against the plain model in three setups (5/5 seeds, intervals exclude 0). It also edges out summary statistics in 4 of 5 setups (0.3–1.6%, not significant). A simple product label matches or beats it, since the related product usually appears in training.
  • Fingerprint source: the earlier product's first 13 weeks across stores. A store that starts carrying an existing product gets that product's first weeks elsewhere.

On public cold start, the fingerprint beats summary statistics on M4, and the per-item version beats the plain model on Iowa.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.