- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3(+1)
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies representations across time series foundation models using CKA. It compares pretrained and randomly initialized models, separates their agreement into input-reference and residual components, and investigates these components through controlled experiments. The findings depend on the readout, and greater agreement does not consistently imply better forecasting.
Critical strengths
The research question is meaningful. Compared with representation studies in foundation models for other modalities, time series representations remain a worthwhile direction to explore.
The experimental design is careful. The paper includes many small experiments and complementary controls that examine different explanations for the observed agreement.
Critical weaknesses
The new contribution needs clearer positioning. The paper lacks a clear central experiment and reads more like an accumulation of small studies. Please identify the main claim and explain which experiments provide its strongest evidence.
The connection to forecasting ability remains unclear. Please state what direct, concrete guidance these findings provide for TSFM pretraining or downstream applications.
Questions for authors
Classification is a common way to evaluate representation quality, and classification results could provide additional evidence that the representations are useful. Could you add a classification related evaluation, such as random forest on frozen features?
Tabby uses deep supervision at intermediate layers, making its representations an interesting case for the depth analysis. I would be interested in whether this supervision changes the balance between input-reference and residual agreement. I explicitly acknowledge that Tabby appeared in September 2026: whether it is included will not affect my score.
4 comments
Your question helped us connect the extra alignment to a testable interpretation: forecast training can change which input relations models share, depending on the training and evaluation distributions. The full-inventory comparison remains central; the two tests below probe this interpretation.
In the revised transfer test, we fit combinations of input kernels, excluding both the test source and model family, then freeze the combination weights. On the same held-out pretrained targets, weights fitted to pretrained models predict last-position geometry better on average than weights fitted to fresh architecture-matched initializations (Section 5.1, Table 2). The joint 95% interval for the mean gain in centered Frobenius alignment excludes zero.
In a controlled study, a small Transformer and CNN first train on one synthetic law, then continue on a version with changed seasonal and trend structure. At the final checkpoints, the mean squared distance between their similarity matrices projected onto the input reference is lower on the new distribution but higher on the original than before continuation. After continuation, we rescale each projected matrix to its norm before continuation and recompute distances; the decrease on the new distribution remains (Section 6.3; Appendix N.2).
For model selection, retain the independent forecast-loss evaluation from our first reply; an agreement-based pretraining objective still requires evidence of forecast improvement.
Thank you for the substantial revisions and additional experiments, particularly the classification evaluation, Tabby depth analysis, and controlled training study. These additions clarify the central claim and substantially address my concerns about the paper’s focus and representation evaluation. Although the implications for improving pretraining remain limited, the functional tests provide useful evidence about the limits of using representational agreement as a proxy for forecasting ability. I will update my assessment accordingly.
Thank you for taking the time to consider our revisions and additional experiments. We appreciate your thoughtful assessment and feedback.
Thank you for the careful reading and for suggesting the classification and Tabby analyses. All references below are to the revised manuscript.
The central experiment compares agreement among pretrained models from different families with agreement among fresh, architecture-matched initializations, using the same histories across the full model and dataset inventory. The input reference is a span of similarity matrices computed from the histories; the residual is the part of model agreement outside that span. With linear CKA on the final-layer features at the last observed position, the pretrained-minus-initialized increases in both components mainly reflect closer matrix alignment; norm contributions remain uncertain. Stronger cross-family sharing is not accompanied by a detected average increase in how much of each model's geometry the input reference covers. At the final layer, mean pooling instead gives a residual increment in linear CKA whose interval includes zero. Our central result is stronger sharing through alignment, without a detected average coverage gain; evidence for a residual increment depends on the readout. The revised introduction makes this comparison and its directional decomposition the main evidence; the smaller studies test its interpretation and limits (Sections 1 and 3.1–3.3, pp. 1–4; Figures 1–3, pp. 4–5).
For practical use, we recommend selecting models or ensemble weights by forecasting loss on an independent calibration window, then testing the locked choice at later forecast origins. We tested separately fitted model/ensemble selection rules with and without the residual increment as an input. Including it slightly worsened historical performance on held-out sources and left choices unchanged at separately scored origins (Section 6.2, p. 9; Appendices K.2, p. 45, and J.3, p. 42). For frozen-feature applications, include inexpensive input statistics as baselines and state the readout clearly. A general forecasting benefit from maximizing agreement during pretraining remains unverified (Section 8, p. 10).
We added the requested random forests on frozen features using a patient-disjoint PTB-XL classification split. Window statistics outperformed Chronos-Bolt and TimesFM 2.5 features by macro-AUROC on this split, supporting the recommendation to check inexpensive baselines (Section 6.2, p. 9; Appendix J.3, p. 42). The forests evaluate complete feature vectors; the contribution of the geometric residual to classification remains unmeasured. These results come from one patient split with unknown pretraining-data overlap.
The Tabby analysis describes how the input-reference and residual increments vary with depth. At the last-position linear readout, the estimated pretrained-minus-initialized residual increment is positive at the measured depths. From quarter-depth to the final layer, the input-reference increment decreases, while the residual increment's change remains uncertain. At the final layer, the mean-pooled linear residual-increment interval includes zero (Section 3.2, p. 4; Appendix M and Figure 11, p. 53).