acceptodds
Under review as a conference paper at ICLR 2027

Information Without Geometry: The Encoding–Reliability Dissociation In Time Series Foundation Models

Abstract

Time series foundation models (TSFMs) are increasingly used as embedding backends for retrieval, anomaly detection, and clustering, all of which assume that embedding proximity reflects predictive similarity. This assumption is rarely tested directly: standard forecasting metrics say nothing about the internal geometry of a representation, and the geometric analysis long applied to vision and language encoders has not been extended to time series. We introduce a conceptual decomposition of “representation quality,” usually treated as a single notion, into three properties: Information (is task-relevant structure encoded?), Geometry (does embedding proximity predict forecast difficulty?), and Utility (does that geometry actually improve a downstream operation?). We show these properties can come apart: a representation may contain useful temporal information, and its local geometry may be meaningful, without making nearest-neighbour retrieval useful. We call this pattern the Encoding–Reliability Dissociation (ERD). A supporting theorem shows the dissociation is possible for any injective encoder, so it is not just an artifact of the specific models we happen to test. Across four architectures spanning three pretraining paradigms, four benchmarks, and matched random-weight controls, we find exactly this pattern: temporal structure is often encoded, local geometry is often real, and yet embedding-based retrieval never outperforms a trivial, model-free raw-window baseline, even at the one point where the geometric signal is strongest. A metric-learning control confirms that geometry can be engineered on demand, but utility still does not reliably follow; utility has to be earned on its own, not assumed from information or geometry. This matters for anyone building retrieval, anomaly-detection, or clustering systems on top of these representations: a diagnostic can be statistically sound and still fail to tell you whether the downstream operation will work, so it should guide further investigation rather than stand in for testing that operation directly.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Time Series Foundation Models, Representation Learning, Embedding Geometry, Time Series Retrieval, Predictive Similarity, Representation Quality, Metric Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wvtHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates whether time-series forecasting models are able to capture useful notions of similarity, i.e., are able to actually learn underlying information on the time series, beyond serving as a predictor. The authors use an information geometry approach to establish an encoding-reliability dissociation representation.

Critical strengths
  • The concept of establishing a clear connection between information, geometry, and utility is important for time-series forecasting.
  • The formalized encoding-reliability dissociation representation is a good framework for evaluating whether representation geometry is useful for downstream tasks.
Critical weaknesses
  • The authors do not explain what type of specific downstream tasks can benefit from this framework. Moreover, it is not clear whether the results hold for any type of time series/downstream tasks, or whether certain assumptions must be made.
  • There has been works on isotropy and related geometry-based approaches that showed benefits in downstream tasks for geometry representations. This seems to contradict the results of this work, and the authors must be more precise in contrasting to those works.
  • The theorems are listed with very little insights or concrete discussions, which makes them inaccessible for the readers.
  • The OOD validations are very limited in scope.
Questions for authors
  • Can you clarify whether the results are general for any time series problem?
  • How do you differ from prior results on isotropy and/or geometric manifold learning in time series?
  • How can you expand the OOD results to showcase better generalization?
Reviewer_XiztOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether frozen TSFM embeddings provide a useful similarity space for downstream retrieval. It introduces ERD, distinguishing temporal concept decodability, local density forecast error alignment, and retrieval augmented forecasting utility. A theoretical construction shows that mutual information can be preserved while embedding geometry changes. Experiments evaluate mean pooled embeddings from 4 TSFM architectures on 4 benchmarks, using random weight controls and raw window retrieval baselines, alongside ablations. The results show that decodable temporal information and positive density error alignment can coexist with retrieval performance below raw windows under Euclidean distance.

Critical strengths
  1. The paper evaluates TSFM embedding retrieval against a raw window baseline under a leakage safe protocol. The oracle reference provides a basis for interpreting retrieval performance, while the results show that mean pooled embeddings consistently underperform raw windows under Euclidean distance across the tested datasets.
  2. The paper separates concept decodability, geometric structure, and downstream retrieval performance in its evaluation. Matched random weight controls help distinguish pretraining effects from architectural effects, providing a more precise interpretation of what the probing results establish.
Critical weaknesses
  1. The empirical evidence for ERD depends on the chosen diagnostics and readout. Information, Geometry, and Utility use different comparison baselines, while PDS measures density forecast error alignment rather than retrieval relevant similarity. The retrieval experiments also rely on final layer mean pooling, leaving unclear whether the observed dissociation reflects the learned representations or the evaluation choices.
  2. The paper does not clearly establish the broader insight provided by ERD. The theoretical result concerns mutual information rather than the linear probe measure used in experiments. While the experiments demonstrate limitations of the tested embeddings for retrieval, they offer limited explanation of when or why the proposed dissociation occurs, or what the framework establishes beyond these observations.
Questions for authors
  1. Does the proposed ERD persist when the comparisons isolate pretraining effects? Can the authors report random weight RAF results and pretrained random PDS comparisons using the same forecast error targets?
  2. Does the raw window retrieval advantage persist with alternative embedding readouts and channel wise evaluation rather than channel averaged series?
  3. Does local embedding density predict retrieval error within individual model dataset settings under a consistent protocol? What supports using density forecast error alignment as a measure of geometry relevant to retrieval?
  4. What does ERD explain beyond the observed retrieval performance gaps? Can the authors identify conditions under which encoded information or geometric structure translates into retrieval utility?
Reviewer_b3TROpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether time-series foundation model embeddings contain temporal information, reflect forecast difficulty through their distances, and support useful retrieval. It finds that these properties do not consistently align, and that raw-window retrieval generally matches or outperforms the tested embeddings.

Critical strengths
  • Addresses a relevant question: whether representations learned for forecasting are also useful for finding similar time series.
  • Includes matched random-weight controls and raw-window baselines, which make the comparisons more informative.
  • Evaluates several models and includes additional distance metrics and retrieval ablations.
  • The finding that a simple raw-window baseline performs well is useful for researchers building retrieval-based forecasting systems.
Critical weaknesses
  • The motivation for the geometry analysis is not clear enough. I understand why we want similar time series to be close in the embedding space. However, the paper mainly measures whether isolated examples are harder to forecast. It needs to explain more clearly how this helps assess whether the model finds useful historical examples.
  • The main contribution is difficult to separate from the broader framing. The finding that raw-window retrieval performs well against the tested embeddings is useful. It is less clear what the additional geometry analysis teaches us beyond that comparison.
  • The experiments show that performance varies across models and datasets, but the reasons are difficult to follow. The additional comparisons would be more useful if the paper brought together what improves retrieval, what does not, and under which conditions.
  • The conclusions seem broader than the evidence supports. The results raise concerns about using these embeddings for the tested retrieval tasks, but do not necessarily show that the representations are generally poor for comparing time series. Making that distinction clearer would strengthen the paper.
  • The practical takeaway needs more development. Including a raw-window baseline is a clear recommendation, but I would also like to understand what researchers should change when their learned embeddings perform poorly and which results support those changes.
Questions for authors
  • Could you explain more clearly why the proposed geometry measure should relate to retrieval quality? A concrete example connecting neighborhood density, forecast difficulty, and useful retrieved windows would help motivate this part of the method.
  • Can you bring together the main findings from the distance-metric and training-objective ablations? It is currently difficult to see which changes improve retrieval, under what conditions, and what this tells us about the source of the poor performance.
  • It would be great to clarify the scope of the main conclusion? In particular, distinguish the evidence that the tested embeddings underperform raw-window retrieval from the evidence that density-based geometry does not predict retrieval performance.
  • Could you make the practical recommendations more specific? Based on the comparisons, explain when readers should use raw-window retrieval and whether any of the tested alternatives provide a reason to use learned embeddings?

Comments on the paper

1 comment

Hi reviewers,

Thank you for your reviewing service. The authors have provided rebuttals and clarifications. Please confirm that you have read them, state which of your concerns are or are not addressed, and whether that changes your recommendations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.