- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a framework that constructs a "feature manual" for the hidden representations of time-series foundation models and then attempts to control their predictions by manipulating these features.
Critical strengths
- The research topic is valuable, with a clearly defined object of investigation.
- The paper provides a relatively broad and systematic analysis.
Critical weaknesses
- The specific advantages and necessity of using Sparse Autoencoders (SAEs) need to be better justified.
- The five proposed dimensions are relatively abstract and rely on manually designed rules.
Questions for authors
1. Would directly analyzing the hidden states without using SAEs achieve comparable results? Alternatively, would other self-supervised feature extraction methods yield similar performance? The authors need to clarify what makes SAEs particularly necessary or advantageous in this framework.
2. The five proposed dimensions appear rather abstract. Even with these five dimensions identified, it remains difficult for humans to understand what the time-series foundation model has actually learned. From a practical perspective, what concrete insights or benefits can ZEITGEIST provide?
3. The core interpretability metrics rely on a series of manually defined choices and evaluation rules. Consequently, the reported "interpretability" may partly reflect the preferences embedded in the evaluation criteria rather than the intrinsic structure of the model's learned representations.
4. The fact that activation samples exhibit similar patterns does not necessarily imply that the corresponding feature has only one semantic meaning. The proposed method does not seem to sufficiently account for the possibility that a single feature may encode multiple distinct semantics.
2 comments
Thank you
I will retain my scores.
Rebuttal
Dear Reviewer wAUT,
We thank you for acknowledging the importance of TSFM interpretability and the systematic analysis of Zeitgeist. We carefully address each concern and question in the following:
Specific Advantages and Necessity of SAEs (W1/Q1):
We first use supervised linear probes to verify that the selected hidden states encode recognizable temporal structure (Sec. 4.1, Tab. 2). These probes perform better on the predefined concept-recognition task. However, supervised probing requires specifying target concepts and corresponding labels in advance [1], so this initial analysis is restricted to the concept inventory supplied by the researcher.
To explore beyond this inventory, we train SAEs through self-supervised reconstruction: the hidden activations themselves provide the reconstruction targets, without requiring concept labels [2]. Our T-SAE additionally derives its contrastive signal from consecutive patches, following [3], rather than from annotated concepts. Feature interpretation takes place after dictionary learning, when Zeitgeist characterizes the learned features through their activating examples, geometric properties, and captions (Sec. 3.3). The synthetic labels used for evaluation do not supervise dictionary learning.
Hence, our choice of SAEs is thereby motivated by their combination of label-free dictionary learning, sparse activations, reconstruction, and explicit decoder directions. These properties connect feature discovery, inspection through activating examples, and intervention within one framework.
The Five Feature Properties and Interpretability of Zeitgeist (W2/Q2/Q3):
The five descriptors answer concrete questions about a learned feature: how often it fires, where in the context it fires, whether activation survives changes to the surrounding context, which shapes recur among its activating examples, and how those shapes can be summarized in language (Sec. 3.3). See Sec. 4.4 for a concrete example. Together, the five features make large feature dictionaries inspectable and help assess the limits of individual interpretations.
Zeitgeist's atlas therefore exposes both the diversity of the SAE's features activating shapes and its contextual dependence that a single caption or selected patch visualization would miss. At the model level, these descriptors reveal systematic differences across layers and architectures, such as the decreasing share of patch-local features with depth across all nine TSFMs (Sec. 4.3).
To further support Zeitgeist's utility for interpretability, we conducted a human evaluation of the captions. See our response to Reviewer kyjG and App. F. This provides evidence that the generated descriptions correspond to the displayed temporal patterns.
Semantics and Heterogeneity (Q4):
We agree that similar activating shapes do not establish a unique semantic meaning. Furthermore, we argue, that semanticity overall is not a useful concept for general time series foundation models, as the explicit notion of a domain and its context are not present. Even within one geometric cluster, similar standardized shapes can have different meanings across domains or contexts. We, therefore, propose geometricity, a measurable concentration of observed shapes.
The Zeitgeist atlas explicitly captures the inherent heterogeneity of time series data and their models. A feature is classified as mono-geometric when its largest cluster contains at least 70% of sampled activating patches. This does not imply that strictly all activations share the same shape or meaning (Sec. 3.3), and we do surface diversity where it is observed.
The example in Sec. 4.4 illustrates this directly. Feature 2735 is mono-geometric because smooth declines account for 77.4% of its sampled activating patches, but broad humps account for another 11.8%. Both patterns are retained in its caption.
References: