- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces CircuitTS, a framework for identifying internal sites and hidden-state differences for 3 different temporal functions: Period, Trend, and Copy. The method constructs function-specific input counterfactuals, ranks candidate sites using Intervention Responsiveness, Function Selectivity, and behavioral alignment, and evaluates shortlisted sites through restoration, reverse patching, and cross-function specificity tests. Additionally, the experiments examine cross-dataset support generalization and cross-model state transplantation across 3 time series foundation models (TSFMs) and 9 datasets, which the authors conclude is possible.
Critical strengths
Systematic causal evaluation, rather than representation analysis:
The paper combines state restoration, reverse patching, and matched vs. mismatched replacement to test the 3 functions: Period, Trend, and Copy. They demonstrate that the selected states affect the models' forecasting behavior. This is a good contribution to causal localization, even if it does not provide a complete explanation of the internal computations.
Broad transfer evaluation with relevant controls:
The authors study 3 TFSMs and 9 datasets and conduct a leave-one-out evaluation. The cross-model experiments are especially interesting as they go beyond simple steering. Here, they compare against a shuffled history and a mismatched function baseline. The matched functions achieve competitive or better results. These experiments provide evidence that intervention sites and aligned state corrections can remain useful after their discovery setting, giving the paper empirical value independently of its stronger circuit claims.
Critical weaknesses
The central "circuit discovery" claim exceeds the demonstrated mechanistic insight:
As of now, the paper supports framing input-conditioned activation differences. The presented method establishes the location and state of a function, rather than the computational dependencies that implement it. It lacks information on how different layers/components interact, which transformations are performed, and how they form the temporal behavior. In particular, Sec. 3.1 states that the selected support specifies where the function is implemented, but the restoration and reverse-patching experiments do not distinguish a site that performs the relevant transformation from one that carries information computed upstream. This affects the paper's central contribution, not only terminology. The evidence supports function-conditioned causal localization more directly than an explanation of temporal computation.
The methodological novelty is insufficiently distinguished from activation steering and function-vector work:
Furthermore, the current method closely resembles activation-steering and function-vector approaches used by many other papers [2,3,4,5,6]. Many of them, similar to the paper at hand, also support intervention layer selection. This is a substantial methodological overlap; however, comparisons or a clear distinction (experimental or methodological) to those methods is missing. The related work is not completely absent, as they already cite [1,5]. These works already connect localization of function- or concept-associated representations with activation steering: [5] uses causal component localization to extract function vectors, while [1] studies temporal-concept localization and steering in TSFMs. The claimed separation between these approaches and CircuitTS therefore needs stronger justification. Temporal counterfactuals, the localization protocol, and cross-model alignment may constitute useful advances, but the current positioning does not convincingly establish what the newly added value of the combination is, particularly what warrants calling the result a discovered circuit.
[1] Wiliński, Michał, et al. "Exploring representations and interventions in time series foundation models." ICML. 2025
[2] Sheng Liu, Haotian Ye, Lei Xing, and James Zou. "In-context vectors: making in context learning more effective and controllable through latent space steering". ICML. 2024.
[3] Rimsky, Nina, et al. "Steering llama 2 via contrastive activation addition." ACL. 2024.
[4] Subramani, Nishant, Nivedita Suresh, and Matthew E. Peters. "Extracting latent steering vectors from pretrained language models." Findings of ACL. 2022.
[5] Todd, Eric, et al. "Function vectors in large language models." ICLR. 2024.
[6] Turner, Alexander Matt, et al. "Steering language models with activation engineering." arXiv preprint arXiv:2308.10248 (2023).
Questions for authors
Q1: What evidence distinguishes the discovered circuits from sites that carry function-relevant information?
As stated above, I have concerns about the mechanistic interpretation of the identified supports. The paper defines a circuit as a support site paired with an input-conditioned displacement family and selects a representative support for each function. How is a site distinguished that implements the temporal computation from one that carries information computed upstream? Do the reported analyses establish any transformations or dependencies between components, or is the intended contribution the identification of causally effective intervention sites?
Q2: What is the methodological advance over activation steering, function vectors, and existing TSFM interventions?
As of now, the paper's contribution is hard to pinpoint. A methodological and experimental comparison with the above-cited works would clarify this and establish the method's contribution and novelty.
Q3: What exactly is modified during activation patching?
Appendix A.1 states that temporal/token dimensions are mean-pooled before computing state displacements, whereas Eq. 15 and 19 describe activation replacement and displacement addition. Is the original activation replaced, is a pooled displacement broadcast across positions, or is another reconstruction used? Please specify whether restoration, reverse patching, specificity testing, and transplantation use the same intervention granularity.
Other minor things
He, H., et al. (Sempo: Lightweight foundation models for time series forecasting): This is a NeurIPS 2025 accepted paper, not 2026.
7 comments
Q3. What Exactly Is Patched?
Existing methods intervene on model activations at different levels. ACDC (Conmy et al., 2023) uses clean/corrupted activation patching to trace causal connections between model components, while Function Vectors (Todd et al., 2024) aggregate causally identified attention-head outputs into task-level vectors for activation injection.
CircuitTS distinguishes localization readouts from causal intervention states. Compact, sometimes token-aggregated readouts are used only to compute M/P/A and rank candidate supports; these readouts are never directly injected.
The actual interventions operate on native activation representations:
- Sufficiency and necessity (Eqs. 15, 17): Replace the full native activation tensor at the selected support with its clean or function-disrupted counterpart, respectively.
- Functional specificity (Eq. 19): Add the full tensor-shaped, function-specific clean-minus-corrupted displacement to the target-function corrupted activation, without broadcasting a pooled vector.
- Cross-model transplantation (Eq. 25): Map donor displacements through function-specific 8D prototype coordinates and reconstruct tensor-shaped displacements in the receiver's native representation space before injection.
Both donor and receiver TSFMs remain frozen; only the low-dimensional ridge mapping is fitted using paired training histories (Appendix A.1).
This distinction ensures that localization scores identify candidate supports, while causal verification and transplantation operate on the corresponding native functional states.
We have also corrected the SEMPO reference to NeurIPS 2025.
Overall, CircuitTS combines function-targeted causal discovery with history-conditioned functional-state transfer, connecting the identification of temporal mechanisms to their reuse across heterogeneous TSFMs.
References
-
Conmy et al. (2023). Towards Automated Circuit Discovery for Mechanistic Interpretability. NeurIPS 2023.
-
Todd et al. (2024). Function Vectors in Large Language Models. ICLR 2024.
-
Wiliński et al. (2025). Exploring Representations and Interventions in Time Series Foundation Models. ICML 2025.
-
He et al. (2025). SEMPO: Lightweight Foundation Models for Time Series Forecasting. NeurIPS 2025.
Thank you for the clarifications. However, my concern regarding the term "circuit" remain. In mechanistic interpretability, a circuit is typically understood as a subgraph of interacting computational components that jointly implement a particular behavior [1,2]. The proposed formulation identifies a single causally effective support together with a family of activation displacements. Still, it does not establish the computational interactions/dependencies/transformations that implement the temporal function.
In particular, the experiments do not distinguish whether the selected site actually performs the relevant computation or merely carries information computed by upstream components. While the reported results provide evidence of causal localization and functional-state transfer, I remain unconvinced that they constitute circuit discovery in the conventional mechanistic interpretability sense.
[1] Conmy et al. Towards Automated Circuit Discovery for Mechanistic Interpretability. NeurIPS 2023. [2] Wang et al. Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 Small. ICLR 2023.
We thank the reviewer for the insightful questions. We clarify the functional-level causal interpretation of CircuitTS, distinguish its contributions from existing circuit discovery and function-vector methods, and provide additional state controls and implementation details.
Q1. What Constitutes a Functional Temporal Circuit?
A central challenge in temporal circuit discovery is identifying not only where a forecasting behavior is causally supported, but also which temporal function the underlying state represents. Since Period, Trend, and Copy coexist within continuous histories, activation importance alone cannot establish functional identity.
CircuitTS addresses this challenge by defining a functional temporal circuit as Cϕ=(sϕ,γϕ), combining a causally effective computational support sϕ with its history-conditioned functional-state family γϕ={Δϕsϕ(X)∣X∈X}.
To establish this correspondence, we construct function-targeted counterfactuals for the same history, localize supports through responsiveness, selectivity, and behavioral alignment (M/P/A), and verify them through causal sufficiency, necessity, and matched/mismatched functional recovery (R/N/V).
Importantly, functional identity is supported by causal intervention rather than activation correlation alone. Under alternative reversible interventions, the discovered Period and Trend states retain over 90% matched-function dominance across all three TSFMs, while Copy exhibits greater intervention sensitivity (Appendix A.6).
These results support function-specific causal mechanisms at the support–state level, providing a functional characterization of temporal computation without requiring a unique anatomical location or a complete edge-resolved graph (Secs. 2.2, 3.1–3.3).
Q2. How Does This Differ from Function Vectors and Steering?
Existing circuit discovery and steering methods establish important foundations, but they do not directly address function-targeted counterfactual discovery and history-conditioned state correspondence across heterogeneous TSFMs. CircuitTS advances both aspects.
(1) From behavioral to temporal-functional counterfactuals. Unlike ACDC's behavior-oriented causal connection tracing, CircuitTS constructs targeted counterfactuals for Period, Trend, and Copy within the same forecasting history. Its three complementary criteria (M/P/A) identify function-responsive, function-preferential, and behaviorally relevant supports, followed by causal verification (R/N/V). This enables functional attribution beyond generic activation importance.
(2) From fixed function vectors to history-conditioned cross-model states. Todd et al. (2024) aggregate causally relevant activations into task-level function vectors, while Wiliński et al. (2025) investigate temporal-concept steering in TSFMs. CircuitTS instead retains each history's counterfactual displacement, Δϕs(X)=as(X)−as(Tϕ(X)), and learns function-specific correspondences between independently pretrained TSFMs through 8D prototype coordinates.
State-representation controls (Appendix A.4.5, Tables 22–23):
These values measure RMSE distance to clean-model predictions, not future-target MSE. Full and same-instance PCA use the current clean/corrupted pair, whereas Fixed Mean uses training-derived states. Thus, PCA provides the more comparable representation control. Full achieves the lowest average distance, although PCA-16 remains competitive for some functions.
Furthermore, correctly matched cross-model alignment outperforms shuffled and fixed-state controls in 16/18 directed model–function cases (Table 6). Shuffling preserves mapping capacity but destroys state correspondence; both unfavorable cases involve Copy transferred into Timer.
Together, these results support the value of preserving input-conditioned functional states and learning their cross-architecture correspondence, extending beyond fixed-state injection without claiming that patching or steering itself is new.