acceptodds
Under review as a conference paper at ICLR 2027

CircuitTS: Discovering Functional Temporal Circuits in Time-Series Foundation Models

Abstract

Time Series Foundation Models (TSFMs) achieve strong zero-shot forecasting, yet their internal temporal mechanisms remain poorly understood. Existing TSF interpretability mainly reveals what temporal information is encoded, while conventional circuit analysis identifies where a behavior is supported; neither jointly characterizes a temporal function by its causal support and internal state. We introduce **CircuitTS**, which defines a functional temporal circuit as a computational support together with its function-specific causal state. CircuitTS comprises two components. **Counterfactual Circuit Discovery** constructs controlled counterfactuals for Period**, Trend, and Copy, localizes candidate supports through intervention responsiveness, function selectivity, and behavioral alignment, and verifies them through sufficiency, necessity, and specificity. **Circuit Utilization** evaluates cross-dataset generalization and aligns causal states for cross-model transplantation. Experiments on three heterogeneous TSFMs and nine real-world benchmarks show that temporal functions can exhibit different anatomical realizations while retaining causal roles, generalize to unseen datasets, and transfer across independently pretrained models. CircuitTS thus moves TSFM interpretability from representation analysis toward function-centered causal mechanisms.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Foundation Models, Time Series Forecasting, Mechanistic Interpretability, Circuit Discovery, Functional Temporal Circuits, Causal Intervention

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UydxOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CircuitTS, a framework for identifying internal sites and hidden-state differences for 3 different temporal functions: Period, Trend, and Copy. The method constructs function-specific input counterfactuals, ranks candidate sites using Intervention Responsiveness, Function Selectivity, and behavioral alignment, and evaluates shortlisted sites through restoration, reverse patching, and cross-function specificity tests. Additionally, the experiments examine cross-dataset support generalization and cross-model state transplantation across 3 time series foundation models (TSFMs) and 9 datasets, which the authors conclude is possible.

Critical strengths

Systematic causal evaluation, rather than representation analysis:

The paper combines state restoration, reverse patching, and matched vs. mismatched replacement to test the 3 functions: Period, Trend, and Copy. They demonstrate that the selected states affect the models' forecasting behavior. This is a good contribution to causal localization, even if it does not provide a complete explanation of the internal computations.

Broad transfer evaluation with relevant controls:

The authors study 3 TFSMs and 9 datasets and conduct a leave-one-out evaluation. The cross-model experiments are especially interesting as they go beyond simple steering. Here, they compare against a shuffled history and a mismatched function baseline. The matched functions achieve competitive or better results. These experiments provide evidence that intervention sites and aligned state corrections can remain useful after their discovery setting, giving the paper empirical value independently of its stronger circuit claims.

Critical weaknesses

The central "circuit discovery" claim exceeds the demonstrated mechanistic insight:

As of now, the paper supports framing input-conditioned activation differences. The presented method establishes the location and state of a function, rather than the computational dependencies that implement it. It lacks information on how different layers/components interact, which transformations are performed, and how they form the temporal behavior. In particular, Sec. 3.1 states that the selected support specifies where the function is implemented, but the restoration and reverse-patching experiments do not distinguish a site that performs the relevant transformation from one that carries information computed upstream. This affects the paper's central contribution, not only terminology. The evidence supports function-conditioned causal localization more directly than an explanation of temporal computation.

The methodological novelty is insufficiently distinguished from activation steering and function-vector work:

Furthermore, the current method closely resembles activation-steering and function-vector approaches used by many other papers [2,3,4,5,6]. Many of them, similar to the paper at hand, also support intervention layer selection. This is a substantial methodological overlap; however, comparisons or a clear distinction (experimental or methodological) to those methods is missing. The related work is not completely absent, as they already cite [1,5]. These works already connect localization of function- or concept-associated representations with activation steering: [5] uses causal component localization to extract function vectors, while [1] studies temporal-concept localization and steering in TSFMs. The claimed separation between these approaches and CircuitTS therefore needs stronger justification. Temporal counterfactuals, the localization protocol, and cross-model alignment may constitute useful advances, but the current positioning does not convincingly establish what the newly added value of the combination is, particularly what warrants calling the result a discovered circuit.

[1] Wiliński, Michał, et al. "Exploring representations and interventions in time series foundation models." ICML. 2025

[2] Sheng Liu, Haotian Ye, Lei Xing, and James Zou. "In-context vectors: making in context learning more effective and controllable through latent space steering". ICML. 2024.

[3] Rimsky, Nina, et al. "Steering llama 2 via contrastive activation addition." ACL. 2024.

[4] Subramani, Nishant, Nivedita Suresh, and Matthew E. Peters. "Extracting latent steering vectors from pretrained language models." Findings of ACL. 2022.

[5] Todd, Eric, et al. "Function vectors in large language models." ICLR. 2024.

[6] Turner, Alexander Matt, et al. "Steering language models with activation engineering." arXiv preprint arXiv:2308.10248 (2023).

Questions for authors

Q1: What evidence distinguishes the discovered circuits from sites that carry function-relevant information?

As stated above, I have concerns about the mechanistic interpretation of the identified supports. The paper defines a circuit as a support site paired with an input-conditioned displacement family and selects a representative support for each function. How is a site distinguished that implements the temporal computation from one that carries information computed upstream? Do the reported analyses establish any transformations or dependencies between components, or is the intended contribution the identification of causally effective intervention sites?

Q2: What is the methodological advance over activation steering, function vectors, and existing TSFM interventions?

As of now, the paper's contribution is hard to pinpoint. A methodological and experimental comparison with the above-cited works would clarify this and establish the method's contribution and novelty.

Q3: What exactly is modified during activation patching?

Appendix A.1 states that temporal/token dimensions are mean-pooled before computing state displacements, whereas Eq. 15 and 19 describe activation replacement and displacement addition. Is the original activation replaced, is a pooled displacement broadcast across positions, or is another reconstruction used? Please specify whether restoration, reverse patching, specificity testing, and transplantation use the same intervention granularity.

Other minor things

He, H., et al. (Sempo: Lightweight foundation models for time series forecasting): This is a NeurIPS 2025 accepted paper, not 2026.

7 comments

We thank the reviewer for the insightful questions. We clarify the functional-level causal interpretation of CircuitTS, distinguish its contributions from existing circuit discovery and function-vector methods, and provide additional state controls and implementation details.

Q1. What Constitutes a Functional Temporal Circuit?

A central challenge in temporal circuit discovery is identifying not only where a forecasting behavior is causally supported, but also which temporal function the underlying state represents. Since Period, Trend, and Copy coexist within continuous histories, activation importance alone cannot establish functional identity.

CircuitTS addresses this challenge by defining a functional temporal circuit as Cϕ=(sϕ,γϕ)\mathcal C_\phi=(s_\phi,\gamma_\phi), combining a causally effective computational support sϕs_\phi with its history-conditioned functional-state family γϕ={Δϕsϕ(X)∣X∈X}\gamma_\phi=\{\Delta_\phi^{s_\phi}(X)\mid X\in\mathcal X\}.

To establish this correspondence, we construct function-targeted counterfactuals for the same history, localize supports through responsiveness, selectivity, and behavioral alignment (M/P/AM/P/A), and verify them through causal sufficiency, necessity, and matched/mismatched functional recovery (R/N/VR/N/V).

Importantly, functional identity is supported by causal intervention rather than activation correlation alone. Under alternative reversible interventions, the discovered Period and Trend states retain over 90% matched-function dominance across all three TSFMs, while Copy exhibits greater intervention sensitivity (Appendix A.6).

These results support function-specific causal mechanisms at the support–state level, providing a functional characterization of temporal computation without requiring a unique anatomical location or a complete edge-resolved graph (Secs. 2.2, 3.1–3.3).

Q2. How Does This Differ from Function Vectors and Steering?

Existing circuit discovery and steering methods establish important foundations, but they do not directly address function-targeted counterfactual discovery and history-conditioned state correspondence across heterogeneous TSFMs. CircuitTS advances both aspects.

(1) From behavioral to temporal-functional counterfactuals. Unlike ACDC's behavior-oriented causal connection tracing, CircuitTS constructs targeted counterfactuals for Period, Trend, and Copy within the same forecasting history. Its three complementary criteria (M/P/AM/P/A) identify function-responsive, function-preferential, and behaviorally relevant supports, followed by causal verification (R/N/VR/N/V). This enables functional attribution beyond generic activation importance.

(2) From fixed function vectors to history-conditioned cross-model states. Todd et al. (2024) aggregate causally relevant activations into task-level function vectors, while Wiliński et al. (2025) investigate temporal-concept steering in TSFMs. CircuitTS instead retains each history's counterfactual displacement, Δϕs(X)=as(X)−as(Tϕ(X))\Delta_\phi^s(X)=a^s(X)-a^s(T_\phi(X)), and learns function-specific correspondences between independently pretrained TSFMs through 8D prototype coordinates.

State-representation controls (Appendix A.4.5, Tables 22–23):

ModelFull ↓PCA-16 ↓Fixed Mean ↓
SEMPO0.04710.07280.2936
Timer0.03070.07980.3261
Chronos0.12710.13800.3556

These values measure RMSE distance to clean-model predictions, not future-target MSE. Full and same-instance PCA use the current clean/corrupted pair, whereas Fixed Mean uses training-derived states. Thus, PCA provides the more comparable representation control. Full achieves the lowest average distance, although PCA-16 remains competitive for some functions.

Furthermore, correctly matched cross-model alignment outperforms shuffled and fixed-state controls in 16/18 directed model–function cases (Table 6). Shuffling preserves mapping capacity but destroys state correspondence; both unfavorable cases involve Copy transferred into Timer.

Together, these results support the value of preserving input-conditioned functional states and learning their cross-architecture correspondence, extending beyond fixed-state injection without claiming that patching or steering itself is new.

Q3. What Exactly Is Patched?

Existing methods intervene on model activations at different levels. ACDC (Conmy et al., 2023) uses clean/corrupted activation patching to trace causal connections between model components, while Function Vectors (Todd et al., 2024) aggregate causally identified attention-head outputs into task-level vectors for activation injection.

CircuitTS distinguishes localization readouts from causal intervention states. Compact, sometimes token-aggregated readouts are used only to compute M/P/A and rank candidate supports; these readouts are never directly injected.

The actual interventions operate on native activation representations:

  • Sufficiency and necessity (Eqs. 15, 17): Replace the full native activation tensor at the selected support with its clean or function-disrupted counterpart, respectively.
  • Functional specificity (Eq. 19): Add the full tensor-shaped, function-specific clean-minus-corrupted displacement to the target-function corrupted activation, without broadcasting a pooled vector.
  • Cross-model transplantation (Eq. 25): Map donor displacements through function-specific 8D prototype coordinates and reconstruct tensor-shaped displacements in the receiver's native representation space before injection.

Both donor and receiver TSFMs remain frozen; only the low-dimensional ridge mapping is fitted using paired training histories (Appendix A.1).

This distinction ensures that localization scores identify candidate supports, while causal verification and transplantation operate on the corresponding native functional states.

We have also corrected the SEMPO reference to NeurIPS 2025.

Overall, CircuitTS combines function-targeted causal discovery with history-conditioned functional-state transfer, connecting the identification of temporal mechanisms to their reuse across heterogeneous TSFMs.

References

  1. Conmy et al. (2023). Towards Automated Circuit Discovery for Mechanistic Interpretability. NeurIPS 2023.

  2. Todd et al. (2024). Function Vectors in Large Language Models. ICLR 2024.

  3. Wiliński et al. (2025). Exploring Representations and Interventions in Time Series Foundation Models. ICML 2025.

  4. He et al. (2025). SEMPO: Lightweight Foundation Models for Time Series Forecasting. NeurIPS 2025.

Thank you for the clarifications. However, my concern regarding the term "circuit" remain. In mechanistic interpretability, a circuit is typically understood as a subgraph of interacting computational components that jointly implement a particular behavior [1,2]. The proposed formulation identifies a single causally effective support together with a family of activation displacements. Still, it does not establish the computational interactions/dependencies/transformations that implement the temporal function.

In particular, the experiments do not distinguish whether the selected site actually performs the relevant computation or merely carries information computed by upstream components. While the reported results provide evidence of causal localization and functional-state transfer, I remain unconvinced that they constitute circuit discovery in the conventional mechanistic interpretability sense.

[1] Conmy et al. Towards Automated Circuit Discovery for Mechanistic Interpretability. NeurIPS 2023. [2] Wang et al. Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 Small. ICLR 2023.

Reviewer_UPDSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces CircuitTS, a framework for discovering functional temporal circuits in frozen time-series foundation models (TSFMs). A functional temporal circuit C_ϕ = (s_ϕ, γ_ϕ) pairs a computational support s_ϕ, an internal site where a temporal function is implemented, with a function-specific causal state family γ_ϕ, the activation displacements induced by a function-disrupting counterfactual. Three temporal functions are studied: Period (characteristic-harmonic removal), Trend (local linear component removal), and Copy (the latest block is replaced by the preceding block). Candidate supports are localized by an equal-weight combination of Intervention Responsiveness (RMS of the displacement), Function Selectivity (contrast against the other interventions), and Behavioral Alignment (CKA between internal and forecast displacements). They are then verified by causal sufficiency and counterfactual necessity (activation patching in both directions) and by functional specificity (a matched-transfer margin). Experiments on SEMPO, Chronos, and Timer over nine benchmarks give three main results. First, anatomical realizations differ across models: SEMPO isolates Trend in its spectral pathway, Chronos concentrates all three functions in a late decoder block, and Timer separates them across depths. Second, the causal roles verify positively, with mean R/N around 0.84-0.97. Third, supports discovered on eight datasets transfer to a held-out ninth and restore corrupted forecasts to near-original MSE in most cells, and a ridge map between 8-dimensional prototype coordinates enables Timer→SEMPO state transplantation that improves corrupted forecasts in 16 to 20 of 20 dataset-horizon cells.

Critical strengths
  • A new formulation for TSFM interpretability. Defining a temporal mechanism as a support plus a function-specific causal state, with explicit localization criteria and a three-part causal verification, moves TSFM analysis beyond probing what is encoded toward causal claims. The methodology is also more disciplined than typical patching papers: contrastive interventions, bidirectional tests, and held-out adjudication.
  • Cross-model functional transplantation. A function state from Timer can be mapped into SEMPO to restore a disrupted function, with both models frozen and only an 8-dimensional ridge map learned. If this result holds up under stronger statistics, it implies a shared functional basis across heterogeneous TSFMs.
Critical weaknesses
  • The causal claims rest on intervention-defined ground truth with limited statistical support. The discovery set has 24 windows, there are no error bars and no significance tests, and the verification metrics are computed under the same interventions used for discovery, so the headline numbers (mean R/N 0.84-0.97) are hard to calibrate. There is also no baseline comparison for localization (for example, random or magnitude-selected sites) to show that the proposed criteria add value.
  • The utilization results are promising but limited in scope. Cross-dataset generalization is shown in the main text only for SEMPO, and transplantation only for Timer→SEMPO. Recovery is partial throughout and is judged against the corrupted model rather than the original forecast. The gap between "improves 20/20 cells" and "restores the function" matters for the paper's strongest claim.
Questions for authors
  1. How do the verification metrics look for control sites, such as randomly selected sites or sites with high activation magnitude but low D_ϕ? Without this, sufficiency/necessity values lack a null reference.
  2. Can you report variance or significance for R, N, V and for the transplantation deltas (across windows, datasets, or bootstrap), and clarify the discovery-window sensitivity (Appendix A.2.1) in the main text?
  3. How should V > 1 (e.g., Chronos Trend 4.098) be interpreted? Does matched transfer overshoot the original forecast, and if so why?
  4. For transplantation, what fraction of the corruption gap (corrupt - original) is closed on average? Reporting normalized recovery would make Table 4 interpretable.
  5. Extending transplantation to at least one additional donor/receiver direction in the main text would test whether the result is pair-specific.
5 comments

We thank the reviewer for the constructive and detailed feedback on the methodological soundness and experimental rigor of CircuitTS. In response, we have strengthened the evaluation with additional localization baselines, statistical uncertainty analysis, discovery-sensitivity tests, and comprehensive cross-model transplantation controls.

Q1. Localization Baselines

A key question is whether the discovered supports reflect function-specific causal contributions or merely high activation and perturbation responsiveness. To distinguish these possibilities, we compare CircuitTS with Random, High Activation, High-Activation/Low-D, and High-M (the site with the largest function-induced activation change), using identical candidate pools and held-out windows.

ModelSelected R/NHigh-MHigh Act.High-Act./Low-DRandom
SEMPO0.836/0.8500.810/0.8240.813/0.8260.801/0.8140.826/0.838
Chronos0.967/0.9720.967/0.9720.967/0.9720.681/0.6780.720/0.705
Timer0.894/0.9020.894/0.9020.894/0.9020.894/0.9020.894/0.902

The results reveal architecture-dependent localization behavior. SEMPO improves mean R/NR/N over the activation-based controls, particularly for Trend (R=0.900R=0.900 versus 0.8290.829 for High-MM). In Chronos, High-MM and High Activation identify the same supports as CircuitTS. Selected/High-MM sites agree in 17/27, 27/27, and 13/27 dataset–function cases for SEMPO, Chronos, and Timer, respectively.

For Timer, full-tensor replacement produces identical patched predictions across all eight candidate layers, despite differences in selected sites. Token-restricted patching restores depth-dependent recovery. This indicates that full-state causal recovery does not necessarily identify a unique computational location, although functional specificity can still differ across sites.

To further examine the contribution of our localization design, we compare its three criteria: Responsiveness (MM) captures function-induced activation changes, Selectivity (PP) identifies function-preferential responses, and Behavioral Alignment (AA) relates internal changes to forecasting behavior. Their combined score DD achieves higher mean R/NR/N (0.902/0.911) than MM alone (0.889/0.898), with comparable functional specificity (VV: 1.860 versus 1.885; main Table 2).

Together, these results support the value of function-discriminative localization while clarifying that its recovery-based advantages depend on the model architecture. CircuitTS aims to identify function-specific causal supports rather than merely selecting highly activated or responsive sites (Sec. 4.2; Appendix A.2, Tables 10–12).

Q2. Statistical Uncertainty and Discovery Sensitivity

To establish that our findings are not artifacts of particular samples or discovery budgets, we assess statistical uncertainty through dataset-stratified paired bootstrap resampling (2,000 replicates for site selection; 1,000 for cross-model alignment) and examine sensitivity to discovery-set size.

Site-selection uncertainty. We compare CircuitTS against Random using paired estimates ± conservative symmetric half-widths covering the 95% bootstrap confidence intervals (Appendix Table 11):

ModelFunctionMetricDifference (95% CI)
SEMPOPeriodRR−0.015 ± 0.007
SEMPOTrendRR+0.056 ± 0.009
SEMPOTrendVV−0.071 ± 0.128
ChronosPeriodNN+0.259 ± 0.007
ChronosTrendRR+0.294 ± 0.008
TimerPeriodRR0.000 ± 0.000
TimerPeriodVV+0.223 ± 0.141

These results confirm that localization gains are architecture- and function-dependent. Chronos shows substantial improvements over Random, while SEMPO benefits particularly on Trend but not Period. Timer exhibits identical full-state recovery across sites, although functional specificity can still vary. This distinction highlights why causal recoverability and functional discrimination should be assessed separately.

Cross-model alignment uncertainty. Since transplantation gains could also depend on particular training–test samples, we examine paired Shuffled–Matched output-distance differences, where positive values favor correctly matched alignment (Appendix Table 20):

Donor → ReceiverFunctionDifference (95% CI)
Timer → SEMPOPeriod+0.0956 ± 0.0187
Timer → SEMPOCopy+0.0372 ± 0.0172
SEMPO → ChronosTrend+0.1150 ± 0.0342
Timer → ChronosCopy+0.0232 ± 0.0174
SEMPO → TimerCopy−0.0007 ± 0.0006
Chronos → TimerCopy−0.0013 ± 0.0013

Matched alignment achieves better point estimates in 16/18 direction–function combinations, with both unfavorable cases involving Copy transferred into Timer. These findings support the importance of correct donor–receiver state correspondence.

Tables 11 and 20 present representative confidence intervals; complete site-selection point estimates and function-wise transfer results are provided in Tables 10 and 19, respectively.

Discovery-budget sensitivity. To assess whether circuit discovery depends strongly on the number of training histories, we vary B∈{6,12,24,48}B\in\{6,12,24,48\}. All 81 model–dataset–function combinations retain positive mean R/NR/N across the tested budgets, while supports selected with 24 versus 48 windows agree in 7/9 model–function settings.

Overall, these experiments demonstrate consistent causal recoverability across the evaluated discovery budgets and statistically characterize the benefits and limitations of localization and cross-model alignment, without assuming that exact support selection converges monotonically (Appendix A.2.1, Table 9).

Q3. Why Can Functional Specificity Exceed One?

Functional specificity measures the recovery advantage of a matched state over mismatched states.

The recovery score G=1−d(patched,clean)/[d(corrupt,clean)+ϵ]G=1-d(\mathrm{patched},\mathrm{clean})/[d(\mathrm{corrupt},\mathrm{clean})+\epsilon] is bounded above by 1 but can be negative when injection further degrades the corrupted prediction. Consequently,

G_{\phi\to\phi}-\max_{\psi\ne\phi}G_{\phi\to\psi} \right]$$ can exceed 1 without implying more than 100% matched recovery. For example, Timer Trend yields matched $G=0.956$ and best mismatched $G=-2.184$, giving $V=3.140$ in the training-candidate evaluation. The headline Chronos Trend $V=4.098$ comes from a separate held-out evaluation. To ensure that large specificity values do not unduly determine support selection, we vary the weight of $V$ in $Q$ from 0.5 to 2 times its default value. All **9/9** model–function support selections remain unchanged. **Thus, $V>1$ is mathematically consistent with our definition, and the selected supports are robust to the tested specificity weights** (Appendix A.2.3, Table 13).

Q4. How Much Forecasting Error Does Transplantation Recover?

Although transplantation can restore disrupted forecasting behavior, its practical effect should also be quantified against ground-truth forecasting errors. We therefore introduce normalized corruption-gap recovery:

GapClosed=MSEcorrupt−MSEtransplantMSEcorrupt−MSEcleanGapClosed = \frac{MSE_{corrupt} - MSE_{transplant}}{MSE_{corrupt} - MSE_{clean}}

This measures the proportion of corruption-induced MSE degradation recovered by transplantation. We compute the ratio from pooled forecast-element-weighted MSE differences, restricting evaluation to cells with valid positive corruption gaps.

Timer→SEMPO: GapClosed (%)

FunctionH=96H=96H=720H=720
Period52.252.0
Trend83.280.3
Copy11.628.7

Trend transplantation recovers approximately 80% of the valid forecasting-error gap at both horizons, while Period recovers about half and Copy remains less effective.

Across the two evaluated directions, 85/120 dataset–horizon–function cells have valid positive corruption gaps; the others are excluded. Recovery is also direction-dependent: SEMPO→Timer Copy yields −83.2% at H=96H=96, indicating increased forecasting error after transplantation.

These results demonstrate that cross-model functional states can recover substantial corruption-induced forecasting errors, particularly for Trend, while their effectiveness varies across functions and transfer directions (Appendix A.4.3, Table 18).

Q5. Additional Transplantation Directions and Failure Analysis

To establish that functional-state transfer is not specific to a favorable donor–receiver pair or generic state injection, we extend transplantation to all six directed TSFM pairs and introduce controlled alignment comparisons.

Revised main-text Table 6 evaluates Matched, Shuffled, and Fixed donor-state transfer under a unified H=96H=96, context-512 protocol. Shuffled preserves mapping capacity while disrupting donor–receiver state correspondence; Fixed uses a training-derived mean donor state.

Mean RMSE distance to receiver-clean predictions (↓):

MatchedShuffledFixed
0.12240.18980.1849

Matched alignment achieves the lowest mean distance in all six directions and outperforms both controls in 16/18 direction–function combinations. The two exceptions are SEMPO→Timer Copy and Chronos→Timer Copy.

These results support the importance of correctly aligned, history-conditioned functional states for cross-architecture transplantation.

Failure diagnosis. To identify what limits restoration, we compare receiver-native Full-State Reference (0.0492), receiver-native 8D Prototype Reference (0.0965), and Mapped Donor (0.1224). The successive mean distance increments are 0.0473 from prototype reconstruction and 0.0259 after cross-model mapping. The unfavorable Timer-Copy cases also exhibit limited receiver-reference recovery, indicating that their failures cannot be attributed solely to alignment.

Importantly, these metrics measure output-space distance to the receiver's clean prediction, not MSE against future targets. Forecast-grounded transplantation and GapClosed are evaluated under a separate full-test protocol (Appendix A.4, Tables 16–21).

Together, the multidirectional controls and failure diagnostics provide evidence of transferable functional-state correspondence while identifying representation- and receiver-dependent limitations.

Reviewer_yP9ZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CircuitTS, a framework for identifying functional temporal circuits in pretrained time-series foundation models (TSFMs). The authors define a functional circuit through its computational support and function-specific causal state. They construct counterfactual interventions targeting three temporal functions—Period, Trend, and Copy—and identify candidate supports using intervention responsiveness, function selectivity, and behavioral alignment. The selected supports are further evaluated through activation patching to assess causal sufficiency, necessity, and functional specificity. Experiments on three TSFMs (SEMPO, Chronos, and Timer) and nine forecasting datasets suggest that the identified supports can generalize across datasets and that function-specific states can be aligned and transferred between models.

Critical strengths

The paper provides a structured framework for investigating temporal functions in TSFMs using controlled counterfactual interventions and activation patching. Combining localization criteria with causal verification is a reasonable approach to moving beyond representation-level interpretability.

The experiments cover three TSFMs with different architectures and include cross-dataset generalization and cross-model transplantation. The latter is particularly interesting because it attempts to establish functional correspondence between independently pretrained models rather than relying solely on within-model intervention results.

Critical weaknesses

The proposed framework largely adapts established techniques from mechanistic interpretability, particularly causal abstraction, activation patching, and function-vector approaches, to time-series forecasting. The combination of computational support and function-specific causal state is conceptually reasonable, but its distinction from existing causal intervention and circuit discovery frameworks is not sufficiently established. The paper would benefit from a clearer explanation of what new methodological capabilities CircuitTS provides beyond applying existing techniques to temporal functions.

The interpretation of the identified circuits as Period, Trend, and Copy relies heavily on the authors' predefined interventions. Successful restoration of predictions after activation patching demonstrates that certain internal states mediate the effects of these interventions, but does not necessarily establish that the model implements the corresponding temporal functions through those states. In particular, the interventions may modify multiple temporal properties simultaneously, making it difficult to distinguish function-specific mechanisms from generic responses to input perturbations. Stronger controls are needed to validate the proposed functional interpretations.

The experiments consider only three models and three predefined temporal functions. Although cross-dataset results are encouraging, cross-model transplantation is substantially less consistent, with some model–function pairs showing limited or negative recovery. This raises questions about whether the identified circuits represent generalizable temporal mechanisms or architecture- and intervention-specific effects. Moreover, selecting a single computational support from a predefined set of candidate sites may overlook distributed mechanisms. The paper does not sufficiently establish the robustness of its findings to alternative candidate sites, intervention designs, or model selections.

Questions for authors
  1. How does CircuitTS differ fundamentally from existing causal abstraction, activation patching, and function-vector methods? Could the authors clarify which aspects of the proposed framework constitute new methodological contributions rather than adaptations to the time-series domain?
  2. How can the authors establish that the discovered circuits genuinely implement Period, Trend, and Copy, rather than simply responding to the corresponding input transformations? In particular, how are Copy and Period distinguished when both involve temporal repetition? Could the authors provide stronger controls, such as testing whether an intervention followed by its inverse recovers the original input and whether the corresponding internal states exhibit consistent functional behavior under such reversible transformations?
  3. How robust are the discovered circuits across alternative intervention designs, candidate support sets, and model architectures? Given the inconsistent cross-model transplantation results, could the authors investigate whether the identified supports are stable and whether unsuccessful transfers indicate differences in internal mechanisms or limitations of the proposed alignment procedure?
2 comments

We thank the reviewer for recognizing the value of causal verification and cross-model transplantation. We have clarified our methodological contribution and added alternative-intervention, localization, horizon, and transplantation controls.

W1. Methodological Novelty

CircuitTS makes two key advances: from behavior-oriented counterfactual discovery (Conmy et al., 2023) to temporal-function-targeted counterfactual discovery, and from fixed function vectors (Todd et al., 2024) to history-conditioned functional-state transfer across heterogeneous TSFMs.

(1) Circuit Discovery: From behavioral counterfactuals to temporal-functional counterfactuals.

Unlike conventional circuit discovery (e.g., ACDC), which traces causal components or edges using clean/corrupted examples of a target behavior, CircuitTS constructs function-targeted counterfactuals for Period, Trend, and Copy within the same continuous forecasting history. This enables causal investigation of distinct yet intertwined temporal functions rather than only a predefined behavioral response.

To localize their computational supports, we introduce three complementary criteria: Intervention Responsiveness (M) captures function-induced activation changes, Function Selectivity (P) identifies function-preferential responses, and Behavioral Alignment (A) links internal changes to forecasting behavior. Candidate supports are subsequently verified through causal sufficiency, necessity, and specificity (R/N/V).

(2) Circuit Utilization: From fixed function vectors to history-conditioned cross-model functional-state transfer.

Unlike task-averaged function vectors (Todd et al., 2024), CircuitTS characterizes each temporal function through an input-conditioned causal-state family rather than a single fixed direction. We demonstrate two forms of utilization:

  • Cross-dataset support generalization: Discovered functional supports are frozen and reused on unseen datasets without rediscovery or target adaptation, demonstrating their cross-dataset causal reusability.
  • Cross-model state transplantation: History-conditioned causal displacements are aligned and transplanted across independently pretrained, architecturally heterogeneous TSFMs, achieving 16/18 wins over shuffled and fixed-state controls.

Together, these advances establish Cϕ=(sϕ,γϕ)\mathcal C_\phi=(s_\phi,\gamma_\phi) as a functional temporal circuit combining generalizable computational supports and transferable, input-conditioned causal states.

W2. Functional Interpretation and Intervention Robustness

Temporal functions often overlap, making it essential to distinguish genuine functional correspondence from intervention-specific responses. While CircuitTS addresses this through function-targeted counterfactuals and causal verification, we further examine whether the discovered functional relationships persist under alternative intervention designs.

Cross-intervention robustness. We introduce three self-inverse alternatives: harmonic-coefficient reflection (Period), slope reversal (Trend), and recent-block reversal (Copy). Applying each twice reconstructs the original input. Without rediscovering supports, we test whether causal states extracted under the original interventions preferentially restore their matched functions under alternative interventions.

Cross-design matched-function dominance (%):

ModelPeriodTrendCopy
SEMPO97.292.160.9
Chronos95.597.545.3
Timer91.091.562.7

Dominance measures the percentage of valid cases in which matched-function recovery exceeds both mismatched alternatives. Period and Trend exceed 90% across all models, supporting functional correspondence beyond the original perturbation design, while Copy shows greater intervention sensitivity.

To quantify potential functional overlap, we also measure input-level target-property shares (original/alternative): 63.2/68.8% for Period, 91.5/91.5% for Trend, and 89.4/71.8% for Copy. These results confirm that the interventions are function-directed but not perfectly isolated.

Copy–Period distinction. Since both functions involve temporal repetition, we further examine whether their causal states remain distinguishable after suppressing selected periodic and trend components in Copy perturbations. On the same controlled Copy target, Copy states outperform Period states in 70.8/65.1/69.8% of valid cases (SEMPO/Chronos/Timer), increasing to 88.9/84.8/77.5% after norm matching. This provides evidence of distinct causal contributions.

Together, these experiments strengthen the functional interpretation of CircuitTS beyond a single counterfactual design, particularly for Period and Trend, while identifying the remaining ambiguity of Copy (Appendix A.6, Tables 26–28).

W3. Robustness of Supports and Cross-Model Transplantation

To establish that the discovered functional circuits are reliable beyond their original discovery settings, we examine support-selection robustness, cross-horizon generalization, and cross-model transplantation.

Site-selection and discovery robustness. A causally effective support may simply be a highly responsive site, while limited discovery samples may yield unstable selections. We therefore compare Random, High Activation, and High-M (the site with the largest function-induced activation change), alongside other site controls, and vary the discovery budget (Sec. 4.2; Appendix A.2).

Chronos Selected substantially exceeds Random in sufficiency (R: 0.968 vs. 0.720), although High-M identifies the same supports. SEMPO shows function-dependent advantages. For Timer, full-tensor replacement produces identical R/NR/N across candidate layers despite different selected sites, whereas token-restricted patching reveals depth-dependent recovery. Selected/High-MM site agreement is 17/27, 27/27, and 13/27 cases for SEMPO, Chronos, and Timer, respectively.

Across discovery budgets of 6–48 windows, all 81 model–dataset–function combinations retain positive mean R/NR/N; supports discovered using 24 versus 48 windows agree in 7/9 model–function settings.

These results demonstrate robust causal recoverability under the tested discovery settings, while clarifying that full-state recovery does not always uniquely identify a computational location.

Cross-horizon generalization. To test whether functional supports remain causally effective beyond the horizon used for discovery, we freeze supports identified at H=96H=96 and reuse them at longer horizons without rediscovery, recomputing only the input-conditioned causal states:

HRNMatched-function dominance
960.9020.91197.7%
1920.8910.90292.4%
3360.8860.89482.8%
7200.8440.85371.6%

All 81 model–dataset–function combinations retain positive mean R/NR/N at every horizon. While causal recovery remains relatively stable, functional discrimination declines more noticeably at longer horizons, particularly for Chronos.

These findings support the cross-horizon reusability of discovered supports, while revealing greater horizon sensitivity in function-specific attribution. Leave-one-dataset-out experiments further validate frozen supports on unseen datasets (Appendices A.3, A.5; Tables 24–25).

Cross-model transplantation. To determine whether transplantation reflects genuine functional-state correspondence, we extend the evaluation to all six directed TSFM pairs, comparing matched alignment with shuffled correspondences and fixed donor states under a common H=96H=96, context-512 protocol:

Donor →\rightarrow ReceiverMatched ↓Shuffled ↓Fixed ↓
Timer →\rightarrow SEMPO0.1100.1800.178
Chronos →\rightarrow SEMPO0.1160.1850.178
SEMPO →\rightarrow Chronos0.1750.2560.257
Timer →\rightarrow Chronos0.1840.2740.257
SEMPO →\rightarrow Timer0.0700.1220.120
Chronos →\rightarrow Timer0.0790.1220.120

These values are RMSE distances to the receiver's clean predictions, not future-target MSE. Matched alignment outperforms both controls in 16/18 direction–function combinations; both exceptions involve Copy with Timer as receiver.

This supports learnable functional-state correspondence across heterogeneous TSFMs, beyond generic mapping or fixed-state injection.

Failure diagnosis. To understand why transplantation remains imperfect, we compare Full-State Reference, 8D Prototype Reference, and Matched Alignment, yielding mean distances of 0.0492, 0.0965, and 0.1224, respectively. The successive increments (0.0473 from prototype reconstruction and 0.0259 after mapping) indicate representation and alignment limitations. The unfavorable Timer-Copy cases also exhibit limited receiver-reference recovery, suggesting that failures cannot be attributed solely to cross-model mapping (Sec. 4.4; Appendix A.4, Tables 19–21).

Overall, these experiments establish the robustness and transferability of CircuitTS at both the computational-support and causal-state levels, while identifying architecture-, function-, and horizon-dependent limitations. We appreciate the reviewer's constructive suggestions.

References

  1. Conmy et al. (2023). Towards Automated Circuit Discovery for Mechanistic Interpretability. NeurIPS 2023.

  2. Todd et al. (2024). Function Vectors in Large Language Models. ICLR 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.