acceptodds
Under review as a conference paper at ICLR 2027

Dynamical Parameters: An Interpretability Framework for Time-Series Foundation Models

Abstract

This work studies a central gap in interpreting time-series foundation models (TSFMs): a dynamical property may be accessible in a hidden state even when the forecast fails to respond correctly as that property changes. We formalize these properties as Dynamical Parameters, including trend slope, oscillation frequency, and autoregressive dependence. We compare their representation accessibility, measured by recovery from hidden states, with their forecast response, measured by agreement with the expected forecast change. Across nine frozen TSFMs and thirteen laws, 42 of 63 model–parameter cells achieve accessibility above , whereas their median reference-aligned response relative to the conditional reference is only . To explain this gap, causal geometry compares the hidden-state change required to produce the reference response with the change induced by the parameter intervention. Directly modifying the hidden state recovers the reference response, but the parameter intervention often moves the state in a different direction. These results show that accessible parameter information need not be expressed in forecasts when input changes miss the required hidden-state direction.

Primary area:
interpretability and explainable AI
Keywords:
time-series foundation models, interpretability, dynamical parameters, representation probing, causal interventions

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_B5PYOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether dynamical parameters that can be read from a model's internal representations are appropriately reflected in its forecasts. It combines linear probes, controlled changes to input histories, and direct hidden-state interventions across nine models and thirteen controlled laws. In the common seven-law comparison, 42 of 63 model--parameter combinations have accessibility above 0.95, while the median normalized forecast response is 0.46. These results provide useful evidence that readable information does not guarantee an appropriate prediction response.

Critical strengths

The controlled interventions and independent data splits support the main empirical finding. Connecting representation analysis with changes in the complete forecast also makes the framework informative beyond probing accuracy alone.

Critical weaknesses

I have the following questions and suggestions.

  1. How much of the target response can a full internal edit recover?

Section 3.4 (p. 5, Eqs. 10--11) constructs an internal edit using a local Jacobian approximation and evaluates the resulting response per unit step. Appendix A.5.2 (pp. 22--23) explains step calibration, while Appendix B.4.1 (pp. 33--35) examines step sensitivity and notes limits to extrapolation beyond the calibrated step.

For cases requiring a reduced step, how much of the original target change does the actual forecast recover when the full edit is applied, with α=1\alpha=1 and without step normalization? This result could help clarify the range of adjustment magnitudes over which the recovery discussed in Section 4.2.3 (p. 8) holds.

  1. How distinctive is the large forecast-insensitive component?

Section 4.2.3 (p. 8) and Appendix B.4.3 (pp. 37--38) report that a median 99.6% of the energy in input-induced hidden-state changes lies in the downstream Jacobian null space. This observation supports the proposed explanation of weak forecast responses.

Appendix A.5.3 (p. 24) already notes the large intervened-state dimensions, which range from 1,536 to 196,608. With a forecast horizon of 16, the Jacobian rank is at most 16. For an isotropic random displacement, the expected null-space energy fraction is 1−r/d1-r/d, where rr is the Jacobian rank and dd is the hidden-state dimension. A large fraction is therefore expected from dimensionality alone.

I am curious whether random changes to the same hidden state, matched in total magnitude to the input-induced changes, would produce similar fractions. Reporting this comparison alongside each model's Jacobian rank would help establish how distinctive the input-induced changes are and how strongly the 99.6% statistic explains weak forecast responses. The random-target controls in Appendix B.4.2 (pp. 35--37) examine constructed edits and do not directly provide this displacement comparison.

Questions for authors

I have one suggestion on this work, that, an overview figure would make the framework easier to understand. The framework connects controlled parameter changes, representation probing, forecast-response evaluation, and hidden-state interventions across Sections 3.1--3.4. An overview figure showing the purpose of each component and how they connect would help readers follow the analysis and better appreciate the proposed framework.

3 comments
AuthorsOpenReview ↗

Response to Reviewer B5PY: Full-Step Recovery, Random Displacements, and Overview

At α=1\alpha=1 without step normalization, the uncapped full edit recovers the original target's reference-aligned component in many, but not all, cells. We evaluate the nonlinear forecast on 4,096 test pairs per cell and focus on the 52 cells that previously required αℓ<1\alpha_\ell<1. The uncapped edit is compared with the original target. The capped edit and correspondingly scaled target are evaluated separately.

Full-step edit / targetMedian pooled gain [IQR]Pooled gain within 10% of oneMedian per-pair gain within 10%
Uncapped / original1.00 [0.96, 1.08]34/5243/52
Capped / scaled1.00 [0.98, 1.05]42/5243/52

Pooled gain weights pairs by target-change energy, while the last column counts cells whose median per-pair gain lies in the band. The cap limits edit norms to three times the cell median and scales each affected target by the same factor. It affects a median of 7% of pairs per cell and at most 23%.

These results show near-unit full-step aligned gain against the original target in many cells, with material exceptions. Unit aligned gain measures projection onto the reference and does not establish complete trajectory recovery. For capped full-step edits, centered relative trajectory error has median 0.186, with severe instability in Moirai seasonal autoregression. In that cell, the uncapped pooled gain is approximately −1.14×105-1.14\times10^5, despite a per-pair median of 0.98, showing why both summaries are necessary. The aligned response is therefore magnitude-dependent, and constructed edits establish response capacity under intervention rather than the model's actual input-induced pathway (Appendix B.11, Table 36).

The large null-space fraction is expected from dimensionality and is not explanatory by itself. We compare each input-induced change with an isotropic displacement at the same hidden state and with the same norm. Null-space percentages are medians over seven laws for each model.

ModelState dimensionRankRandom, expectedRandom, measuredInput-induced
Chronos-213,8241699.8899.8999.04
TimesFM 2.510,2401699.8499.8599.49
TimesFM 312,8001699.8899.8899.92
Toto-Open-Base 1.03,0721699.4899.5099.61
Moirai6,5281699.7599.7799.81
Sundial12,2881699.8799.8899.59
Time-MoE196,60815-1699.9999.9999.94
TTM-R21,5361698.9699.0098.48
Timer-XL3,0721699.4899.5094.56

The expected fraction is 1−r/dh1-r/d_h. Rank counts eigenvalues of JJ⊤JJ^\top above 10−1010^{-10} of the largest. Only one of 28,672 Time-MoE pairs has rank 15, and the displayed expectation takes rank 16. The measured random fractions closely match the dimensional expectation. Input-induced fractions vary by model, particularly for Timer-XL, but the high percentage alone does not explain weak responses. Across 63 cells, the null fraction has little association with matched-input response (ρ=0.07\rho=0.07, p=0.59p=0.59). We therefore retain the 99.6% statistic as a measurement but remove it from the explanatory argument.

The controls instead support structured but partial reference alignment. We also compare with another pair's input-induced change from the same model-law cell. This control preserves the empirical change distribution and breaks the pairing with the current history, but it is not pairwise norm-matched.

Median over 63 cellsOwn changeAnother pair's changeIsotropic random
Null-space fraction99.55%99.59%99.85%
Row-space cosine with required edit0.480.040.00
Linearized aligned gain of JΔhJ\Delta h0.280.020.00

Actual input changes have greater median reference alignment than either control, although their transmitted gain remains far below one. Trend and saturating growth retain alignment under the other-pair control, indicating shared law-level directions rather than universal pair specificity. These scores measure alignment with the reference, not total forecast displacement (Appendix B.11, Tables 34-35).

Figure 1 now provides the requested overview. It connects matched parameter changes, representation probes, complete-forecast responses, and hidden-state interventions, and states the purpose of each component. The illustrative linear-trend example compares the reference, reversed matched-input response, local-edit response, and predicted-edit response.

Thank you for the detailed response.Accept (raised from Weak Accept). The full-step experiments answer my question about how much of the target change the internal edits can recover. The random-displacement control also leads to a more convincing interpretation: the large null-space fraction is expected from dimensionality, while the new alignment controls give a clearer account of the weak forecast response. I appreciate this revision and the overview figure described in the response. My concerns are resolved; please incorporate these results and the revised explanation into the manuscript.

AuthorsOpenReview ↗

Thank You for Acknowledging Our Revisions

Thank you very much for your careful follow-up and updated assessment. We sincerely appreciate your constructive feedback and your reconsideration of the paper in light of our response. We will incorporate these results and the revised explanation into the final manuscript.

Reviewer_FDrBOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1(−1)
Soundness
2
Read review
Paper summary

The paper studies whether dynamical properties of a time series (e.g., trend slope, frequency) can be probed linearly from the hidden states of time-series foundation models (TSFMs) and whether these are reflected in their forecasts. To study such impact, they use synthetic matched pairs that differ in a single parameter, and compare the forecast response across nine TSFMs. They find that most parameters are highly accessible, while forecast responses are often weak or even reversed. A "causal geometry" analysis based on Jacobian inversion attempts to attribute this gap to the input change inducing a hidden-state change that differs from the edit required to produce the reference response.

Critical strengths
  • How TSFMs use what they represent is a relevant and timely question.
  • The evaluation covers a wide panel of models, and the experimental details are extensively documented in the appendix.
Critical weaknesses

Major

  • W1. My main concern is the limited contribution. The paper identifies a gap, but neither explains it in depth nor proposes how to close it.

    • The central finding, i.e., that information linearly decodable from a hidden state need not be used by the downstream computation, is expected. This is well established in the probing and intrinsic interpretability literature, neither of which is discussed, and the paper itself states this premise (ll. 43-46). To me, confirming this for TSFMs is not enough of a contribution on its own.
    • The findings are not translated into anything actionable. Training objectives that align accessibility and response are only mentioned as future work in the Appendix. The "so what?" is missing: does the gap predict forecasting failures on real data? Can it guide model selection or fine-tuning? For a venue like ICLR, I would expect an analysis paper to either deliver unexpected findings or translate them into actionable insights, at least partially closing the identified gap. In my opinion, the paper does neither.
  • W2. The interpretation of the results is largely missing from the main text. Results are mostly reported as counts and medians ("42 of 63", "56 of 63", ...) without discussing \textit{why} models behave differently. For example, why do some models reverse the trend response (Fig. 3)? When can the results be obtained by looking at the forecasts alone? When are the accessibility and causal-geometry machinery necessary?

  • W3. It could be me genuinely not understanding here, but I am a bit concerned that the causal-geometry results are not very informative. The hidden state is much larger than the forecast size, so in principle there are many ways to edit it to obtain a desired forecast change. Finding such an edit (ll. 388-392, though these lines are hard to follow) is therefore not surprising. For the same reason, most hidden-state changes, even random ones, barely affect the forecast, so the claim that input changes "miss" the required direction (ll. 408-416) is hard to interpret without a random baseline.

Minor

  • W4. The presentation is unnecessarily technical and does not accompany the reader with intuitions.

    • Several symbols and terms are not defined in the main text, e.g., Jm,j\mathcal{J}_{m,j} and δ\delta (l. 137); TT and HH (despite trivial, these are used at l. 129, and defined only at l. 283), among others. Terminology is not curated, e.g., the "common state-space form", "nonlinear response criterion", "calibrated step", "rank-curve area", and "linearized aligned gain" (ll. 389-413).
    • The same quantity seems to me to sometimes carry two names (e.g., ui,m,ju_{i,m,j} and gig_i ?), and eight variants of UU are introduced (UU, Uˉ\bar{U}, U⋆U^\star, UpoolU^{pool}, ...).
    • The main text never shows a graphical summary. All figures are numeric heatmaps. A schematic of the pipeline and an example of a matched pair with the reference and TSFM responses would help.
    • Sec. 4 is a sequence of short, bold-headed paragraphs, which fragments the discussion and makes it hard to follow a coherent story.
  • W5. Limitations are only discussed briefly in Appendix.

  • W6. The term "causal geometry" seems inflated to me. As the authors acknowledge (ll. 223-225), it seems standard intervention analysis.

Questions for authors
  • Q1 (related to W3). How does the analysis in Sec. 4.2.3 compare with a random-direction baseline?
  • Q2 (related to W1 and W2). TSFMs do not observe the generating law. Is matching the law-conditioned reference always desirable? For example, damping trends is often good practice when forecasting real data. How do the authors distinguish a failure from a reasonable inductive bias?

While I remain open to a constructive discussion, I believe the paper requires a substantial revision in terms of both contribution and presentation. At this stage, I'm not inclined to raise my score, as the gap between the current submission and a version that would meet the bar for acceptance feels too wide.

8 comments
AuthorsOpenReview ↗

Response to Reviewer FDrB (1/3): Contribution and Actionability (W1)

The established observation that linearly decodable information may not affect downstream computation is the premise rather than the endpoint of our analysis. Our contribution is a controlled, quantitative test of whether a dynamical parameter is expressed in the complete forecast with the correct direction, magnitude, and temporal shape, followed by a test of whether the diagnosed mismatch can guide response-specific correction. This addresses two limitations of a probe score: it does not reveal how a parameter affects the forecast, and it does not determine whether the resulting forecast change agrees with the predictable future change.

Matched histories make this distinction measurable. Each pair differs in one parameter while sharing all remaining parameters and stochastic innovations, so the predictable difference between their futures defines the reference response. Comparing the corresponding forecasts with this reference separates parameter accessibility from forecast expression.

This comparison exposes a double dissociation that accessibility alone cannot identify. On the trend-response histories, eight models have slope accessibility of 0.998 to 0.999, yet TTM-R2, Timer-XL, and Time-MoE reverse the response across all four signed interventions. TimesFM 3 shows the complementary behavior: accessibility is 0.37 in the inspected transformer states, while its conditional-reference-normalized aligned trend gain is 1.01 because its detrending pathway removes the fitted slope before the transformer and restores its extrapolation afterward. Near-perfect accessibility in the inspected state is therefore neither sufficient nor required for an aligned response from the complete pipeline. Similar probe scores can conceal opposing forecast behavior, while pathways outside the inspected state can transmit the response.

Because the reference identifies the discrepancy quantitatively, it also provides a response-alignment objective. We test this consequence on the three reversing models through full fine-tuning and learned rank-4 modules, while retaining a forecasting loss on ETTh1, Electricity, and Weather. Forecasting-only controls determine whether ordinary adaptation produces the same correction. Correction variants are validation-selected and evaluated on independent test pairs and later real windows over three seeds.

The table reports pooled aligned gain gg and centered relative trajectory error ee. The prespecified correction criterion is g∈[0.9,1.1]g\in\left[0.9,1.1\right] and e<1e<1. Here, gg is the pooled correction gain rather than the dose-aggregated, conditional-reference-normalized Uˉ\bar U in Figure 4.

ModelPretrained g/eg/eResponse fine-tuning g/eg/eLearned rank-4 g/eg/e
TTM-R2-0.70 / 2.661.00 / 0.350.98 / 0.40
Timer-XL-0.74 / 2.781.01 / 0.271.00 / 0.31
Time-MoE-0.37 / 4.171.00 / 0.400.98 / 0.43

Both response-alignment approaches meet the criterion in all three models, whereas the forecasting-only controls have g≤0.47g\le0.47 and e≥2.23e\ge2.23. The correction therefore follows from response supervision rather than ordinary forecasting adaptation.

Correction success does not by itself establish that the diagnosed geometry provides an informative basis. We test this stronger claim by restricting the rank-4 module to the shared intervention subspace. This module reaches g/e=1.01/0.96g/e=1.01/0.96 in TTM-R2 and 0.92/0.690.92/0.69 in Timer-XL, while matched-rank random and generic Jacobian row-space modules trained with the same objective have g≤0.06g\le0.06. The gain advantages are 0.90 to 1.34, with paired 95% bootstrap intervals over test pairs excluding zero after seed averaging. The shared module fails in Time-MoE (g/e=−0.07/3.07g/e=-0.07/3.07), showing that the correction value of the geometry is model-dependent.

These results establish a bounded form of actionability. The diagnosis can guide correction of a specified response, and the diagnosed subspace can provide an effective correction basis in some models. It does not establish general model selection or guaranteed forecasting improvements. Full response fine-tuning changes mean real-dataset error by -4% to +5%, learned response modules by -2% to +2%, and shared modules by +6% to +59% relative to the pretrained models. Forecasting-only rank-4 modules improve error by 5% to 10%, so learned response modules carry an accuracy cost relative to those controls. Leave-one-law-out tests also show limited transfer, and response alignment degrades some previously aligned Time-MoE responses. Correction details appear in Section 4.2.3 and Appendix B.12, Tables 40-45. Response 3 addresses real-background behavior and when the law-conditioned reference is desirable (Section 4.3 and Appendices B.5 and B.8).

AuthorsOpenReview ↗

Response to Reviewer FDrB (2/3): Model Interpretation and Random Controls (W2, W3, Q1)

The three measurements answer different questions. Forecasts show whether a controlled parameter change produces the reference response. Probes show whether the parameter is linearly accessible in the inspected state. Interventions test selected-state response capacity and divide the matched-input response into state and bypass contributions. Thus, forecasts reveal reversal, while probes and interventions establish accessibility, response pathways, and capacity.

On unintervened synthetic trend histories, Time-MoE, TTM-R2, and Timer-XL extrapolate against the observed slope, with continuation gains of -0.82, -0.86, and -1.36. Across the nine models, continuation gain and normalized trend response have Spearman correlation 0.97. The matched-pair reversals therefore reflect broader continuation behavior, not only the intervention construction. The slope nevertheless remains linearly accessible in all three models, ruling out inaccessibility in the inspected state and locating the mismatch downstream.

For each matched pair, we replace A's selected state with B's while fixing A's other input-dependent quantities. The resulting forecast change is the state contribution. The remainder is the bypass residual, including normalization, detrending, output rescaling, and interactions with the replaced state. The terms sum exactly under this baseline and intervention order, which also makes the split intervention-defined rather than unique.

Trend settingMatched-input gainSelected-state gainBypass-residual gain
Timer-XL, aggregate-0.74+0.04-0.77
TimesFM 3, detrending always active1.000.001.00

Values are rounded. The table pools gains from 4,096 pairs at one intervention magnitude. The TimesFM 3 row is restricted to absolute slopes of at least 0.3. Timer-XL's reversal lies predominantly in the bypass residual, although its state contribution also reverses at low slopes. For TimesFM 3, detrending removes the fitted slope before the transformer and restores its extrapolation afterward. This conditional 0.00/1.00 split identifies a bypass that permits a strong response, while the 0.37 accessibility is measured on the wider trend-response distribution. TimesFM 2.5's shared-trunk mean and median heads have gains of 0.65 and 0.92, locating the difference downstream. These analyses localize the response under specified interventions, while its training origin remains unresolved.

The random-direction comparison changes the interpretation of the null-space statistic. For forecast Jacobian JJ with rank rr and state dimension dhd_h, an isotropic displacement has expected null-space fraction 1−r/dh1-r/d_h. The measured rank is 16 except for one Time-MoE pair with rank 15.

Median over 63 cellsOwn input changeAnother pair's changeIsotropic random
Null-space energy fraction99.55%99.59%99.85%
Row-space cosine with required edit0.480.040.00
Linearized aligned gain of JΔhJ\Delta h0.280.020.00

The isotropic direction is norm-matched to the current change. The other-pair control preserves the empirical distribution within the model-law cell but is not pairwise norm-matched. Because all three null fractions are large, the 99.6% fraction is not distinctive. We retain it and the 0.028 full-space cosine as measurements but exclude both from the explanatory argument. At the median, actual input changes have greater row-space alignment and gain than either control, while their gain below one indicates partial transmission. Trend and saturating growth retain alignment under the other-pair control, indicating shared law-level directions rather than universal pair specificity. Near-zero aligned gain can also coexist with forecast movement in other directions.

Constructed edits test finite-step response capacity, not the actual input-induced pathway. When dhd_h exceeds the forecast dimension, full row rank permits many linearized solutions, so finding an edit is not itself informative. Regularized inversion selects one solution, and nonlinear injection tests it at a finite step. At α=1\alpha=1 without step normalization, uncapped edits against original targets have median pooled gain 1.00, with 34 of 52 previously reduced-step cells within 10% of one. For capped edits against scaled targets, 42 of 52 cells fall within this band. Their median centered relative trajectory error is 0.186, with severe outliers including Moirai seasonal autoregression. Aligned gain measures reference projection, whereas trajectory error measures full-vector fidelity. Thus, the edits establish substantial but nonuniform response capacity, while state replacement measures actual input-induced contributions. Sections 4.2.2-4.2.3 and Appendix B.11, Tables 34-39, report these analyses.

AuthorsOpenReview ↗

Response to Reviewer FDrB (3/3): The Reference, Real Data, and Presentation (Q2, W4-W6)

Matching the law-conditioned reference is not always desirable for real-data forecasting. The reference measures fidelity to specified dynamics, while prediction error under the target distribution determines whether attenuation or reversion is beneficial. It receives the generating law and documented nuisance variables in addition to the history, whereas the TSFM receives only the history. It is therefore an informed diagnostic target rather than an information-matched accuracy benchmark. Appendix B.8 evaluates nuisance marginalization and inference over an eleven-law mixture for five parameters under the generator priors. Parameter-reference correlations remain comparatively stable, but normalization and some cross-model comparisons change. Trend is outside this test.

Within the controlled linear-trend law, positive aligned gain below one indicates attenuation of the aligned component, while negative gain indicates an opposing component. The three reversing models remain negative across all four signed interventions. This identifies a controlled response mismatch under that law, not a general forecasting failure.

Synthetic response scores track controlled response behavior on real backgrounds, not general forecasting accuracy or model-selection performance. We inject trend and sinusoid components into ETTh1, Electricity, and Weather at three strengths. The table summarizes nine across-model Spearman correlations per component.

Synthetic scoreTrend: range (median)Sinusoid: range (median)
Response0.73 to 0.98 (0.82)0.73 to 0.97 (0.93)
Accessibility-0.22 to 0.37 (-0.14)0.72 to 0.97 (0.90)

For trend, response scores distinguish behavior despite nearly tied accessibility in eight models. For sinusoid, both measures are informative. Because each setting contains the same nine models, these are fixed-panel associations rather than a prospective selection test.

Unmodified real series provide a separate test of forecasting behavior. Across 4,096 windows per dataset, continuation gain is the regression coefficient of standardized future or forecast tilt on context slope. The table correlates synthetic trend response with model continuation across the nine models.

DatasetFuture continuation gainCorrelation ρ\rhopp
ETTh1-1.360.480.19
Electricity-8.820.750.02
Weather-0.960.580.10

Negative future gains are compatible with damping or reversion, not evidence that either is universally optimal. Only Electricity's continuation association has p<0.05p<0.05. On ETTh1, weaker synthetic response accompanies a larger forecasting penalty on strongly trending relative to weakly trending windows (ρ=−0.73\rho=-0.73, p=0.03p=0.03). The corresponding trend-specific accuracy associations in Electricity and Weather are nonsignificant. Thus, the evidence supports controlled real-background relevance and two dataset-specific associations, not a universal accuracy claim (Section 4.3 and Appendix B.5, Tables 18-19).

The revision also clarifies the pipeline, notation, limitations, and claim scope. Figure 1 connects parameter changes, probes, forecast responses, and state interventions through an illustrative linear-trend example. Section 3 defines TT, HH, Im,j\mathcal I_{m,j}, δ\delta, and JJ, points to the common state-space form, and explains validation-selected calibration and the response band [0.85,1.05]\left[0.85,1.05\right]. Section 4.2.3 defines rank-curve area, while Appendix B.4.3 defines the linearized gain of JΔhJ\Delta h.

Sections 3.3-3.4 distinguish ui,m,j=giu_{i,m,j}=g_i, equal-pair Um,jU_{m,j}, reference-energy-weighted pooled gg, conditional U⋆U^\star, reference-normalized dose-aggregated Uˉ\bar U, and edit subscripts, as well as directional agreement and trajectory error. Section 4.2 now connects accessibility, forecast response, intervention capacity, state/bypass contributions, and targeted adaptation, and adds the extrapolation and detrending interpretations.

The main-text limitations now state the reference's scope, the linearity of probe accessibility, limited dynamical coverage, intervention-defined contributions, law-specific correction, and forecasting costs. The limitations in Section 5 add dependence on the selected state, step size, forecast horizon, and broader dynamics. Sections 3.4 and 4.2.3 are renamed Intervention Geometry. Section 3.4 defines effects through selected-state modifications with fixed downstream computation. Constructed edits are described as response-capacity tests, while state replacement measures actual input-induced contributions. The native forecasting algorithm and the training origins of these behaviors remain open.

Reviewer_6RaoOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes a framework for studying whether dynamical parameters, such as trend slope and oscillation frequency, are accessible (in terms of recovering their value from hidden states) in time-series foundation models and reflected in their forecast responses. They compare parameter recovery from hidden states with forecast changes under matched parameter interventions. They also construct hidden-state interventions to test whether desired forecast responses can be recovered.

Critical strengths

The main strength of this work is the breadth of the experiments across models and dynamical laws. Matched interventions and reference comparisons provide useful evidence that parameter accessibility does not guarantee an appropriate response in the forecast. The appendix also includes concrete model-specific diagnostics, particularly the analysis of the detrending pathway and differences between forecast heads.

Critical weaknesses

My main concern is the limited explanatory value (crucial in interpretability) of the causal geometry analysis. The hidden-state edits are constructed with desired forecast response, so their success demonstrates that the model can produce this response under some intervention, but does not establish how it uses the corresponding dynamical information. Although the authors acknowledge this distinction, the connection between the geometry analysis and the model-specific explanations in the appendix remains unclear.

Questions for authors

How does the causal geometry analysis contribute to the model-specific explanations?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 6Rao: Connecting Intervention Geometry to Model-Specific Explanations

Intervention geometry contributes to model-specific explanation by localizing the observed response, not by treating target-derived edit success as an explanation. State replacement quantifies how the actual matched-input response divides between the selected hidden state and bypass pathways. Code-level diagnostics then relate those contributions to the corresponding computations. Constructed edits answer a separate question: whether changing the selected state can produce the reference response.

For a matched pair, we start from history A, replace its selected state with the state from history B, and fix A's other input-dependent quantities. The resulting forecast change is the state contribution. The remainder of the complete matched-input response is the bypass residual, which includes normalization, detrending, output rescaling, and interactions with the replaced state. The terms sum exactly under this baseline and intervention order, but the split is intervention-defined rather than unique. The reference scores the response and does not construct the replacement state.

Model or settingMatched-input gainSelected-state gainBypass-residual gain
TimesFM 3, detrending always active1.000.001.00
TTM-R2, aggregate-0.70-0.18-0.51
Timer-XL, aggregate-0.74+0.04-0.77
Time-MoE, aggregate-0.37-0.13-0.24

Gain measures the response component aligned with the reference. One denotes the reference amplitude, and negative values denote reversal. Aggregate rows pool 4,096 geometry-test pairs at one intervention magnitude. The TimesFM 3 row is restricted to ∣S∣≥0.3\left|S\right|\ge0.3. Values are rounded.

The pathway split distinguishes models with similar forecast-level failures. For TimesFM 3, code inspection shows that detrending removes the fitted slope before the transformer and restores its extrapolation afterward. When detrending always activates, the 0.00/1.00 state/bypass split quantifies this route. When it never activates, the state carries 0.60 of a total gain of 0.90. The conditional split identifies a bypass that permits a strong response, while the 0.37 accessibility is measured on the wider trend-response distribution.

Timer-XL's aggregate reversal lies predominantly in the bypass, whereas TTM-R2 and Time-MoE have negative contributions through both pathways. Timer-XL's state contribution also reverses at low slopes. The same forecast-level failure therefore has different pathway contributions across models and slope regimes, which forecasts or accessibility alone cannot reveal. Across four models and three slope bins, the split agrees in sign with the code-level shape/scale decomposition in 22 of 24 contribution comparisons, with both exceptions within 0.03 of zero. This agreement is expected for standardizing wrappers. The added value is a common quantitative intervention across models and laws. TimesFM 2.5 provides complementary evidence: its shared-trunk mean and median heads have trend gains of 0.65 and 0.92, locating the difference downstream of the shared representation (Appendix B.7.2; the state-replacement analyses appear in Appendix B.11, Tables 37-39).

The shared subspace provides a separate, model-dependent correction test. With each TSFM frozen, we train a rank-4 module at the trend-selected layer with response alignment and a real-data forecasting loss. The table reports three-seed means on separate test pairs as aligned gain gg and centered relative trajectory error ee.

InterventionTTM-R2: g/eg/eTimer-XL: g/eg/e
Pretrained-0.70 / 2.66-0.74 / 2.78
Shared subspace1.01 / 0.960.92 / 0.69
Random subspace0.02 / 1.84-0.42 / 2.43
Jacobian row space0.06 / 1.910.02 / 2.18

The shared subspace meets the prespecified criterion, g∈[0.9,1.1]g\in\left[0.9,1.1\right] and e<1e<1, while both rank-matched controls fail. Its gain advantages are 0.90 to 1.34, with paired 95% bootstrap intervals over test pairs excluding zero after forecasts are averaged across seeds. This establishes correction utility in these two models, not their native computation. The result is also bounded: shared modules change real-data error by +59%, +6%, and +17% in TTM-R2, Timer-XL, and Time-MoE, respectively, and Time-MoE fails (g/e=−0.07/3.07g/e=-0.07/3.07). In contrast, unconstrained learned rank-4 modules correct all three at +2%, -2%, and -1% error, with Time-MoE reaching g/e=0.98/0.43g/e=0.98/0.43. The revised Section 4.2.3 and Appendices B.11 and B.12 therefore separate observed pathway localization, component-level interpretation, and model-dependent correction utility. The native forecasting algorithm and the training origins of these behaviors remain open.

Dear Reviewer 6Rao,

Thank you again for your feedback. Our added state-replacement analyses connect observed forecast responses to model-specific pathways, separately from constructed edits that test response capacity. We would appreciate your assessment of whether this addresses your concern about explanatory value. Please let us know if any questions remain that we can clarify before the discussion closes.

Thank you for the additional analyses. The state-replacement experiments clarify how the intervention analysis connects to model-specific pathways and address my concern about explanatory value. I also appreciate the revised interpretation following the random controls and the explicit reporting of correction costs. Considering these additions, I will raise my score.

Thank you very much for your careful review and updated assessment. We appreciate your constructive feedback and will include these results and clarifications in the final manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Summary of Revisions

The revision strengthens the paper's central result: linear accessibility does not determine whether a Dynamical Parameter is expressed in the complete forecast. The revised framework separates accessibility, forecast expression, downstream response capacity, and selected-state versus bypass contributions. Figure 1 and Sections 3.3-3.4 make these distinctions explicit. All substantive changes in the revised manuscript are marked in red.

The revised controls narrow the geometric interpretation. Appendix B.11 compares input-induced changes with norm-matched isotropic and other-pair controls, no longer treats the high null-space fraction as explanatory, tests full-size edits at α=1\alpha=1, and uses state replacement to localize model-specific pathway contributions.

Targeted response alignment provides a bounded actionability test. Appendix B.12 evaluates full fine-tuning and rank-4 modules against shared-subspace, random-subspace, and Jacobian row-space controls, and reports their forecasting and transfer costs.

The real-background and reference analyses define the claim's scope. Section 4.3 and Appendix B.5 evaluate controlled injections and unmodified real windows, Appendix B.8 examines reference sensitivity, and Section 5 distinguishes response fidelity from ordinary forecasting accuracy.

The reviewer-specific responses provide the detailed evidence, numerical results, and limitations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.