- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies whether dynamical parameters that can be read from a model's internal representations are appropriately reflected in its forecasts. It combines linear probes, controlled changes to input histories, and direct hidden-state interventions across nine models and thirteen controlled laws. In the common seven-law comparison, 42 of 63 model--parameter combinations have accessibility above 0.95, while the median normalized forecast response is 0.46. These results provide useful evidence that readable information does not guarantee an appropriate prediction response.
Critical strengths
The controlled interventions and independent data splits support the main empirical finding. Connecting representation analysis with changes in the complete forecast also makes the framework informative beyond probing accuracy alone.
Critical weaknesses
I have the following questions and suggestions.
- How much of the target response can a full internal edit recover?
Section 3.4 (p. 5, Eqs. 10--11) constructs an internal edit using a local Jacobian approximation and evaluates the resulting response per unit step. Appendix A.5.2 (pp. 22--23) explains step calibration, while Appendix B.4.1 (pp. 33--35) examines step sensitivity and notes limits to extrapolation beyond the calibrated step.
For cases requiring a reduced step, how much of the original target change does the actual forecast recover when the full edit is applied, with and without step normalization? This result could help clarify the range of adjustment magnitudes over which the recovery discussed in Section 4.2.3 (p. 8) holds.
- How distinctive is the large forecast-insensitive component?
Section 4.2.3 (p. 8) and Appendix B.4.3 (pp. 37--38) report that a median 99.6% of the energy in input-induced hidden-state changes lies in the downstream Jacobian null space. This observation supports the proposed explanation of weak forecast responses.
Appendix A.5.3 (p. 24) already notes the large intervened-state dimensions, which range from 1,536 to 196,608. With a forecast horizon of 16, the Jacobian rank is at most 16. For an isotropic random displacement, the expected null-space energy fraction is , where is the Jacobian rank and is the hidden-state dimension. A large fraction is therefore expected from dimensionality alone.
I am curious whether random changes to the same hidden state, matched in total magnitude to the input-induced changes, would produce similar fractions. Reporting this comparison alongside each model's Jacobian rank would help establish how distinctive the input-induced changes are and how strongly the 99.6% statistic explains weak forecast responses. The random-target controls in Appendix B.4.2 (pp. 35--37) examine constructed edits and do not directly provide this displacement comparison.
Questions for authors
I have one suggestion on this work, that, an overview figure would make the framework easier to understand. The framework connects controlled parameter changes, representation probing, forecast-response evaluation, and hidden-state interventions across Sections 3.1--3.4. An overview figure showing the purpose of each component and how they connect would help readers follow the analysis and better appreciate the proposed framework.
3 comments
Thank you for the detailed response.Accept (raised from Weak Accept). The full-step experiments answer my question about how much of the target change the internal edits can recover. The random-displacement control also leads to a more convincing interpretation: the large null-space fraction is expected from dimensionality, while the new alignment controls give a clearer account of the weak forecast response. I appreciate this revision and the overview figure described in the response. My concerns are resolved; please incorporate these results and the revised explanation into the manuscript.
Thank You for Acknowledging Our Revisions
Thank you very much for your careful follow-up and updated assessment. We sincerely appreciate your constructive feedback and your reconsideration of the paper in light of our response. We will incorporate these results and the revised explanation into the final manuscript.
Response to Reviewer B5PY: Full-Step Recovery, Random Displacements, and Overview
At α=1 without step normalization, the uncapped full edit recovers the original target's reference-aligned component in many, but not all, cells. We evaluate the nonlinear forecast on 4,096 test pairs per cell and focus on the 52 cells that previously required αℓ<1. The uncapped edit is compared with the original target. The capped edit and correspondingly scaled target are evaluated separately.
Pooled gain weights pairs by target-change energy, while the last column counts cells whose median per-pair gain lies in the band. The cap limits edit norms to three times the cell median and scales each affected target by the same factor. It affects a median of 7% of pairs per cell and at most 23%.
These results show near-unit full-step aligned gain against the original target in many cells, with material exceptions. Unit aligned gain measures projection onto the reference and does not establish complete trajectory recovery. For capped full-step edits, centered relative trajectory error has median 0.186, with severe instability in Moirai seasonal autoregression. In that cell, the uncapped pooled gain is approximately −1.14×105, despite a per-pair median of 0.98, showing why both summaries are necessary. The aligned response is therefore magnitude-dependent, and constructed edits establish response capacity under intervention rather than the model's actual input-induced pathway (Appendix B.11, Table 36).
The large null-space fraction is expected from dimensionality and is not explanatory by itself. We compare each input-induced change with an isotropic displacement at the same hidden state and with the same norm. Null-space percentages are medians over seven laws for each model.
The expected fraction is 1−r/dh. Rank counts eigenvalues of JJ⊤ above 10−10 of the largest. Only one of 28,672 Time-MoE pairs has rank 15, and the displayed expectation takes rank 16. The measured random fractions closely match the dimensional expectation. Input-induced fractions vary by model, particularly for Timer-XL, but the high percentage alone does not explain weak responses. Across 63 cells, the null fraction has little association with matched-input response (ρ=0.07, p=0.59). We therefore retain the 99.6% statistic as a measurement but remove it from the explanatory argument.
The controls instead support structured but partial reference alignment. We also compare with another pair's input-induced change from the same model-law cell. This control preserves the empirical change distribution and breaks the pairing with the current history, but it is not pairwise norm-matched.
Actual input changes have greater median reference alignment than either control, although their transmitted gain remains far below one. Trend and saturating growth retain alignment under the other-pair control, indicating shared law-level directions rather than universal pair specificity. These scores measure alignment with the reference, not total forecast displacement (Appendix B.11, Tables 34-35).
Figure 1 now provides the requested overview. It connects matched parameter changes, representation probes, complete-forecast responses, and hidden-state interventions, and states the purpose of each component. The illustrative linear-trend example compares the reference, reversed matched-input response, local-edit response, and predicted-edit response.