- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper examines how a low-dimensional hidden-state intervention affects subsequent predictions in a recurrent world model. Although the resulting state difference quickly leaves the original intervention subspace, it remains largely captured by that subspace after propagation through the model’s linearized dynamics. Restarting from the linearly predicted state also retains much of the intervention’s improvement toward a reference rollout produced by processing an edited observation. These findings recur across trained GRU models and an LSTM comparison. The paper also analyzes local output sensitivity and shows that a single linear approximation becomes less accurate at the full intervention amplitude. Its main contribution is therefore an empirical account of how low-dimensional interventions propagate within high-dimensional recurrent dynamics.
Critical strengths
-
The geometric analysis is strengthened by testing whether the approximation preserves subsequent predictions. In Section , Eq. , the authors propagate the intervention subspace through the model’s linearized dynamics and compare it with the actual state difference. However, subspace overlap alone would not establish that this approximation preserves information relevant to prediction. The restart experiment addresses this concern by initializing the unchanged model with the linearly predicted state perturbation and allowing it to continue autonomously. Section and Figure show that these restarts retain much of the original intervention’s improvement across independently trained models. The result therefore connects low-dimensional structure in the recurrent state to its effect on future predictions.
-
Replication across trained models strengthens the main empirical conclusion. Section aggregates results at the checkpoint level, while Section distinguishes the propagation and prediction results from the stricter overall evaluation, which passes in of trained GRU models. Section also reproduces the propagation result in an LSTM, although its initial intervention uses privileged counterfactual information, as explained in Sections and . This comparison provides some evidence of generality across architectures, within the stated initialization protocol. The distinction in Section between a low-dimensional intervention and a self-contained reduced state is also appropriate, because the experiments continue to use the full recurrent dynamics.
Critical weaknesses
-
The relative recovery metric leaves the absolute size of the prediction improvement unclear. Section evaluates the fraction of the original intervention’s improvement retained by the approximate restart, restricting this calculation to cases where the original improvement exceeds . Section and Figure principally report this relative measure. The retained fraction measures how much of the original intervention’s benefit survives the restart. However, a high value can reflect either a substantial prediction change or a small difference between otherwise similar rollouts. Therefore, paired prediction errors for the unmodified, intervened, and approximate-restart rollouts are needed to assess the magnitude of the reported effect. The coverage and specificity checks provide useful safeguards, but they do not directly resolve this reporting gap.
-
I am less convinced by the analysis of how the intervention changes local output sensitivity. Sections compare the leading right-singular subspaces of the output-sensitivity Jacobian—that is, the derivative of future predicted outputs with respect to the current recurrent state. Increased overlap establishes greater similarity between the state directions that most strongly affect those predictions. However, it does not establish that identical perturbations produce similar output changes. The validation in Section does not close this gap because most perturbation directions differ between the intervened and reference states, and Eq. (7) measures rankings rather than prediction accuracy for effect magnitudes. Moreover, Section 4.4 reports considerably smaller overlap gains for one checkpoint, and , alongside less consistent agreement between the complete Jacobians. The authors acknowledge these limitations. Nevertheless, the behavioral significance and robustness of this second finding remain less well established than those of the propagation result.
Questions for authors
-
How large are the prediction improvements underlying Figure ? Could the authors report paired errors for the unmodified rollout, the original hidden-state intervention, and the approximate restart, together with the distribution of the original intervention’s improvement? Please also summarize cases excluded by the threshold in Section 3.2. These quantities would distinguish substantial retained changes in prediction from high relative scores obtained when the original improvement is small, which would materially affect my interpretation of the main behavioral result.
-
Could the authors report the relative error over rollout time? Eq. measures how much of the actual state difference lies in the propagated subspace, but it does not measure whether Eq. predicts the correct direction and magnitude within that subspace.
-
How sensitive are the subspace comparisons to the chosen truncation rank?