- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper argues that the weights of looped Transformer models are "approximately equivariant" when trained for the next-state prediction of linear stochastic systems. By fitting trained models to a (relatively complicated) equivariant ansatz, it is claimed that the resulting "core matrices" form a small and interpretable set of parameters that are useful for transfer across tasks.
Critical strengths
The idea of looking for exploitable structure in the weights of trained Transformers is interesting. It is shown that training induces a shared low-rank bias in the weights, supporting certain "folklore" statements in the field.
Critical weaknesses
The paper is very hard to understand and the results are largely unconvincing. It is entirely empirical, which is okay and could still be valuable, but right now the work presents as a collection of poorly explained experiments that do little to support the authors' conclusions. It is not even clear that the baseline task is meaningful: next-state prediction of stochastic dynamical systems (measured by MSE) is dubious since random noise is deliberately injected at each step. The performance of the baseline models is also not discussed at all, making it impossible to interpret the authors' results. How can I understand that the equivariant projection is meaningful here when the only metric is "Final MSE / Projected MSE"? MSE is also unnormalized, so raw MSE values do not communicate anything about how the model is actually performing.
I should also mention that it is very difficult to separate the work of the authors from the work of generative AI in this paper. The scope of work here is something that could be potentially "one-shot" by current frontier AI, as it relies on experimental analysis performed with a very simple architecture, and the figures and prose are heavily AI-assisted. While I trust that the authors have exercised appropriate scientific judgement, I do not believe that this work is sufficiently novel or interesting to warrant publication.
Questions for authors
Can you provide convincing evidence that the training task is meaningful, or that the equivariant approximation is satisfied? Can you present simple experiments that illustrate the baseline model's performance alongside that of the equivariant approximation?
2 comments
References
-
Cole, F., Zhao, Y., Lu, Y., and Zhang, T. (2025). In-Context Learning of Linear Dynamical Systems with Transformers: Approximation Bounds and Depth-Separation. NeurIPS.
-
Vlădărean, M.-L., Zhang, X., and Sra, S. (2026). On learning linear dynamical systems in context with attention layers. ICLR.
-
Du, Z., Balim, H., Oymak, S., and Ozay, N. (2023). Can Transformers Learn Optimal Filtering for Unknown Systems?. IEEE Control Systems Letters, 7, 3525–3530.
Thank you for your feedbacks and for pointing out where our explanation of the prediction task and the role of the equivariant projection was unclear. In our general response, we explain the scope of the task family and how we revised the manuscript to make the model design and analysis easier to follow. We chose stochastic linear systems because they provide a commonly used setting for prediction and filtering. Recent work has also studied how Transformers learn these tasks in context (Du et al., 2023; Cole et al., 2025; Vlădărean et al., 2026). Building on this work, we ask how a looped Transformer with softmax attention uses past transitions to predict the next state of an unknown system and whether the learned computation can be reused across input dimensions.
Prediction with process noise. The transition matrix Bτ is unknown to the model but fixed within each trajectory, so past transitions xt→xt+1 provide information about the systematic part of the next step, Bτxt. We train on noisy next states. Since the next noise increment is independent and has zero mean, it adds only a constant to the expected squared error (Appendix A.7). Because we generate the trajectories, we can evaluate each prediction against the expected next state under the true dynamics, Bτxt. In the revised Section 2.1, we distinguish this conditional-mean MSE from the noisy-target MSE used in training.
Baseline performance. Following your suggestion, we revise Appendix B.1 to report training loss and predictive performance across model configurations, including NMSE to account for differences in state scale. We normalize the total squared prediction error by the total squared norm of the conditional-mean targets. In Table 1, the reference model achieves all-position NMSE of 0.050 on ID tasks and 0.084 on OOD tasks, well below persistence (0.637 and 0.932, respectively). Causal ridge is more accurate on both sets (0.042 and 0.031).
Original and projected models. We agree that the original label “Final MSE / Projected MSE” in Fig. 5(a) was ambiguous. Fig. 5 is actually intended to test circuit interventions after projection, but not to measure the predictive performance of the projected model. In the revised manuscript, we updated the y-axis label in Fig. 5(a), giving the MSE with the intervention divided by the MSE without it, both for the same projected model. Fig. 3(c) evaluates the effect of equivariant projection itself by comparing prediction errors before and after projection on the same trajectories. Across 25 models, projection changes outputs by 3.3% on ID and 4.2% on OOD, while reducing conditional-mean MSE by 1.79% and 1.73% on average (Appendix C.4)(These relative changes are identical for MSE and NMSE because the NMSE denominator depends only on the evaluation data, not on the model). Fig. 3(b) also shows how closely the weights match the predicted structure: on average, it captures over 99% of the squared Frobenius norm for the embedding, readout, and OV operators, and about 90–93% for the routing forms.
Theory was essential to identifying the learned structure. Our study began with an empirical puzzle, and numerical experiments played a substantial role throughout. Even in this simple architecture, the trained OV operators developed striking spectral degeneracies. Those patterns hinted to a hidden symmetry, but by themselves did not tell how the hidden states were organized. It took several rounds of theoretical reasoning and numerical investigation to turn that initial clue into a hypothesis with a testable consequence.
Specifically, if hidden states transform as vector channels in a shared basis and the modules respect that transformation, Theorem 1 determines the form their weights must take. For the OV operators, this form also explains the repeated eigenvalues that first drew our attention. In this process, the theory told us what to look for. For each of 25 trained models, we fitted a basis using hidden activations alone, then examined whether the predicted structure appeared in the weights. Figure 3(d) illustrates the weight structure revealed by this change of basis, and Figure 3(b) quantifies how much of each operator’s squared norm the projection preserves. We have revised Sections 3–4 to make this reasoning explicit.
As our AI Use Statement acknowledges, AI tools provided substantial assistance. In our experience, however, describing the work as “one-shot” overlooks its central scientific challenge: deciding what the spectral evidence could mean, developing a theory precise enough to guide the next experiment, and revising that theory in light of the results.