- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper studies whether different loops of a Looped Transformer perform different computations despite sharing the same weights.
- Different loops can correspond to different amounts of graph-walk progress (one hop, two hops, or staying at the same node).
- After six loops, the paper freezes the Transformer F and train an affine map J applied before the seventh loop. Different maps J_one and J_two produce one-hop and two-hop predictions with 99.6% and 98.2% accuracy.
- By patching attention patterns in the seventh loop, the paper shows that attention routing causally influences which transition is predicted.
- Intermediate supervision changes which transitions are easier to select using J.
Critical strengths
I think this paper is a good try to explain the mechanisms of looped transformers, and I thought there should be more of such paper to understand the model.
Critical weaknesses
See the question below.
Questions for authors
There is one issue that I want to ask the author: why is the problem studied non-trivial?
- Different graph-walk progress does not imply different algorithms.
Consider the linear Transformer
With and where is the adj matrix,
Although the effective walk length increases exponentially, every loop performs exactly the same operation: matrix cubing.
- The affine map may simply select existing representations.
It seems this paper is not exponential, but polynomial according to figure 2.
The first six loops may already encode information about . Then could extract the representation needed to predict the two-hop successor, rather than inducing a new algorithm in .
- Attention pattern couldn't establish a different mechanism since your input changes (it should attend to the next two node if your embedding is actually the next node, they are consistent)
5 comments
2/3
2. High-accuracy linear readability of the target forms primarily in the subsequent frozen computation
"The affine map J may simply select existing representations."
"The first six loops may already encode information about A, A², ... ."
This explanation requires distinguishing whether already directly produces an easily readable target from whether the subsequent must still retrieve, combine, or transform existing information in a state-dependent way.
To test this, we fit separate linear probes at each stage of the loop-3 experiment described in Part 1, using independent training graphs, and evaluate the answer-token representations on unseen graphs. Appendix I.3, Table R5 and Figure R5 (p. 42) report:
| Controlled target | After , before | After frozen |
|---|---|---|
| : | 22.03% | 99.79% |
| : | 27.15% | 99.92% |
These results show that the high-accuracy linear readability of the target associated with successful control is not already established immediately after ; its main increase occurs during the subsequent frozen computation. The head ablations in Table R5 and Figure R6 further show that this increase depends selectively on different components.
We do not interpret limited probe performance as evidence that target information is absent. Nor do we exclude higher-order relations being stored in graph-side tokens and subsequently accessed through conditional retrieval. Even if such precomputation occurs, it leaves a mechanistic question: how does the entering state determine which information the subsequent module uses, and which components turn that information into an output-ready target? Our additional experiments make this question testable through specific component dependencies.
3/3
3. Beyond attention changes: testing current-node readability and the causal role of addressing on the same examples
"Attention pattern couldn't establish a different mechanism since your input changes (it should attend to the next two node if your embedding is actually the next node, they are consistent)"
We agree that changes in attention, including their causal effects on the output, may still be explained by changes in a latent query address. To constrain this explanation further, we test whether the current node remains readable and whether intervening on addressing changes the target on the same examples.
We use the D8L6 seed6 setting with inserted between the two layers, documented in Appendix I.3, Figure R8 (p. 44). The following joint-subset statistics are an additional diagnostic reported here, rather than numbers displayed in Figure R8. A node probe is trained on native states from independent graphs and then frozen. Averaged over three controller fits, 88.66% of test examples simultaneously satisfy both conditions: after either , this fixed probe still decodes , and the subsequent computations reach their respective one-hop and two-hop targets.
On this common diagnostic subset, selected using the two unpatched runs, we retain the receiver's V and residual and exchange only the second-layer answer-query attention patterns across all four heads. Transferring the pattern into the run yields 40.74% two-hop target accuracy under the original output head; the reverse transfer yields 51.07% one-hop target accuracy. Thus, while the same node remains readable by a fixed probe, changing addressing weights is sufficient to redirect some examples toward the other target.
This does not imply that all latent addresses remain unchanged; the pattern itself may also transmit address information. It shows that the readable identity of the current node is insufficient to characterize how the entering state affects subsequent computation, and that addressing is a causal pathway through which control operates.
This pathway must also work with content. In the earlier D8L8 loop-3 experiment, transferring the pattern alone into produces only 1.15% two-hop target accuracy under the output head, whereas jointly replacing Q and V reaches 74.63%. These results are reported in Appendix I.3, Figure R7 (p. 43), where the QK and direct-pattern implementations agree. Control therefore cannot generally be reduced to independently switching a single attention pattern: it involves coordination among the query, retrieved content, and receiving state.
Accordingly, we will revise expressions such as "many algorithms" that could equate task-level transitions with independent algorithms. Our mechanistic conclusion is: the computational parameters are shared, while the entering state can change how subsequent components address, retrieve, and use information, and can change the causal contributions of different heads to target formation. This conclusion allows precomputation and conditional retrieval to participate in the implementation. Through generalizable state control and component interventions, our work explains concretely how a shared module supports different task-level transitions.
1/3
Thank you for highlighting the distinctions between task progress, representation selection, and algorithms. We understand your central concern: a single fixed program operating on different states can produce different progress and attention patterns, so why do these observations provide new mechanistic insight? To address this, we added paired interventions from the same native state, stage-specific probes, and component interventions to test when target representations form and how the entering state changes the causal contributions of shared components.
1. Beyond differences in progress: identifying state-dependent component contributions
We agree: a fixed update rule can produce different amounts of graph-walk progress. One-hop and two-hop readouts alone therefore do not establish two distinct algorithms, and the finite-loop curves in Figure 2 do not establish asymptotic complexity.
Prior work has explored learning reusable intermediate steps as a route to length generalization in looped Transformers, for example Fan et al., Looped Transformers for Length Generalization, ICLR 2025. This motivates a concrete mechanistic question: in a trained model, how does the entering state organize shared components to implement different task-level transitions?
In frozen D8L8 seed5, we start from the same native state before loop 3, apply either J1 or J2, and execute the same frozen block F, targeting f3(s) and f4(s), respectively. Neither target wraps around to the starting node. Each controller is shared across unseen test graphs and their different starting nodes. The setup is documented in Appendix I.3, p. 42.
We then ablate second-layer attention heads and evaluate using probes trained on the corresponding clean controlled outputs and subsequently frozen. Specifically, we zero the selected head's answer-token context before the output projection and execute the remainder of the loop. Table R5 and Figure R6 (pp. 42-43) report:
Here H0 and H2 are zero-indexed heads in the second layer. H0 is more important for target formation under J1, whereas H2 is more important under J2; the original output head reproduces this difference (Figure R6, upper panel).
Thus, the new evidence goes beyond describing distances traveled: with identical parameters, invocation position, and initial native state, different state interventions change the causal dependence of target formation on specific heads. This provides an empirical account of the functional organization within the shared module.