- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks when facts stored in a looped (weight-tied) transformer become readable across recurrences. In single-hop retrieval on synthetic biographies, readable bits, measured with a tuned lens, rise gradually over loops, and retrieval completes later as more individuals are stored. Meanwhile, answer-position states shift from organization by individual to organization by attribute value. In multi-hop reasoning, probes on one fixed- and one dynamic-recurrence checkpoint from Kohli et al. (2026) decode bridge entities in chain order. The dynamic model solves chains earlier, and patching the frontier entity redirects the final answer.
Critical strengths
-
The patching experiments show that the model uses its intermediate states. With frontiers fixed on validation data, entity-subspace patches at recurrences 2-5 redirect the final answer far more often than norm-matched random-subspace patches (Fig. 7). Probing alone cannot establish this.
-
The single-hop setting is well controlled. Known attribute entropy gives the bit measure a meaningful zero and ceiling, and the load ordering holds for both loop budgets and all five attributes. For the value score, a structure-preserving permutation sets the floor, and a control group whose facts the model never saw ties value structure to stored facts rather than to the prompt. Together, these choices give both measures a calibrated floor.
Critical weaknesses
-
No experiment separates recurrence from depth, or the depth retrieval "requires" from the trained loop budget. The Introduction posits "a fundamental difference in how the two architectures process information" (p.1 l.41-42), yet every depth-resolved analysis uses looped models only. Standard transformers show similar layer-wise patterns, with subject enrichment preceding attribute extraction (Geva et al., EMNLP 2023) and bridge entities resolving before the second hop (Biran et al., EMNLP 2024). The depth the paper calls "required" (p.2 l.77-78) also tracks the loop budget, as Sec. 2.2 itself notes. At equal parameter count, the single-loop model stores as many bits as the looped models (Fig. 5), so one block application suffices when training targets it (Q1). By my reading of Fig. 1 (bottom-right), the eight-loop model at the lightest load needs more loops to complete retrieval than the four-loop model at the heaviest, for every single-token attribute. Without these controls, the results show how these looped models use depth, not what recurrence adds or what the load requires. I would ask for the same analyses on untied models of equal unrolled depth, one at equal width (matched compute) and one narrowed to match parameters. I would also like completion depth normalized by the loop budget, as Sec. 2.3 does for the midpoint, and described as depth used rather than required.
-
The conversion account and the fixed-versus-dynamic comparison do not yet rule out simpler explanations. Because Figs. 2 and 6 (right) rescale every trained curve to peak at one, a falling curve has to cross a rising one. The crossing therefore says little, and neither does the claim that the next entity "eventually overtakes" the current one (p.7 l.345-346); the midpoint, by contrast, survives the rescaling. The key-score decline also shows up in the control group, so it happens without retrieval and cannot by itself show conversion. The fixed-versus-dynamic comparison rests on one checkpoint per regime with unreported training differences (Q3). Sampling loop counts also supervises the answer at several depths, which the paper's own case for final-loop supervision (Sec. 2.1) predicts will yield earlier answers. Earlier solving alone is therefore weak evidence of an inductive bias beyond the objective (p.7 l.335-336). Both matter: the key-to-value transition is a stated contribution, and the regime comparison carries the Conclusions' lead recommendation. Unnormalized scores, or a common-scale effect size such as the share of variance the grouping explains, would address the first. For the second, I would ask for at least three seeds per regime on matched data, or a claim limited to the two checkpoints.
Questions for authors
-
Does a per-layer lens on the single-loop model show retrieval completing within one block application?
-
Is the tuned lens split by individual or by paraphrase, and how do its train and test readouts compare at early depths?
-
How do the two multi-hop checkpoints differ in training data, steps, and loop-count sampling?
-
Each single-hop configuration is one run, as the Limitations acknowledge, and Figs. 1-2 show no error bars. How large is seed-to-seed variability relative to sub-loop effects such as the midpoint shift from 20K to 50K individuals? Please also state which checkpoint Sec. 3.3 patches, how Fig. 2 is normalized, the entity-subspace rank, and the single-hop models' layers per block, optimizer, learning rate, and training length.
-
The paper trains no single-hop model with intermediate supervision and evaluates no exit rule, so its intermediate-supervision and early-exit recommendations remain hypotheses. Testing either one would give the corresponding advice empirical support.
8 comments
W2
We agree that the peak-normalized crossings and the decline in the key score do not, by themselves, establish conversion from key to value representations. We also agree that the original comparison of two checkpoints cannot establish an advantage of dynamic-loop training beyond differences in supervision and training history. We have revised and expanded the analyses in the revised version of the paper.
Figures 2 and 6 in the submission are now Figures 4 and 14, respectively. Instead of normalizing each curve to its own maximum, they report , where is the fraction of variance explained by the grouping and is its mean under label permutation. Thus, the plotted quantity is a difference in explained variance. In the single-hop models, the key score decreases from 0.81–0.83 to 0.66–0.76, while the value score increases and remains below it. The final value scores are 0.15–0.17 at 20K, 0.09 at 50K and 0.05 at 100K (Section 2.3, Figure 4).
The control group is essential to interpreting these trends. For individuals in split C, who were never seen during backbone training, the key scores remain within 0.02 of those on A, whereas . We therefore do not interpret either identity separation or the decline in the key score as evidence of retrieval. The distinction between trained and unseen individuals lies in the emerging structure associated with attribute values. These results support a descriptive account of that emergence across depth, but do not by themselves establish that key information is converted into value information. We retain the geometric midpoint as a timing statistic: it is the first depth at which the permutation-normalized value score reaches half of its final excess above baseline, with interpolation in log-excess (Section 2.3; Appendix E.1).
For the multi-hop task, we have removed the claim that the next entity eventually overtakes the current one. Within the selected set of 1069 chains, the current entity's rises from 0.05 at layer 4 of loop 2 to 0.63 at layer 3 of loop 3, then falls to 0.42 at layer 4 of loop 5. Over the same interval, the next entity's score rises from 0.003 to 0.40 but remains lower at every measured depth. This is a description of the selected bridge transition (Appendix D.1, Figure 14).
For the fixed-versus-dynamic multi-hop comparison, we now make the supervision difference explicit. Dynamic training applies the loss only at the sampled final recurrence of each batch; across batches, however, it supervises answers at several depths. This leaves precisely the objective-based explanation raised by the reviewer: earlier answers need not reflect an inductive bias beyond supervision at earlier depths. The released checkpoints also differ in curriculum range and training duration, now documented in Section 3.1 and Appendix D.2, Table 2. We properly rephrased thge corresponding claims.
We have added three independently trained seeds per regime using the same source code, data and curriculum, with seeds varying initialization and data order. We compare the checkpoints at the end of the same 13-hop curriculum stage (Section 3.2, Figure 7; Appendix D.2). The metric is the first loop at which endpoint accuracy reaches 0.8. For 2--6 hops, dynamic models reach this threshold at the same loop or one earlier; for 7--10 hops, they generally reach it at the same loop. For 11 and 13 hops, they reach it one loop later, and for 12 hops, at the same loop or one later. At 13 hops, all three dynamic seeds require seven loops, compared with six for the fixed models. Thus, the matched-curriculum comparison does not show a consistent timing advantage for dynamic training.
Q1
Yes. The new per-layer lens analysis shows factual readability developing within a single application of the 11-layer block (Section 2.2, "Single-pass comparisons" paragraph; Appendix B, Figure 11). In these parameter-matched single-loop models, the mean readable bits across the five attributes first becomes positive at layers 3, 5, 7, 8 and 8 for 20K, 50K, 100K, 200K and 500K individuals, respectively.
At 20K, 50K and 100K, the lens reaches half of its final score at 65%, 83% and 92% of the block's depth. The corresponding fractions of unrolled depth are 25%, 40% and 65% for four-loop models, and 23%, 42% and 61% for eight-loop models. Thus, the dependence of readout timing on memorization load is not specific to recurrence. We distinguish reaching a fraction of a model's final score from recovering all the information in the data: at 500K, for example, the final score is 4.06 bits, compared with a source entropy of 9.32 bits.
Figure 11 also reports the controls. Readable bits on never-trained C remain below the uniform-value log-loss baseline, and native readouts become positive later than lens readouts at every load. Constraining the lens to an identity map plus an update of rank 16, 32 or 64 changes the interpolated depth at which it first reaches 95% of its final score by at most 0.23 layers.
Q2
The split is by individual, not by paraphrase. A, B and C contain 60%, 35% and 5% of the individuals, respectively, with no names shared between them. The lens is fitted on 95% of A and validated on the remaining 5%; B is used for neither fitting nor model selection. Both A and B are included in backbone training, whereas C is unseen by the backbone. The reported A curves include both the fitting and validation individuals (Section 2.1; Appendix C).
Figure 2 and Appendix C, Table 1 compare A and B across five seeds. The table below gives the largest A--B gap over depths and seeds, together with the first-loop readable bits on B. The latter are reported as mean 1 standard deviation across seeds.
| Individuals | Largest A--B gap, four loops (bits) | Largest A–B gap, eight loops (bits) | First-loop B score, four loops (bits) | First-loop B score, eight loops (bits) |
|---|---|---|---|---|
| 20K | 0.69 | 0.70 | ||
| 50K | 0.39 | 0.38 | ||
| 100K | 0.21 | 0.21 |
The A--B gaps are below 0.71 bits at every measured depth and at most 0.017 bits at the final depth. The ordering by load also persists on B. At 20K, 50K and 100K, respectively, the mean depths at which the four-loop models first reach 95% of their final score are 1.92, 2.92 and 3.75 loops on A, compared with 1.99, 2.98 and 3.77 on B. For eight-loop models, the corresponding depths are 3.53, 5.28 and 6.65 on A, and 3.86, 5.53 and 6.73 on B. Thus, the timing pattern is not confined to individuals used to fit the lens. Scores on never-trained C remain below the uniform-value log-loss baseline (Appendix C).
We appreciate the reviewer feedback and in what follows we address their concerns
W1
We agree that our measurements describe how trained models use depth, rather than the depth intrinsically required to retrieve a fact. We have revised the abstract accordingly and removed the claim of a fundamental architectural difference from the Introduction. Section 2.2 now defines the reported “completion” depth operationally: the first interpolated depth at which a readout reaches 95% of its own final score.
We really appreciate the reviewer's suggestion. To address the architectural comparison, we added untied single-hop models with 44 and 88 layers, matching the unrolled depths of the four- and eight-loop models. At each depth, we compare an equal-width model and a narrower model with approximately the same number of non-embedding parameters as the looped model. The data, person-level split, training steps, optimizer and global batch size are matched; each configuration uses one run with seed 0 (Section 2.2, Figure 3; Appendix E.2, Table 4). The table below reproduces the lens results. Each entry gives the first depth at which the lens reaches 50% and 95% of its final score, expressed as a fraction of the model's total unrolled depth.
In these experiments, the looped models reach half of their final readable bits earlier in their unrolled computation than either untied control. Final scores are 8.93 bits, except for the narrowed 88-layer model at 100K, which reaches 8.46 bits. One detail worth emphasizing is that parameter matching is approximate: the narrowed 88-layer model has about 4.5% fewer non-embedding parameters. The sampling resolutions also differ: looped models are read at loop boundaries, whereas untied models are read at every layer. In particular, the 20K four-loop model already exceeds the 50% threshold at the first sampled loop, so its reported depth of 0.25 is an upper bound. These comparisons do not imply that load-dependent timing is unique to recurrence; the single-block analysis also shows this dependence (see our answer to Q1 below).
We also report depth relative to the trained loop budget. At 20K, 50K and 100K, respectively, the mean 95% crossing (across five seeds) occurs at 0.50, 0.74 and 0.94 of the four-loop budget, and at 0.48, 0.69 and 0.84 of the eight-loop budget (Section 2.2, Figure 1). Models trained with more loops use more recurrences in absolute terms, while increasing the load delays the crossing within either budget. The reviewer reading of the four single-token attributes is correct: their 95% crossings occur later in the 20K eight-loop model than in the 100K four-loop model, for both native and lens readouts. We report in Appendix E, Figure 18, the comparison for each attribute independently.