acceptodds
Under review as a conference paper at ICLR 2027

When Do Stored Facts Become Available in Looped Transformers?

Abstract

Looped transformers do not acquire new knowledge during a forward pass, yet repeated applications of their shared block can change when and how stored knowledge becomes available. We study this process in looped transformers trained on synthetic tasks, focusing on factual retrieval and multi-hop reasoning. We first investigate single-hop factual retrieval. While prior work shows that looped and non-looped transformers store the same number of bits in their weights, it remains unclear how recurrence makes this knowledge available during inference. We find that stored facts become readable progressively across recurrences rather than through an immediate lookup. Moreover, the number of recurrences required to recover them increases with memorization load. Through representational analysis, we characterize this progression as a transition from representations of the query key to representations of its associated value. We then study multi-hop reasoning, where each retrieved entity serves as the key for the next relation. We find that intermediate entities emerge in logical order across recurrences, while the hidden-state geometry progressively shifts from the current frontier entity toward its successor. Comparing fixed- and dynamic-recurrence training, we find that dynamic recurrence exhibits faster state turnover and enables earlier solutions. We further show that these representations are functionally involved in the computation: counterfactual patches to the frontier entity redirect subsequent bridge entities and ultimately change the model’s answer. Together, these results show that stored knowledge becomes progressively available in representations that support subsequent computation. More broadly, these findings suggest that recurrence organizes when and how stored knowledge can be used, rather than increasing storage capacity, motivating training and early-exit strategies that account for the progressive availability of knowledge across recurrences.

Primary area:
generative models
Keywords:
Looped transformers, knowledge retrieval, multi-hop reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QzwUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks when facts stored in a looped (weight-tied) transformer become readable across recurrences. In single-hop retrieval on synthetic biographies, readable bits, measured with a tuned lens, rise gradually over loops, and retrieval completes later as more individuals are stored. Meanwhile, answer-position states shift from organization by individual to organization by attribute value. In multi-hop reasoning, probes on one fixed- and one dynamic-recurrence checkpoint from Kohli et al. (2026) decode bridge entities in chain order. The dynamic model solves chains earlier, and patching the frontier entity redirects the final answer.

Critical strengths
  1. The patching experiments show that the model uses its intermediate states. With frontiers fixed on validation data, entity-subspace patches at recurrences 2-5 redirect the final answer far more often than norm-matched random-subspace patches (Fig. 7). Probing alone cannot establish this.

  2. The single-hop setting is well controlled. Known attribute entropy gives the bit measure a meaningful zero and ceiling, and the load ordering holds for both loop budgets and all five attributes. For the value score, a structure-preserving permutation sets the floor, and a control group whose facts the model never saw ties value structure to stored facts rather than to the prompt. Together, these choices give both measures a calibrated floor.

Critical weaknesses
  1. No experiment separates recurrence from depth, or the depth retrieval "requires" from the trained loop budget. The Introduction posits "a fundamental difference in how the two architectures process information" (p.1 l.41-42), yet every depth-resolved analysis uses looped models only. Standard transformers show similar layer-wise patterns, with subject enrichment preceding attribute extraction (Geva et al., EMNLP 2023) and bridge entities resolving before the second hop (Biran et al., EMNLP 2024). The depth the paper calls "required" (p.2 l.77-78) also tracks the loop budget, as Sec. 2.2 itself notes. At equal parameter count, the single-loop model stores as many bits as the looped models (Fig. 5), so one block application suffices when training targets it (Q1). By my reading of Fig. 1 (bottom-right), the eight-loop model at the lightest load needs more loops to complete retrieval than the four-loop model at the heaviest, for every single-token attribute. Without these controls, the results show how these looped models use depth, not what recurrence adds or what the load requires. I would ask for the same analyses on untied models of equal unrolled depth, one at equal width (matched compute) and one narrowed to match parameters. I would also like completion depth normalized by the loop budget, as Sec. 2.3 does for the midpoint, and described as depth used rather than required.

  2. The conversion account and the fixed-versus-dynamic comparison do not yet rule out simpler explanations. Because Figs. 2 and 6 (right) rescale every trained curve to peak at one, a falling curve has to cross a rising one. The crossing therefore says little, and neither does the claim that the next entity "eventually overtakes" the current one (p.7 l.345-346); the midpoint, by contrast, survives the rescaling. The key-score decline also shows up in the control group, so it happens without retrieval and cannot by itself show conversion. The fixed-versus-dynamic comparison rests on one checkpoint per regime with unreported training differences (Q3). Sampling loop counts also supervises the answer at several depths, which the paper's own case for final-loop supervision (Sec. 2.1) predicts will yield earlier answers. Earlier solving alone is therefore weak evidence of an inductive bias beyond the objective (p.7 l.335-336). Both matter: the key-to-value transition is a stated contribution, and the regime comparison carries the Conclusions' lead recommendation. Unnormalized scores, or a common-scale effect size such as the share of variance the grouping explains, would address the first. For the second, I would ask for at least three seeds per regime on matched data, or a claim limited to the two checkpoints.

Questions for authors
  1. Does a per-layer lens on the single-loop model show retrieval completing within one block application?

  2. Is the tuned lens split by individual or by paraphrase, and how do its train and test readouts compare at early depths?

  3. How do the two multi-hop checkpoints differ in training data, steps, and loop-count sampling?

  4. Each single-hop configuration is one run, as the Limitations acknowledge, and Figs. 1-2 show no error bars. How large is seed-to-seed variability relative to sub-loop effects such as the midpoint shift from 20K to 50K individuals? Please also state which checkpoint Sec. 3.3 patches, how Fig. 2 is normalized, the entity-subspace rank, and the single-hop models' layers per block, optimizer, learning rate, and training length.

  5. The paper trains no single-hop model with intermediate supervision and evaluates no exit rule, so its intermediate-supervision and early-exit recommendations remain hypotheses. Testing either one would give the corresponding advice empirical support.

8 comments

We appreciate the reviewer feedback and in what follows we address their concerns

W1

We agree that our measurements describe how trained models use depth, rather than the depth intrinsically required to retrieve a fact. We have revised the abstract accordingly and removed the claim of a fundamental architectural difference from the Introduction. Section 2.2 now defines the reported “completion” depth operationally: the first interpolated depth at which a readout reaches 95% of its own final score.

We really appreciate the reviewer's suggestion. To address the architectural comparison, we added untied single-hop models with 44 and 88 layers, matching the unrolled depths of the four- and eight-loop models. At each depth, we compare an equal-width model and a narrower model with approximately the same number of non-embedding parameters as the looped model. The data, person-level split, training steps, optimizer and global batch size are matched; each configuration uses one run with seed 0 (Section 2.2, Figure 3; Appendix E.2, Table 4). The table below reproduces the lens results. Each entry gives the first depth at which the lens reaches 50% and 95% of its final score, expressed as a fraction of the model's total unrolled depth.

DepthModelNon-emb. params20K50K100K
44looped, loop-44.89M0.25 / 0.500.40 / 0.730.65 / 0.96
untied, equal width19.6M0.81 / 0.940.82 / 0.950.81 / 0.95
untied, approx. matched4.92M0.93 / 0.990.97 / 1.000.97 / 1.00
88looped, loop-84.89M0.23 / 0.510.42 / 0.690.61 / 0.84
untied, equal width39.1M0.88 / 0.960.88 / 0.970.83 / 0.95
untied, approx. matched4.68M0.96 / 1.000.99 / 1.000.99 / 1.00

In these experiments, the looped models reach half of their final readable bits earlier in their unrolled computation than either untied control. Final scores are 8.93 bits, except for the narrowed 88-layer model at 100K, which reaches 8.46 bits. One detail worth emphasizing is that parameter matching is approximate: the narrowed 88-layer model has about 4.5% fewer non-embedding parameters. The sampling resolutions also differ: looped models are read at loop boundaries, whereas untied models are read at every layer. In particular, the 20K four-loop model already exceeds the 50% threshold at the first sampled loop, so its reported depth of 0.25 is an upper bound. These comparisons do not imply that load-dependent timing is unique to recurrence; the single-block analysis also shows this dependence (see our answer to Q1 below).

We also report depth relative to the trained loop budget. At 20K, 50K and 100K, respectively, the mean 95% crossing (across five seeds) occurs at 0.50, 0.74 and 0.94 of the four-loop budget, and at 0.48, 0.69 and 0.84 of the eight-loop budget (Section 2.2, Figure 1). Models trained with more loops use more recurrences in absolute terms, while increasing the load delays the crossing within either budget. The reviewer reading of the four single-token attributes is correct: their 95% crossings occur later in the 20K eight-loop model than in the 100K four-loop model, for both native and lens readouts. We report in Appendix E, Figure 18, the comparison for each attribute independently.

W2

We agree that the peak-normalized crossings and the decline in the key score do not, by themselves, establish conversion from key to value representations. We also agree that the original comparison of two checkpoints cannot establish an advantage of dynamic-loop training beyond differences in supervision and training history. We have revised and expanded the analyses in the revised version of the paper.

Figures 2 and 6 in the submission are now Figures 4 and 14, respectively. Instead of normalizing each curve to its own maximum, they report Δη2=η2−ηnull2\Delta\eta^2=\eta^2-\eta^2_{\rm null}, where η2\eta^2 is the fraction of variance explained by the grouping and ηnull2\eta^2_{\rm null} is its mean under label permutation. Thus, the plotted quantity is a difference in explained variance. In the single-hop models, the key score decreases from 0.81–0.83 to 0.66–0.76, while the value score increases and remains below it. The final value scores are 0.15–0.17 at 20K, 0.09 at 50K and 0.05 at 100K (Section 2.3, Figure 4).

The control group is essential to interpreting these trends. For individuals in split C, who were never seen during backbone training, the key scores remain within 0.02 of those on A, whereas ∣Δηvalue2∣<0.01|\Delta\eta^2_{\rm value}|<0.01. We therefore do not interpret either identity separation or the decline in the key score as evidence of retrieval. The distinction between trained and unseen individuals lies in the emerging structure associated with attribute values. These results support a descriptive account of that emergence across depth, but do not by themselves establish that key information is converted into value information. We retain the geometric midpoint as a timing statistic: it is the first depth at which the permutation-normalized value FF score reaches half of its final excess above baseline, with interpolation in log-excess (Section 2.3; Appendix E.1).

For the multi-hop task, we have removed the claim that the next entity eventually overtakes the current one. Within the selected set of 1069 chains, the current entity's Δη2\Delta\eta^2 rises from 0.05 at layer 4 of loop 2 to 0.63 at layer 3 of loop 3, then falls to 0.42 at layer 4 of loop 5. Over the same interval, the next entity's score rises from 0.003 to 0.40 but remains lower at every measured depth. This is a description of the selected bridge transition (Appendix D.1, Figure 14).

For the fixed-versus-dynamic multi-hop comparison, we now make the supervision difference explicit. Dynamic training applies the loss only at the sampled final recurrence of each batch; across batches, however, it supervises answers at several depths. This leaves precisely the objective-based explanation raised by the reviewer: earlier answers need not reflect an inductive bias beyond supervision at earlier depths. The released checkpoints also differ in curriculum range and training duration, now documented in Section 3.1 and Appendix D.2, Table 2. We properly rephrased thge corresponding claims.

We have added three independently trained seeds per regime using the same source code, data and curriculum, with seeds varying initialization and data order. We compare the checkpoints at the end of the same 13-hop curriculum stage (Section 3.2, Figure 7; Appendix D.2). The metric is the first loop at which endpoint accuracy reaches 0.8. For 2--6 hops, dynamic models reach this threshold at the same loop or one earlier; for 7--10 hops, they generally reach it at the same loop. For 11 and 13 hops, they reach it one loop later, and for 12 hops, at the same loop or one later. At 13 hops, all three dynamic seeds require seven loops, compared with six for the fixed models. Thus, the matched-curriculum comparison does not show a consistent timing advantage for dynamic training.

Q1

Yes. The new per-layer lens analysis shows factual readability developing within a single application of the 11-layer block (Section 2.2, "Single-pass comparisons" paragraph; Appendix B, Figure 11). In these parameter-matched single-loop models, the mean readable bits across the five attributes first becomes positive at layers 3, 5, 7, 8 and 8 for 20K, 50K, 100K, 200K and 500K individuals, respectively.

At 20K, 50K and 100K, the lens reaches half of its final score at 65%, 83% and 92% of the block's depth. The corresponding fractions of unrolled depth are 25%, 40% and 65% for four-loop models, and 23%, 42% and 61% for eight-loop models. Thus, the dependence of readout timing on memorization load is not specific to recurrence. We distinguish reaching a fraction of a model's final score from recovering all the information in the data: at 500K, for example, the final score is 4.06 bits, compared with a source entropy of 9.32 bits.

Figure 11 also reports the controls. Readable bits on never-trained C remain below the uniform-value log-loss baseline, and native readouts become positive later than lens readouts at every load. Constraining the lens to an identity map plus an update of rank 16, 32 or 64 changes the interpolated depth at which it first reaches 95% of its final score by at most 0.23 layers.

Q2

The split is by individual, not by paraphrase. A, B and C contain 60%, 35% and 5% of the individuals, respectively, with no names shared between them. The lens is fitted on 95% of A and validated on the remaining 5%; B is used for neither fitting nor model selection. Both A and B are included in backbone training, whereas C is unseen by the backbone. The reported A curves include both the fitting and validation individuals (Section 2.1; Appendix C).

Figure 2 and Appendix C, Table 1 compare A and B across five seeds. The table below gives the largest A--B gap over depths and seeds, together with the first-loop readable bits on B. The latter are reported as mean ±\pm 1 standard deviation across seeds.

IndividualsLargest A--B gap, four loops (bits)Largest A–B gap, eight loops (bits)First-loop B score, four loops (bits)First-loop B score, eight loops (bits)
20K0.690.705.67±0.765.67\pm0.762.32±0.382.32\pm0.38
50K0.390.381.21±0.531.21\pm0.53−0.09±0.11-0.09\pm0.11
100K0.210.21−0.11±0.11-0.11\pm0.11−0.56±0.07-0.56\pm0.07

The A--B gaps are below 0.71 bits at every measured depth and at most 0.017 bits at the final depth. The ordering by load also persists on B. At 20K, 50K and 100K, respectively, the mean depths at which the four-loop models first reach 95% of their final score are 1.92, 2.92 and 3.75 loops on A, compared with 1.99, 2.98 and 3.77 on B. For eight-loop models, the corresponding depths are 3.53, 5.28 and 6.65 on A, and 3.86, 5.53 and 6.73 on B. Thus, the timing pattern is not confined to individuals used to fit the lens. Scores on never-trained C remain below the uniform-value log-loss baseline (Appendix C).

Reviewer_9xuZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates when stored factual knowledge becomes readable across recurrent computation in looped Transformers. In synthetic single-hop retrieval, it measures depth-dependent performance using native and trained readouts, and reports that greater memorization load delays retrieval despite similar final performance. Representational analyses characterize this progression as a shift from query-key organization toward value organization. In synthetic multi-hop tasks, intermediate entities become decodable in logical order. Fixed- and dynamic-recurrence checkpoints exhibit different decoding trajectories, with the dynamic checkpoint reaching solutions earlier. Counterfactual patches at the decoded frontier reportedly redirect downstream entities and the model’s final answer. The paper interprets these results as evidence that recurrence progressively organizes stored knowledge into computationally useful intermediate states.

Critical strengths
  1. The study distinguishes multiple stages of information access. Comparing native readout, trained decoding, and representational geometry provides a richer account than final-answer accuracy alone. The load-dependent retrieval trajectories across three dataset sizes and two loop budgets are an interesting empirical finding. The unmemorized-person and block-preserving permutation controls strengthen the interpretation of the value-related geometry.

  2. The intervention experiments extend beyond probe correlations. Section 3.3 reports that frontier patches propagate to downstream entities and ultimately change native model outputs. Reported donor-implied endpoint accuracy is 79.07–98.57%, while different-context donors reaching the same frontier entity preserve the original answer at 99.57–99.87%. Subject to clarification of the intervention construction and evaluation population, these are consequential results supporting a functional role for intermediate states.

Critical weaknesses
  1. The entity-subspace intervention is not sufficiently specified to assess its selectivity. Section 3.3 defines the subspace as the row space of the entity classifier, but does not report its rank, any singular-value truncation, or the numerical rule used to construct the projector. This matters mathematically: if a classifier weight matrix has full column rank in a hidden space of dimension dd, its row-space projector is the identity. The intervention then becomes full-vector replacement, and an equal-dimensional random-subspace projector would also be the identity. This is a conditional concern, not a claim that the reported implementation necessarily has this degeneracy. The manuscript needs enough information to establish that the entity/random comparison is well defined and selective. It should also report retained-pair counts and selection coverage for the frontier-conditioned intervention results.

  2. The single-hop interpretation depends on what the trained lens learns. Lines 123–124 argue that freezing the output projection prevents the lens from learning a new representation-to-text mapping. This does not follow: the learned affine transformation and bias can change the effective decoder even with a frozen LM head. The paper states that lens training and evaluation use disjoint data splits, but does not clarify whether person identities are disjoint or whether different paraphrases of the same people occur across splits. Without this distinction and appropriate decoder controls, it is difficult to determine how much of the measured trajectory reflects the model's retrieval dynamics versus the accessibility learned by the diagnostic decoder. The untrained-person geometry control is useful, but does not directly resolve this lens-specific concern.

Questions for authors
  1. Intervention definition: What are the classifier dimensions and retained subspace ranks at each intervention point? Please specify the projector construction, any centering or singular-value threshold, and how the random control matches both dimension and perturbation norm. Which checkpoint is used for these interventions?

  2. Readout interpretation: Are the single-hop lens splits disjoint by person identity or only by prompt/paraphrase? Please provide the fitting objective and split sizes, and explain what controls distinguish decoding existing value information from learning additional key-to-value associations. Please also revise the claim that a frozen LM head prevents learning a new effective decoding map.

  3. Quantitative clarification: For Figures 4 and 7, please report the number of eligible and retained pairs, uncertainty estimates, and whether donor/base pairs with identical endpoints are excluded. Is cross-entropy converted from nats to bits before calculating M(t)M(t)? If Figure 1's unit label is only a caption error, please confirm.

4 comments

We thank the reviewer for their thorough analysis of our paper and the feedbaclks.

W1

In the new Appendix D in the revised version of the paper, we report more details concerning the projector construction. As now specify in Appendix D.3, we extract the classifier's 200 entity-class weight vectors, each with 768 coordinates, and subtract their mean vector. We compute the singular value decomposition of this centered matrix and retain right singular vectors whose singular values exceed 10−510^{-5} times the largest. With these orthonormal vectors as columns of VV, the projector is P=VV⊤P=VV^\top. Centering limits the subspace to at most 199 independent directions. The intervention therefore replaces only the component in this subspace of the 768-dimensional residual state, leaving its orthogonal component unchanged. When the selected frontier is the starting entity (frontier 0), the same construction uses entity embeddings instead of classifier weights.

The random control uses a subspace of the same dimension, obtained by QR-orthonormalizing Gaussian vectors. For each base--donor pair, we rescale the projected difference to match the entity intervention's Euclidean norm (Appendix D.3, "Random control"). Neither projector is the identity. Table 3 compares the entity, random-subspace and full-vector interventions using final counterfactual-answer accuracy, with 95% relation-group bootstrap confidence intervals.

Table 3 also reports selection coverage and retained-pair counts. A query is eligible when its frontier equals the most frequent frontier on validation data. At recurrences 1–5, respectively, eligible queries comprise 100.0%, 96.8%, 91.5%, 51.4% and 67.0% of 14,400 test queries pooled across three evaluation seeds. These seeds vary queries and donors.

W2

We agree: a frozen LM head does not prevent learning a new effective decoder. Section 2.1 of the revised manuscript, paragrah "Reading intermediate depths", now makes this explicit. The lens measures readability through a fitted affine map followed by the frozen terminal LayerNorm and output head, not what the native head already exposes.

Person-disjoint controls. The backbone trains on A∪BA\cup B; the lens fits on 95% of A and selects its learning rate and epoch on the remaining 5%. B is held out from lens fitting, not backbone training; C is unseen by the backbone. No person occurs in multiple groups (Appendix C of the revised paper).

The new controls in Section 2.2, Fig. 2, and Appendix C, Table 1, cover five seeds per entry (30 models). For the five-attribute mean, A exceeds B by less than 0.71 bits at every depth and at most 0.017 bits at the final depth; the ordering of first-95%-crossing depths by load is unchanged. C remains below the uniform-value log-loss baseline for every attribute and depth. The native readout also shows load-dependent timing (Fig. 1), and the qualitative delay persists with rank-16/32/64 corrections to the identity map (Appendix B, Fig. 10).

These results support transfer beyond lens-fitting individuals and show that the qualitative trend persists across the tested readouts.

Q1

Appendix D.3 of the revised paper gives the full construction; see also W1 above for a summary. The classifiers map 768 normalized features to 217 logits. Their 200 centered entity rows yield rank-199 projectors using a 10−510^{-5} relative singular-value threshold; the recurrence-1 projector instead uses entity embeddings. The Gaussian/QR control matches rank and each patch's norm.

Q2

The split is by person, not paraphrase. Appendix C gives A/B/C sizes of 12,000/7,000/1,000 at 20K, 30,000/17,500/2,500 at 50K and 60,000/35,000/5,000 at 100K. The backbone sees A∪BA\cup B; the lens fits only on A.

Initially, at each depth, Wt=IW_t=I and bt=0b_t=0. We minimize gt attribute-token cross-entropy through the frozen backbone, terminal LayerNorm and embedding-tied head, using one rendered biography per person. We fit on 95% of A and use the other 5% to select learning rate and epoch independently per depth: AdamW, 30 epochs, batches of 1,024 cached scored-token rows, learning rates {3⋅10−4,10−3,3⋅10−3,10−2}\{3\cdot10^{-4},10^{-3},3\cdot10^{-3},10^{-2}\}, and 10% linear warmup followed by cosine decay. Neither B nor C is used for fitting or selection (Appendix C, "Fitting and scoring").

The A/B/C comparisons, native readout and constrained-update controls are reported in Fig. 2, Table 1 and Appendix B, Fig. 10.

Q3

Figures 4 and 7 in the submitted manuscript are now Figures 8 and 16; both analyses use the same retained base--donor pairs. Appendix D.3 and Table 3 now report the selection procedure, coverage and sample counts. Of the 14,400 test queries across three evaluation seeds, 100.0%, 96.8%, 91.5%, 51.4% and 67.0% satisfy the frontier-selection criterion at recurrences 1--5, respectively. We retain 2,304 pairs at each recurrence except recurrence 4, where 2,286 are available. Recurrence 6 is reported separately because it contributes only 767 pairs from one evaluation seed. These seeds vary the test queries and donor selection, not the model weights.

For recurrences 1--5, we first average within each relation group and then give equal weight to the resulting 72 group means across seeds. We obtain 95% percentile confidence intervals by bootstrapping these group means. For the descendant-readout results, interval half-widths are at most 0.73 bits for descendants one or two hops beyond the frontier, 1.03 bits for descendants three hops beyond it, and 3.7 percentage points for the top-1 rate (Appendix D.3). Table 3 reports the confidence intervals for final counterfactual-answer accuracy.

The pair-selection procedure does not explicitly exclude identical endpoints. However, inspection of the retained pairs confirmed that base and donor differ in both their frontier entity and their final answer. This statement concerns the primary intervention pairs, not the auxiliary same-entity controls.

Yes, cross-entropy is converted from nats to bits by dividing by ln⁡2\ln 2 before computing M(t)M(t); the entropy baseline is also expressed in bits (Section 2.1 and Appendix C). The original Figure 1 unit label was therefore a caption error, not an error in the calculation. The revised caption reads "Readable bits M(t)M(t)”; the reported values are unchanged.

References

  • [1] H. Kohli, S. Parthasarathy, H. Sun, Y. Yao. Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers. arXiv:2604.07822, 2026.

We would like to thank the reviewer once more for the constructive suggestions.

We hope that the rebuttal and the revised version of the manuscript address reviewer's concerns.

We are more than happy to further discuss with the reviewer if any concerns remain.

Thank you

Reviewer_9xuZOpenReview ↗

Official Comment by Reviewer 9xuZ

Thank you for the detailed response. The subspace specification and person-disjoint lens controls address my original methodological concerns, and I appreciate the additional experiments and more carefully scoped claims.

My remaining reservation concerns the revised contribution's explanatory scope: the single-hop findings characterize training-dependent readability, while the multi-hop causal evidence remains limited to one checkpoint. These are useful observations, but I do not find their combined contribution sufficiently substantial to change my recommendation. So I decide to maintain my rating.

Reviewer_WKvKOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the point at which intermediate computations are accessible inside of looped transformers that solve multi-hop reasoning tasks. Intermediate points of computation are probed with linear probes and representation similarity analyses that measure the attributes by associated to clustering in activation space.

Critical strengths

The paper analysis is sound and the premise makes sense. Given past results on how looped transformers can do multi-hop computation, this paper asks: is this actually what happens inside of looped transformers, and probes the internal computation.

Critical weaknesses

-- The experiments probing looped transformers on multi-hop reasoning tasks are effectively already present in "Inside the Loop: A Mechanistic Study of Weight-Tied Transformers on Depth-Bound Algorithmic Tasks", which has been accepted to NeurIPS 2026.

-- It's unclear what extra insight is derived from this paper beyond what was already known about looped transformers and multi-hop in e.g. Saunshi et al. 2025. As such, I think that the result is too incremental to merit publication.

  • For instance, the authors write in their abstract that "More broadly, these findings suggest that recurrence organizes when and how stored knowledge can be used, rather than increasing storage capacity, motivating training and early-exit strategies that account for the progressive availability of knowledge across recurrences". However, early-exit strategies such as mixtures-of-depths and mixture-of-recursions are already well explored in the literature, and it is clear that storage capacity is not increased because the number of parameters doesn't go up (in fact this is the first sentence in the paper).
Questions for authors

What actionable suggestions for changing architectures does this analysis suggest.

3 comments

We would like to thank the reviewer for their feedback.

W1:
Thank you for highlighting this closely related work. We would first like to draw their attention to the timing relative to the ICLR submission deadline.

Chronology. The arXiv submission history dates the first version of Sesodia et al. [1] to 16 October 2026, after the ICLR 2027 full-paper deadline of 25 September 2026. The NeurIPS 2026 author-notification date was 24 September 2026, just one day before that deadline. Given this timing, we were not aware of this work when we submitted our manuscript. We therefore ask that the overlap be assessed in the context of concurrent work.

We now cite Sesodia et al. [1] in Sec.1, 3.2 and 4. Both studies decode intermediate entities in chain order and use activation patching. Their untied comparison also finds late intermediate-node readout, consistent with our untied single-hop results (Sec. 2.2, Fig. 3). The multi-hop tasks differ in how graph connectivity is provided. Sesodia et al. supply the graph as an explicit list of source–destination node pairs in each prompt, together with a starting node and a hop count. In our multi-hop task, the query contains only a starting entity and an ordered sequence of relation labels; the corresponding destination entities are not supplied. The entity–relation–destination mappings must instead be learned during training (Sec. 3.1, Fig. 5). This distinction directly connects our work to a specific open question in their Limitations section: whether the staircase survives on tasks that mix retrieval with computation. Our results answer this question in the tested stored-fact setting: intermediate entities become decodable in chain order, and interventions on their representations redirect subsequent computation (Sect. 3.2–3.3). The revised Section 4 makes this connection explicit. Separately, our single-hop experiments measure how factual readability changes with memorization load (Sec. 2.1–2.3, Figs. 1–4).

Our causal tests ask whether changing the retrieved entity changes the subsequent answer. We replace only the component of the hidden state selected by an entity classifier. Compared with a random-subspace replacement matched in dimension and perturbation magnitude, this increases accuracy on the answer implied by the donor entity by 67.5–81.6 percentage points at recurrences 2–5. When the donor instead reaches the same entity through a different relation prefix, the original answer is preserved in 99.57–99.87% of cases. Together, these controls test whether the intervention changes the answer through the retrieved entity, rather than merely disrupting the hidden state (Sec. 3.3; App. D.3 for construction and the same-entity control; Table 3 for counterfactual results).

The revised paper also tests the single-hop findings with untied models (Sec. 2.2, Fig. 3; App. E.2, Table 4), decoder controls (Fig. 2; App. B–C), and every-loop supervision with depth-specific readouts (App. C.1, Figs. 12–13, including teacher-forced exit). These are controls on our interpretation, not claims of new supervision or early-exit methods.

W2a We clarified the distinction in Sec.4. Saunshi et al. [2] compare final-output performance across looped and non-looped models, study in-context reasoning tasks and memorization benchmarks, and show that looped transformers can simulate CoT steps. We build on those results.

Our question is when stored facts become readable within a frozen model's forward pass, and whether the resulting entity representations support later composition. The revised paper provides:

  • A load-dependent readability schedule. At 20K/50K/100K, the first 95% crossing occurs at 0.50/0.74/0.94 of the four-loop budget and 0.48/0.69/0.84 of the eight-loop budget, while final five-attribute means remain 8.93 bits (Sec. 2.2, Fig. 1; App. C, Table 1). The score is source-referenced log-loss reduction, not an unconditional mutual-information bound (Sec. 2.1).
  • Tests of that interpretation. Per-layer single-block and equal-depth untied comparisons show that depth-dependent readability is not unique to recurrence (Sec. 2.2–2.3, Figs. 2–4; Appe. B–C and E). The combined every-loop supervision/readout experiment makes near-final first-loop readability possible in all eight tested cells (App. C.1, Fig. 12).
  • Causal tests on stored-fact chains. Entity-component patches redirect subsequent answers much more than rank- and norm-matched random patches; same-entity, different-prefix controls largely preserve the answer (Sec. 3.3; App. D.3).

These are depth-resolved measurements and controlled interventions, not a new claim that looping improves reasoning or storage capacity.

W2b We agree that early exit is established work, not a contribution of this paper and Section 4 now discusses existing adaptive-computation methods and states that we propose no new mechanism.

Capacity. Ouro [3] motivates our capacity check (App. A, Fig. 9). Equal parameter counts alone do not establish equal measured capacity. We do not claim any novelty there.

What the new experiment adds. App. C.1, Fig. 13 tests a native-confidence threshold under teacher forcing, not a new early-exit method or a reproduction of CALM's [4] confidence statistic. At 20K/50K, the smallest tested threshold within 0.5 percentage points of full-depth accuracy uses 3.76–3.97 of 4 loops or 6.90–7.56 of 8. At 100K none of the tested thresholds qualifies: best accuracy is 0.786 versus 0.795 at full depth for loop-4, and 0.739 versus 0.795 for loop-8. These results qualify, rather than assume, the usefulness of confidence-based exit in our setting.

Q1 Our evidence supports a training/readout choice, not a new architecture. The revised Section 2 takeaway and Appendix C.1 (Figs. 12–13) give the actionable result: when early factual readout matters, every-loop loss together with depth-specific LayerNorm/readouts can bring near-final readability to the first loop. Across eight tested cells, the model's own first-loop score is 8.89–8.90 bits, within 0.02 bits of its final score. Supervision and readouts change together, so this does not identify which change is responsible.

The inference lesson is to validate early-exit accuracy directly rather than infer it from readable bits or native confidence. In the 100K every-loop twins, fixed first-loop exit loses approximately 0.078 and 0.502 percentage points of full-depth accuracy for loop-4 and loop-8; at 100K, the final-loop-only models do not meet the 0.5-point criterion at any tested confidence threshold (Appendix C.1, Fig. 13). These are teacher-forced measurements, including gold intermediate tokens for dates, not a validated free-generation exit policy. We do not extrapolate this recommendation to multi-hop training.

References

  • [1] Sesodia M, Sock J, Torr P, de Witt CS. Inside the Loop: A Mechanistic Study of Weight-Tied Transformers on Depth-Bound Algorithmic Tasks. arXiv preprint arXiv:2610.24135. 2026 Oct 16
  • [2] Saunshi N, Dikkala N, Li Z, Kumar S, J Reddi S. Reasoning with latent thoughts: On the power of looped transformers. In International Conference on Learning Representations 2025 May 1 (Vol. 2025, pp. 14855-14881).
  • [3] Zhu RJ, Wang Z, Hua K, Zhang T, Li Z, Que H, Wei B, Wen Z, Yin F, Xing H, Li L. Scaling latent reasoning via looped language models. arXiv preprint arXiv:2510.25741. 2025 Oct 29.
  • [4] Schuster T, Fisch A, Gupta J, Dehghani M, Bahri D, Tran V, Tay Y, Metzler D. Confident adaptive language modeling. Advances in Neural Information Processing Systems. 2022 Dec 6;35:17456-72.

We would like to thank the reviewer once again for their feedback.

We hope that the revised version of the manuscript addresses the reviewer’s concerns.

We now explicitly cite and highlight the overlaps and differences between our work and both Sesodia et al. [1] and Saunshi et al. [2].

We would also like to respectfully emphasize once again that the arXiv submission history dates the first version of Sesodia et al. [1] to 16 October 2026, after the ICLR 2027 full-paper deadline of 25 September 2026. The NeurIPS 2026 author-notification date was 24 September 2026, just one day before the ICLR deadline. Given this timing, we were not aware of this work when we submitted our manuscript. Therefore, according to the ICLR reviewer FAQ, we kindly ask that the overlap be assessed in the context of concurrent work.

References

  • [1] Sesodia, M., Sock, J., Torr, P., and de Witt, C. S. Inside the Loop: A Mechanistic Study of Weight-Tied Transformers on Depth-Bound Algorithmic Tasks. arXiv preprint arXiv:2610.24135, 16 October 2026.
  • [2] Saunshi, N., Dikkala, N., Li, Z., Kumar, S., and Reddi, S. J. Reasoning with Latent Thoughts: On the Power of Looped Transformers. In International Conference on Learning Representations, 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.