acceptodds
Under review as a conference paper at ICLR 2027

Depth Is Not Enough: Compositional Generalization In Looped Transformers

13 views

Abstract

Compositional generalization requires reusing familiar operations in new combinations. For computations whose dependencies form a Directed Acyclic Graph (DAG), this involves both locating each operation's operands (syntax inference) and computing with them (syntax execution). Looped Transformers have demonstrated impressive generalization abilities by repeatedly applying a shared block, but it is unclear to what tasks they can generalize and why. In this work we introduce a task ladder, from prefix scans to serialized arithmetic trees, that progressively increases the difficulty of syntax inference. Models are trained on small compositions and tested on larger ones. The reference computation of every task is fully known, but models must infer its syntax from the input and then execute it. This lets us supply the true operands to a trained model and test the cause of a compositional failure. On this ladder, looped models fit every task in the training range but extrapolate only when each operand lies at a fixed offset from its operation. Larger models, other positional encodings, chain-of-thought and a pretrained 3.5B looped model do not close this gap. We establish syntax inference as the cause of this generalization failure through test time interventions: restricting a frozen model's attention to the true operand positions substantially improves extrapolation. Adding iterations, by contrast, does not help: beyond a few iterations, accuracy stops improving. Computational depth is therefore not enough: looped models must also learn to locate operands in a way that holds beyond the training range. Our results highlight limitations of current looped models and improve our understanding of what enables compositional generalization.

Primary area:
foundation or frontier models, including LLMs
Keywords:
compositional generalisation, looped transformers, recurrent computation, neural algorithmic reasoning, operand binding

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_n4ewOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates the causes of failure to achieve compositional generalization in Looped Transformers. Models are trained of small (short) instances of a task and asked to perform longer instances. For the chosen tasks the reference computational algorithm is known. On longer instances the model must infer the syntax of the instance and compute the answer. The authors investigate possible causes for failures and find that syntax inference is implicated.

Critical strengths

The setup is very good, with a ladder tasks (some related) that make operand location progressively more difficult. The novel experimental setup helps separate syntax inference from syntax execution. This is conceptually valuable. The finding that training models with oracle-provided operand locations helps them generalize (and the converse) is good. Showing that introducing oracle-provided attention masks at inference time can increase model accuracy is strong experimental evidence. Ditto brief adaption training, and top k restriction. The use of left caterpillar, right caterpillar and complete tree tasks adds detail to the picture. Compare to many in distribution / OOD papers this is a detailed and nuanced exploration. Well done.

Critical weaknesses

The paper assumes that the reference DAG is the algorithm that the model learns. Without showing the algorithm the model learnt, this remains an assumption. Other algorithm-variants are mathematically feasible.

The model has at least these capabilities: A) Identify the operands by determining the correct dependency positions B) Retrieve their values (sometimes via intermediate representations), and C) execute the calculation. The paper groups A and B together. I'd like to see them named/investigated separately, via a few more experiments.

The oracle mask intervention is good evidence, but may hide other mechanisms. Maybe the model's attention distribution is too diffuse to retrieve the intermediate representations reliably.

The abstract's "We establish syntax inference as the cause of this generalization failure" seems overstated.

With additional experiments that directly measure binding representations and intermediate execution, this could be a very useful framework for mechanistic investigation of how models learn and fail to generalize compositional algorithms.

Questions for authors

Are more experiments planned? I would like to see:

  • Directly test whether the models infer the correct operand identities by training an explicit probe on the models' intermediate representations.
  • Measure execution errors separately from retrieval errors.
3 comments

Thank you for the encouraging review. You suggested that, with probes and separate execution measurements, this could become a useful framework for mechanistic investigation. We ran exactly these, in a new §4.3 and Appendix E.

W1. The reference DAG is assumed. New perturbation tests (L.463–466, App. B.2, Fig. E.3). If the model computes as in Eq. (1), each operator's output depends only on its operands' outputs. We test this three ways: resampling the leaves under an operand, swapping the operand's state, and resampling leaves outside the subtree. In the training range all three hold on ragged trees: at least 92% of tested operators respond as Eq. (1) predicts. So its computation is consistent with the reference DAG there, which makes the decomposition informative. At depth 8, the three properties still hold on ragged trees (86–97%) but fail on complete trees (45–59%). That is where operators stop reading their operands, the identification failure (A, see W2).

W2 and Q2. Separating A, B and C. New §4.3 (method in App. B.2). We first set aside errors propagated from a wrong operand. Each other wrong operator gets one label: • (A) identification, tested by its attention on its operands. • (B) retrieval, tested by a linear probe on its state. • (C) execution, otherwise. For the base model on complete trees at depth 8, 84% of operators are wrong. Almost all are classified as (A): 78% of all operators, against under 1% for (B) and (C) (Fig. 6, E.1). This also measures execution separately from retrieval (Q2).

W3. Diffuse attention. Sharpening attention does not solve the problem. On complete trees at depth 8, the left-operand head of wrong operators places 0.94 of its attention on tokens other than the operator and its operands (Fig. E.2). Sharpening it to the model's own top positions (top-k, Fig. 4) does not help. On scans you are partly right. A training-length window repairs them as well as the true mask, because a scan's operand is always recent. The same window does not help trees (L.367–369, Fig. E.8).

W4. Overstated abstract. We agree, and rewrote it to state the claim by task and depth (L.24–28).

Q1. Probe for operand identities. New probe (L.470–472, Fig. E.4). A linear probe reads where the left operand lies from the operator's state. It succeeds for 87% of correct operators but only 54% of wrong ones (ragged trees, depth 8).

Thank you again. Your suggestions made the analysis much more precise.

Thank you for your detailed engagement with the reviewers. As a result of your efforts I'm sure your paper is stronger now. Best wishes

Thank you again for your kind words and constructive feedback. Your suggestions directly motivated the new analyses: the separate measurement of identification, retrieval and execution errors (§4.3, L.427–476), the operand-identity probe (L.470–472, Fig. E.4), and the abstract restated by task and depth (L.24–28). We believe they have strengthened the paper.

If you feel that our revisions have further strengthened the case for acceptance of our paper, we would be very grateful if you would consider updating your score to reflect this.

Reviewer_KAn4OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper separates two things a looped Transformer must do to evaluate a composition: locate each operation’s operands (syntax inference) and compute with them (syntax execution). A task ladder from prefix scans to postfix arithmetic trees progressively increases the demands on operand location. Models are trained on short compositions and tested on longer ones. On trained checkpoints, the authors insert the true operand mask, restrict attention to the model’s own top-k positions, and sweep the iteration budget. Supplying operands, followed by brief adaptation, nearly repairs B1 and B2. On trees, recovery shrinks with depth. Extra iterations do not restore extrapolation in the failing checkpoints.

Critical strengths
  1. The ladder provides a useful design for varying how operands must be found, and the known reference DAG makes the mask intervention possible.

  2. The scan result is convincing. The frozen mask takes B1 at depth 32 from 0.21 to 0.95 while top-k with the same number of positions stays at 0.21. The base model’s accuracy stops improving after about 16 iterations. Together, these support operand access as a major bottleneck on B1.

Critical weaknesses
  1. The causal claim is broader than the intervention supports. On right caterpillars at depth 8, supplying operands raises accuracy from 0.14 to 0.20, or 0.31 after adaptation. The remaining errors could reflect execution limitations or incompatibility with the inserted mask. Section 4.1 acknowledges this uncertainty, but the abstract and Contribution 3 attribute failure more broadly to syntax inference. The claim should be qualified by task and depth.

  2. Adaptation lacks a matched baseline. The adapted condition combines supplied operands with 50 additional training steps. The same adaptation without the mask would clarify how much recovery depends on supplied structure under the same optimization procedure. This concern does not undermine the frozen-mask results.

Questions for authors
  1. Can the abstract and Contribution 3 state the causal claim by task and depth?

  2. How does 50-step adaptation without the mask perform, with the same data, optimizer, and full-trajectory gradients?

  3. On right caterpillars at depth 8, how is the masked model’s attention distributed between self and operands, and how sensitive are predictions to perturbing operand states?

2 comments

Thank you for the precise review and for finding the scan result convincing. We agree with both weaknesses: the claim needed qualifying by task and depth, and adaptation needed a matched baseline.

W1 and Q1. Claims by task and depth. The abstract (L.24–28) and Contribution 3 (L.103–104) now state the claim by task and depth, with operand location as "a main cause". §4.1 (L.374–377) gives the split by tree family and depth (base → frozen → adapted). Right caterpillars go from 0.14 → 0.47 → 0.81 at depth 5, but only 0.14 → 0.20 → 0.31 at depth 8. Complete trees recover at depth 5 (0.85) but not at depth 8 (0.12).

W2 and Q2. Adaptation without the mask. We ran the requested 50-step control, with the same data, optimiser and full-trajectory gradients, and extended all conditions to 5,000 steps (§4.1 L.407–412, Fig. E.5, E.6). On depth-8 ragged trees after 50 steps, accuracy is 0.15 without the mask, 0.11 with a wrong mask and 0.48 with the true mask. After 5,000 steps, it is 0.15, 0.12 and 0.66. The gain comes from the supplied operand positions.

Q3. Right caterpillars, depth 8, with the mask. New analyses (§4.3 L.473–476). • Attention (Fig. E.2e). Under the mask, the left-operand head gives the operator itself, its left operand and its right operand 0.18 / 0.51 / 0.30 of the attention when the operator is correct, and 0.24 / 0.47 / 0.29 when it is wrong. Without the mask, wrong operators put only 0.03 on their left operand (Fig. E.2c). • Sensitivity to operand states (Fig. E.3, middle row). When we replace the left operand's state by its state on another input, the output switches to the new value in 84% of cases, against 56% without the mask and 78% for the trained oracle. With the mask, predictions do depend on the operand states. • Error causes (Fig. E.1k). The remaining errors are rarely execution errors: 4% of all operators (error C), out of 44% that are wrong. Most come from the learned computation not yet using the supplied operands: 17% of operators place too little attention on an operand (error A), and 16% do not encode its value decodably (error B). This matches the incompatibility you suggested, and adaptation removes most of it: accuracy rises to 0.88 after 1,000 steps (Fig. E.5).

Thank you again. We are happy to discuss further.

Thank you again for your review. As the discussion period comes to an end, we would like to check whether our revisions have addressed your concerns. To summarise our previous response:

  • Scope of the claim. The abstract (L.24–28), Contribution 3 (L.100–104) and §4.1 (L.374–377) now state it by task and depth.
  • Matched adaptation baseline. Adapting without the mask or with a wrong mask, under the same data and optimiser, leaves accuracy unchanged. The true mask, with adaptation, raises it (§4.1 L.407–412, Fig. E.5, E.6).
  • Attention and sensitivity under the mask (Q3). Answered in §4.3 (L.473–476) and Fig. E.2, E.3.

We also added an error decomposition (§4.3), which traces each wrong operation to the step that failed (locating, retrieving or computing). Beyond the training range, task C (tree) errors arise in locating operands rather than in computing with them, giving independent support for the main claim.

If our response addresses your previous concerns, we would be grateful if you would consider updating your assessment. Otherwise, we are happy to answer any remaining questions.

Reviewer_9LWkOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

In this paper, the authors propose a synthetic experimental setup to understand the failure in compositional generalisation in looped models. To separate the failure modes of syntax inference and syntax execution, a “ladder” of tasks is build, each introducing a new requirement, making extrapolation more difficult step by step and allowing different causes of failure to be investigated. The results indicate that the models struggle with syntax inference when testing on OOD tasks. While an oracle, also trained on the same distribution but with masked attention such that the operation can only attend to the necessary operands, extrapolates well, all other models fail once the operands' relative location becomes more complex. Introducing the masking post-hoc does increase OOD performance on the looped model, but cannot close the gap.

Critical strengths

Clear hypothesis phrasing: A clear, testable hypothesis is formulated and tested. While this is standard in many research fields, I feel that it is often missing in our community and is therefore much appreciated.

Strong experimental setup and execution: This aligns with the first point. The experimental setup is chosen carefully to minimise potential confounders. The resulting execution, as well as the chosen baselines, are well designed and provide strong support for the main claims.

Strong ablations: While the main text mostly focuses on investigating syntax inference and computational depth, the appendix includes several additional ablations and controls examining factors that can contribute to generalisation, such as model size and architectural choices

Critical weaknesses

Missing investigation of training length: While the contribution of model scale has been tested, I am missing a study on the amount of training data (the number of tokens seen). Increasing the training budget could potentially improve syntax inference on its own, and therefore I believe an ablation on this would strengthen the conclusions.

Some anomalies are not sufficiently discussed: The base model performs better on B2 than on B1. Furthermore, the convolutional model often outperforms the baseline even on tasks where its inductive bias does not give it an obvious advantage. Lastly, the underperformance of the base model on right caterpillars within the training range is only briefly discussed. I think these observations deserve further investigation, particularly to better understand whether they arise from the learned strategies or from issues with training.

Limited contribution: The fact that syntax inference can be an issue for neural networks is already well established. The contribution of this paper, in my understanding, is mostly to provide a framework for investigating this failure mode and to show that additional recurrent depth alone cannot solve it. While I appreciate this contribution, I would consider its overall novelty rather limited.

However, given the mostly clean execution I still believe, that the paper provides some valuable insights and therefore vote 3.

Questions for authors
  • Could the authors provide an additional study on the training budget, using a fixed model size but vary the token budget
  • Could the authors discuss the three anomalies mentioned in “Critical Weakness”.
  • I understand that training larger models is expensive, but using a CoT model of sizes 1.05M seems very limiting. Are there specific reasons why the authors have not chosen a larger one? Could they ideally provide results on a larger model?
  • The ablation (oracle, adapted) increases performance, but still falls far from the actual oracle. Could the authors clarify if this is due to the few update steps or the use of RoPE, which the oracle does not use. Ideally the authors would include a plot which shows performance of (oracle, adapted) vs number of update steps.
  • Minor: The color choice in plots is sometimes quite poor. Especially in Figure 4, where base, (oracle, frozen) and (oracle, adopted) can be hard to distinguish.
3 comments

Thank you for valuing the clear hypothesis and the controlled design. Each of your questions led to a new experiment or analysis.

W1 and Q1. Training budget. New experiment (§3, Fig. E.7). We trained the same model 4× and 16× longer. Data are generated online, so this also adds examples. It gains one depth on trees (ragged, depth 5: 0.49 → 0.63) but not more (depth 8: 0.15 → 0.19). It does not reliably help the scans. Longer training thus delays the failure but does not remove it (L.338–340).

W2 and Q2. Anomalies. Now discussed in §3 (L.298–318), each with a new analysis. • B2 above B1. This is consistent with B2's decoys making its training streams about 4× longer. Relative to the training length, both scans decline alike (Fig. E.8a). • Convolution above base. The convolution is more accurate at every operand distance (Fig. E.9a). Training 16× longer does not close the gap: the base model reaches 0.31, against 0.40 for the convolution. • Right caterpillars at depth 4. This is under-training. Training 4× longer fixes it (0.48 → 1.00), but not depth 5 (Fig. E.7, E.9b).

Q3. Larger chain-of-thought models. New experiment (L.330, Fig. E.10). We trained trace models up to 34M parameters, and a 4.2M model 16× longer. All fall beyond depth 4 like the looped model.

Q4. Few steps or RoPE? It depends on whether operand distances were seen in training (Fig. 1, last row), as a new adaptation sweep shows (§4.1 L.407–411, Fig. E.5). On right caterpillars, whose distances were seen, steps were the limit: 0.30 after 50 steps, 0.88 after 1,000. On complete trees, whose distances were not, more steps barely help (0.29 after 5,000). There, RoPE at unseen distances is the likely limit: a model trained with the mask and RoPE, without role labels, reaches 1.00 on right caterpillars but only 0.43 on complete trees (Fig. B.1). Our oracle avoids this by using role labels instead of RoPE, though that comparison changes both at once.

Q5. Colours. Fixed (Fig. 4).

W3. Limited contribution. We agree that locating operands is a known difficulty. What was missing is a way to test it as the cause of a failure. Studies of compositional failure rely only on empirical evaluation and report out-of-distribution error, which cannot separate wrong operands from missing computation (e.g. Dziri et al. 2023, Zhou et al. 2024, L.77–80). Looped models that do extrapolate usually have the dependencies built in, so they only test execution (L.72–77). Expressivity results say what added depth can compute in theory, not what actual training may find in practice (L.66–69).

Our framework fills this gap (L.81–84). The ladder keeps the operations fixed and makes operands progressively harder to locate. Because each task has a known reference DAG, we can supply the true operands to the same trained model, and trace each wrong operation to the step that failed. On looped Transformers, it shows that extra depth does not fix the failure while supplied operands largely do (§4.1, §4.2), and that errors arise in locating operands (§4.3).

Thank you again. We are happy to answer further questions.

Thank you for the detailed response and the additional analysis. These address most of my concerns, and I would support acceptance of the paper.

One question remains for me. The new experiments for the convolutional model do show its advantage, but I am unsure what explanation they support. Could you clarify the interpretation? I do not expect a complete mechanistic explanation, but if the reason for the advantage remains unresolved, I think it should be explicitly acknowledged.

Thank you for supporting acceptance. In short, the convolution keeps long operand distances that the base model loses, even distances seen in training. A new analysis, a quick evaluation of existing checkpoints that we added to the revised PDF (Fig. E.11), shows this by splitting Fig. E.9a by stack height. For operators that complete the whole expression read so far (stack height 1, Eq. 3), training contains every distance up to 16, and lookups at distance 12 are correct 0.99 of the time for the convolution against 0.53 ± 0.04 for the base model (five seeds).

The convolutional model both removes RoPE and adds a convolution. A NoPE model, which only removes RoPE, loses these distances too (0.63 ± 0.01), so the gain comes from the convolution itself. How it helps is not yet understood, and §3 now says so (L.301–302). This does not change our conclusion: the convolution locates seen operands more reliably, but like the base model it fails once the binding rule must extrapolate (Fig. 2).

Thank you again for your careful reading and your questions, which have improved the paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

GENERAL RESPONSE TO REVIEWERS

We thank the reviewers for their careful and constructive reviews. Changes are in blue in the revised PDF, including a new §4.3 and Appendix E. In each reply, W and Q number the weaknesses and questions of that review, in their order.

  1. New analysis: an error decomposition (§2.1, §4.3, Fig. 6, method in App. B.2). For every operation the model gets wrong, we identify which step failed: (i) locating its operands, (ii) retrieving their values, or (iii) computing. Errors inherited from a wrong operand are counted separately. Beyond the training range, most errors of the base model on trees are classified as (i). This gives a second, independent line of evidence for the paper's main claim (L.24–28). The interventions of §4.1 show that supplying operands repairs the scans and, one depth beyond training, most tree failures. The decomposition shows that, without them, errors arise in locating operands, not in computing.
  2. Text revised: claims are now stated by task and depth (abstract, Contribution 3, §4.1).
  3. New experiment: adaptation sweep of up to 5,000 steps, with no-mask and wrong-mask controls (§4.1, Fig. E.5, E.6).
  4. New controls: longer training and larger chain-of-thought models (Fig. E.7, E.10). New analyses explain three unexpected results: B2 above B1, the convolution above the base model, and right caterpillars failing at depth 4, inside the training range (§3, Fig. E.8, E.9).
  5. Text added: the contribution is stated explicitly (Introduction, Summary). Figure 4 is recoloured.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.