acceptodds
Under review as a conference paper at ICLR 2027

Computation Persists, Routing Degrades: A Mechanistic Analysis of Transformer Length Extrapolation

Abstract

Transformer models trained on algorithmic tasks often fail to extrapolate to sequence lengths beyond those observed during training. This behavioral degradation is frequently interpreted as a fundamental collapse in reasoning capacity, but we investigate whether this failure is mechanistically decomposable. Using dynamic programming (DP) tasks as case studies, we analyze the internal execution of RoPE-based Transformer model organisms. We find that length extrapolation failure is heterogeneous across learned mechanisms. Specifically, in the studied Longest Increasing Subsequence (LIS) models, internal representations associated with DP state construction remain substantially decodable beyond the training horizon, while downstream positional routing mechanisms degrade sharply. Crucially, through out-of-distribution (OOD) activation patching, we demonstrate that this unmodified OOD state representation remains functionally usable: correcting downstream routing alone recovers substantial behavior relative to the OOD baseline, and upstream restoration provides no additional benefit under this intervention. Furthermore, evaluating sequences at a fixed length reveals that dependency distance is a key correlate of representational degradation. In contrast, on the Longest Common Subsequence (LCS) task, where state construction itself requires global cross-sequence interactions, both intermediate representations degrade alongside behavioral performance under extrapolation. Layer-specific positional interventions and Randomized Positional Training further implicate positional geometry as a major constraint. Together, these results suggest that algorithmic extrapolation failure depends on the positional interaction requirements of individual learned mechanisms, rather than reflecting a uniform loss of algorithmic information.

Primary area:
causal reasoning
Keywords:
Mechanistic Interpretability, Length Extrapolation, Transformer Architectures, Algorithmic Reasoning, Positional Encodings, Out-of-Distribution Generalization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BxuWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work studies simple (2-layer) decoder-only transformers with RoPE learning a Longest Increasing Subsequence dynamic programming task, using interpretability tools like linear decoding and causal interventions to understand how the model fails on length generalization, with extensions to deeper and different transformer models and the Longest Common Subsequence task. The main findings include that the model learned to encode the DP state for the task in residual stream and this representation is causally linked to task performance, and that limited positional exposure especially hurts routing -- retrieving the right numbers from prior sequence.

Critical strengths
  • This work offers interesting insights on failures of length generalization on RoPE transformers on a few representative dynamic programming tasks, including what representations are learned/supports successful inference and what limits OOD performance.
  • This work extends and compares findings beyond a single-model/single-task setup, including deeper models/different architecture, as well as different tasks.
  • The causal experiments are informative at revealing the use of DP-state representation as well as how much OOD performance can be restored by surrogate positional representations.
Critical weaknesses
  • Overall, I think the work would be much stronger if it offered a more comprehensive account of how the model solves or fails at the LIS task. As it stands, the overall evidence seem to only partially account for the mechanism that the model learned or failed. E.g., the surrogate position interpolation intervention only minimally restored OOD performance. The work also only focuses the fixed-length training regime, potentially undermining positional generalization learnable during varying-sequence-length training.
  • Some findings don't appear very surprising. For example, it can be unsurprising that the L1 representations from the input tokens can degrade more since it has no output supervision and no incentive to unlearn information carried over from residual connections. It's also unsurprising that the model is weaker at further look-back jumps due to their sparsity in training sequences and the bounded length.
  • One thing that makes the overall story unclear is inconsistent framing. E.g., sometimes L0 representations are prescribed as retaining robustly preserving information about the DP state (4.1); but later referenced as severely degrading (4.2).
  • Some terms are under-defined, e.g., "geometric requirements".
Questions for authors
  • Have you considered training models on varying sequence lengths? I'm curious to hear your intuitions on how that might impact the OOD generalization patterns.
  • In 5.2, which sequences had restored accuracy (or which output tokens)? It'd be interesting to know if the position interpolation were helping to restore specifically one kind of mechanism, and what other failure exists from more behavioral failure analysis.
Reviewer_jks6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper explores the mechanism of limited length generalization for transformers on sythetic LIS and LCS tasks. It uses linear probing to show that the intermediate hidden states still holds certain DP-state infomation while the output accuracy drops on longer sequences beyond the training length. The paper argues that length extrapolation failure does not necessarily mean a loss of algorithmic ability. In the studied LIS models, intermediate computation remains partly usable, while position-related routing failures contribute to incorrect outputs.

Critical strengths
  1. The paper takes an interesting view of length generalization. It uses algorithmic tasks to separate intermediate state computation from downstream routing and study where failures may occur.

  2. It conducts lots of intervention experiments across model depths and tasks

Critical weaknesses

Throughout the paper, the authors’ explanation rests on the idea that the model computes DP states and uses them to choose which tokens to output. However, this process has not been clearly shown, even on in-distribution inputs. The two main experiments leave this connection unclear:

  1. The probe shows that DP labels can be read from hidden states. It does not show whether the rest of the model uses this information to choose its answers.
  2. The oracle directly gives the model the correct positions to read, so it may recover by copying the selected tokens.

The paper needs more direct evidence that the model computes and uses DP states in the proposed way. This would give a firmer basis for interpreting the probing and intervention results as evidence that computation remains usable while routing fails.

Questions for authors

Following the weakness mentione above:

  1. could the authors provide more direct evidence that the model uses DP states to choose its output tokens, starting with in-distribution inputs?

  2. Does the oracle recovery depend on these DP states? Could the model still recover if this information were removed while the input-token information remained available?

Minor: after the main results begin, the paper makes several claims about length-extrapolation failure with few supporting references. For example, sec4 describes a “prevailing assumption” that length-extrapolation failure reflects degraded algorithmic-state computation.

Reviewer_PsBoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates the problem of Transformers failing at extrapolation for long tasks. The authors hypothesize that the failure is a result of two compounding effects: the state representation in dynamic programming (DP) when computing the state in intermediate layers, and routing which describes the task performance on the final layers. Through linear probes and activation patching on two types of tasks (LIS: Longest Increasing Subsequence, LCS: Longest Common Subsequence) they show that intermediate representations remain decodable even if the output behavior collapses, and that routing degrades when the distribution of positional geometries changes. The proposed hypotheses are validated on GPT-2 and LLaMA-style architectures. As potential mitigation for out-of-distribution collapses, they propose randomized positional training.

Critical strengths

The finding that at 2x extrapolation the probe accuracy is 61% at intermediate layers despite 2% behavioral output, i.e. task performance, is very interesting. Moreover, comparing the influence of routing layers vs. compute layers supports their claims that routing layers are significantly affected by activation patching.

Critical weaknesses
  1. The causal decomposition raises some questions. It seems that the model used for activation patching runs with corrected positional geometry on all layers, not just layer 1. So when layer 1 activations are patched, they were done on top of a Layer 0 that also had corrected positions. Therefore, the patched activations carry upstream effects, not just routing corrections. Although the authors acknowledge this, it makes it hard to fully attribute the behavior gap between 14.5% and 81% (by the oracle) to routing alone.

  2. The presented conclusions are fairly limited in terms of scope by being only tested on two types of DP problems, which are also similar (LIS, LCS). Therefore, the results may be limited for this type of problems and the conclusions might change if the problem definition or model changes from GPT-2 to larger scale architectures as more commonly used nowadays.

Questions for authors
  • Can the layer interplay be distangled such that layer i activation patching does not inherit upstream changes but uncorrected earlier layer representations?
  • Can you try other tasks, e.g. sorting, to analyze the compute vs. routing decomposition also in other domains, even outside of DP specific settings?

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

Following Reviewer PsBo's suggestion, we have uploaded a revised PDF with all additions and edits highlighted in blue. None of the original submission's core measurements were altered; the highlighted sections correspond directly to the new reviewer-requested controls (erasure × oracle in Sec 5.1, variable-length training in Sec 6.2, non-DP sorting in Sec 7.2 / App G, and moving the preliminary patching table to App B). Thank you all for your time and feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.