acceptodds
Withdrawn from ICLR 2027

Same Head, Different Calls? Auditing Functional Role Claims in Looped Transformers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Looped Transformers apply one block across calls within a forward pass. Its weights are shared, yet its recurrent states evolve: a changing intervention effect can therefore invite a claim that a head changes function. What does such a profile establish? We specify the recipient states, donor rule, readout horizon, and response construction, pairing each claim with a diagnostic comparison. In pretrained Ouro-2.6B, a selected head shifts the answer margin at a trained early exit by 0.72 logits, yet the effect vanishes after the next call. Across six behavior-selected pointer models, target-unrelated donors reproduce 46% of the matched patch contrast and show progress-aligned structure while promoting their own states. Matched donors additionally promote the intended clean state: contrast improvement and target promotion are distinct. In separate vector analyses, fitted phase gains depend on response coordinates, and the role-slot gain weakens when the clock represents the task boundary. Sparse within-clock phase variation and head-weighting sensitivity leave phase information beyond time unresolved. In a cued positive control, scalar checks retain a known terminal effect. The evidence supports transient influence and donor-conditioned target promotion, but does not identify functional role switching.

Primary area:
interpretability and explainable AI
Keywords:
Looped Transformers, Mechanistic Interpretability, Activation Patching, Functional Role Attribution, Recurrent Computation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_2XuCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper concerns interpretability of looped-transformers. More specifically, it studies the role and functionality of attention heads, mainly using activation-patching. In looped-transformers, the head can react differently to interventions across different iterations through the network. The authors then ask whether this implies that the attention head has different roles through these iterations. First, the authors observe that a patched head can have an immediate effect on the logits at the current call but have no effect on the final answer since later calls suppress it. Second, they then focus on small toy-models they train themselves and show that an unrelated random donor patching can still improve the clean-versus-corrupt score, mainly by suppressing the corrupted state, whereas matched donors promote the clean state. They then show that there are different ways to organize measurements of intervention effects across calls, with each making certain patterns easier or harder to observe. Finally, they ask whether knowing additional information such as whether the model has reached the final answer can help explain better the effect of the head.

Critical strengths
  • The general question of extending current mechanistic-interpretability methods to looped-transformers is interesting

  • The authors are very honest about their results and report a lot of negative results throughout the paper. They show that patching and intervention methods can lead to false interpretation when applied to looped-transformers, which can be useful for the community.

  • The toy-model setting is useful to understand the behavior of looped-transformers across iterations.

Critical weaknesses
  • The paper is hard to read and very dense because it introduces many terms before making the basic experimental design understandable.

  • The main experiment focuses on a single pre-trained model and studies the behavior on a single attention head. For the other experiments they train their own toy models on a small synthetic task. This limits the scope of the paper.

  • A significant part of the paper is dedicated to C2 and the fact that unrelated donor patching can still improve the clean-versus-corrupt score by suppressing the corrupted state. This is important to faithfully interpret activation-patching results but it is not specific to looped-transformers, which questions its presence in the main text.

  • The authors study specific role-switching interpretations of attention heads with a series of negative results showing that changing effects does not establish change in functions. This leaves the main question unresolved as to what can distinguish whether a head changes function or not in different states.

Questions for authors

Clarification:

  • Why is the observation about unrelated donor effects specific to looped-transformers ? Could it not be instead because of the task ?

Suggestion:

  • Why not considering other mechanistic interpretability methods in the context of looped-transformers ? The paper focuses on activation-patching for attention heads. Why attention heads only ? What about steering vectors, J-lens ?

  • What evidence could establish a change in function ?

Reviewer_m62DOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper seeks to answer a very important question about looped transformers - whether the same component functions that same when it is repeatedly applied. Looped transformers are a form of depthwise-recurrent networks, and it is possible that the same component performs different functions (as determined by their different effects on output logits) when applied to an evolving underlying state.

Critical strengths

The core question is interesting and important. No other identifiable strengths.

Critical weaknesses

This paper exemplifies the problem with AI generated academic work. The framing of the work, and indeed the core question that the paper purports to address, is a fascinating one, with a potentially rich array of analysis and experiments that probe how depthwise recurrence interacts with the transformer architecture, and may challenge how mechanistic interpretability identifies components with function. However, the paper is effectively unintelligible due to extensive LLM writing. Even if the experiments were meaningfully addressing this core question, the paper would still receive a poor recommendation, as it does not effectively communicate scientific content to a human reader. More troublingly, the paper is a bait-and-switch. The question at hand is about identifying component with function, but the analyses do not tackle anything that could reasonably be described as "functional". No effort is made at identifying the computations that are actually performed by an attention head, only at identifying the downstream effects of applying a component to a state. For example, the Ouro analysis merely identifies that a head appears important in an early recurrent block application degrades by the end of applications. This is has very little to do with "function". Similarly, Section 5 appears to be an analysis of the downstream patching metric (the difference of two logits) rather than an analysis of the "function" of an attention head.

This sort of conceptual slippage is academically and scientifically deleterious, and confusing to the community that it is attempting to engage with.

Questions for authors

No questions.

Reviewer_hFX8OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper applied activation patching methods in looped transformers to study whether the same head changes its function across different iterations. The paper studied four claims about how the causal effects and functional roles of the same head would change throughout the loop. The authors claimed that the causal effects of the same head may change after iterations but such changes would not testify the changes in the functional roles of these heads.

Critical strengths
  1. The paper applied the causal mediation analysis and activation patching methods in the looped transformer which is an under-explored topic in the mechanistic interpretability research.
  2. The paper studied both the open-source large-scale models and the simple baseline models.
  3. The paper showed the interesting phenomenon in a pointer-chasing task that the patching effects changed during different iterations and the model process step by step like traversal on the symbol sequence.
Critical weaknesses
  1. The paper is poorly written.

It is like an AI-generated summary of the project which introduced many invented obscure jargons without clear explanations rather than use commonly used terms in the mech interp research starting from the introduction throughout the whole paper. It is really hard to understand the main message without 'deciphering' the paper, which causes me hard to evaluate the actual contribution of the paper. It would be better if the authors could explain the main contributions of the paper in plain language during the rebuttal. And the paper needs to be rewritten in a way that the readers can follow easily.

  • Most invented terms lack clear explanations or definitions. Like "Horizon", "Donor", "Clock", "pointer models", "donor rules", "role-slot gain", "the recipient states", "readout horizon", "task boundary" these terms are introduced in the abstract/introduction and Figure 1. But no proper definition is provided.
  • The titles of each section from 3 to 6 are quite confusing.
  • The paper could be organized in a better way. The section 3 could start with the tasks studied, then introduce the models to be studied, and finally different patching methods and corresponding metrics.
  • It would better if the paper used some examples to explain the pointer-chasing tasks studied in the paper and more importantly the patching/intervention procedure in figures. Currently it is hard to know the nuances between two different patching settings.
  • The paper invented a different taxonomy for the standard procedure of activation patching/causal mediation analysis in Line 126-Line 139.
  • It would be better if the paper could elaborate on the four claims which the authors wanted to test.
  1. Some experimental designs are questionable.
  • Why would the paper only select one head in one layer for testing C1 claim? How did the authors select the head? Is the selection procedure different from the one
  • Could the authors explain the rationale behind adopting two different patching settings, i.e., patching from the clean prompt to a corrupted prompt of the same question or patching between different questions at the same depth and call?
  • Although the paper claimed that the changes in causal effects of the same head does not mean the switch of the function roles. But it is not quite clear to me how testing C1-C4 could provide evidence for this claims. C1-C4 just showed that the causal effects may be affected by many factors, e.g., the quantities you are looking at. It would be better if the authors could elaborate on how the relationships between C1-C4 claims and the hypothesis that the functional roles of the head change during the iteration.
Questions for authors
  1. What kinds of functional roles did you study for the attention heads? Do the functions mean different relative positions, like the last symbol, the current symbol, etc? what does this claim "A functional role claim names a task variable and readout boundary, not a unique internal algorithm." mean?
  2. What does "Peaks near completion" mean? What does "this is equivalent to the implementation’s explicit final-correctness guard. " mean? What does "These phases are post-hoc operational labels for analysis, not an assumed internal controller or a deployable phase detector." mean?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.