acceptodds
Under review as a conference paper at ICLR 2027

When Do Neural States Realize the Same Causal Variable?

Abstract

Mechanistic explanations often describe neural states in terms of high-level variables. For such descriptions to function as causal abstractions, the distinctions they omit must be irrelevant to the downstream counterfactual behavior of interest. Yet causal adequacy is usually assessed at a particular neural state, leaving open whether the same description remains adequate as computation unfolds. We study this question in a controlled retrieval task where direct and relational queries reach the same selected entity through different computations. The relevant semantic intermediate is fixed by construction, while we independently vary how the entity is selected, which value it is assigned, and where its complete assignment record appears. We find three stages with different causal organization: entity identity is sufficient at an early direct-query state under our interventions, later selection states also depend on assignment-record arrangement, and later states are organized by answer identity. State-restoration interventions connect these stages causally: an effect established at selection is later reversible through a downstream context-sensitive state. Meanwhile, states reached through the direct and relational procedures become increasingly interchangeable even as the distinctions required for causal sufficiency become finer. Thus causal granularity changes across computation even though the semantic intermediate is held fixed: the same high-level variable can be causally adequate at one stage and too coarse at another. Mechanistic explanations must establish not only which high-level variable a neural state corresponds to, but also over which part of the computation that variable remains causally adequate.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, causal abstraction, causal equivalence, causal interventions, representation alignment, entity binding, relational reasoning, language models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_d6i1OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the assessment of causal adequacy. The authors employ a controlled environment and vary the entities to analyze this problem. The authors find three stages with different causal organizations. Finally, the authors claim that causal granularity changes across computations even though the semantic intermediate is held fixed.

Critical strengths

This paper has a relatively complete structure. The experiments are comprehensive.

Critical weaknesses
  1. The first apparent weakness is the logic of the abstract. I think some of the logic is not coherent. For example, in line 020, the authors claim that they find three stages. However, which three stages are not explicitly pointed out. Were they entity identity, later selection, and the later states? I would suggest that the authors refine their abstract.

  2. The second weakness is in the writing of the introduction section. Typically, the introduction should end with the list of contributions. However, the authors use a paragraph as the end of the introduction.

  3. The name of the section also has problems. For example, the second section is named "Background"; however, its content is more like "Related Work".

  4. Figure 2 is not well drawn. For example, the letter "g" of "entity-following" seems to be cut off.

Questions for authors

None, please refer to the weakness part.

Reviewer_cUbkOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates whether neural states associated with the same semantic variable remain causally equivalent across different stages of computation. Using controlled entity-retrieval tasks, the authors apply activation patching and counterfactual interventions to identify transitions between entity-level, context-sensitive, and answer-level representations.

Critical strengths

The paper addresses a meaningful question and presents a relatively systematic experimental design. However, the contribution mainly consists of applying established causal intervention techniques to observe layerwise changes in a controlled retrieval task.

Critical weaknesses

I find the methodological novelty limited. Causal abstraction, activation patching, and interchange interventions are established techniques. The paper mainly applies these methods to a controlled retrieval task and categorizes intervention outcomes into three computational stages. While this analysis has some value, it does not introduce a new causal abstraction methodology or a predictive theory explaining how these stages emerge. The contribution is primarily an empirical extension of existing frameworks.

The paper interprets layer-dependent patching outcomes as changes in causal abstraction granularity. However, replacing full residual states simultaneously modifies multiple latent factors, making it difficult to determine whether the observed effects originate from entity identity, record position, or other uncontrolled representational properties. In particular, sensitivity to record ordering does not directly establish the emergence of a new causal variable. The proposed three-stage mechanism remains largely based on predefined behavioral categories rather than a sufficiently identified internal computation.

Questions for authors

where are your contributions in sec 1?

Reviewer_WikCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines whether a single high-level causal variable (such as a selected entity's identity) remains an adequate description of a neural state as computation progresses through a language model. Using a controlled in-context retrieval task where direct and relational queries select the same target entity through different reasoning paths, the authors show that causal granularity is temporal and stage-dependent. Even when the semantic referent stays fixed, the neural representation shifts across layers from encoding pure entity identity, to context-sensitive selection (entity identity plus sentence record position), and finally to answer identity.

Critical strengths

The paper shows clean experimental separation of semantic identity, cross-procedure reuse, and causal granularity across layers—proving that a neural state's causal description changes over the course of a single forward pass even when the underlying concept stays fixed.

Critical weaknesses
  1. Prior entity-binding papers already established that LLMs use a mix of positional/ordinal slot-indexing and lexical mechanisms to bind entities in context. This paper takes a known phenomenon (models switching from lexical entity lookup to positional slot-indexing on templated lists) and repackages it in the formal language of causal abstraction granularity.
  2. The techniques themselves—full residual-stream activation patching, activation restoration, and orthogonal subspace projection—are established methods in the mechanistic interpretability toolkit.
  3. The entire empirical evaluation is restricted to a single toy world structure consisting of four consecutive, identically templated assignment sentences ("X holds the v.") followed by four relation sentences ("X's mentor is Y.").
Questions for authors
  1. All four assignment sentences appear consecutively at the start of every prompt in identical syntactic templates ("X holds the v."). If the four entity-value assignments are embedded in variable-length, non-parallel sentences or interleaved with distractor sentences (breaking the 1-to-4 ordinal list structure while keeping the semantic facts and relations intact), does the Block 20 →\rightarrow 21 transition in DD and the Block 25 →\rightarrow 27 cross-procedure convergence still exhibit context-arrangement sensitivity, or does the model fall back to an entity-level representation?
  2. Extending the relational procedure beyond a single mentor hop (f(f(QR))f(f(Q_R))) to test whether each additional relational hop re-triggers or deepens positional record dependence could be interesting.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.