acceptodds
Under review as a conference paper at ICLR 2027

Composition and Closure: Causal Tests of Linguistic Operators in Transformers

Abstract

A causal account of grammar in a Transformer must explain not only whether individual linguistic operations can be realized, but whether those operations continue to act correctly on states produced by earlier operations. We formalize this requirement as complexity-bounded causal realization: rank-bounded interchange maps are learned from unary counterfactuals at a fixed layer and span, then evaluated under held-out composition and on their intervention closure, the states reach- able through valid intervention sequences. We show formally that perfect unary correctness on observational states does not imply correct composition, and give conditions under which closure-level one-step control bounds error along composed interventions. To study this distinction empirically, we introduce OpClosure, a synthetic benchmark with exact counterfactuals and controlled operator depth. Across independently trained Transformers, maps that perform well on unary interventions degrade systematically when applied to closure states, including after an earlier operation has been realized correctly. A modular reference is substantially more robust, while training with composed targets recovers much of the lost closure-level performance. We further observe analogous failures in controlled English experiments across pretrained models from multiple families and scales. These results suggest that unary causal success is insufficient evidence for compositional grammatical mechanisms: causal realization should be evaluated on the states that the proposed mechanism itself produces.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, causal abstraction, interchange interventions, compositionality, intervention closure, linguistic operators, Transformers, natural language processing, synthetic benchmarks

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_AuTvOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The article asks whether an intervention for controlling the linguistic grammar of a string output by a language model can be stacked with another such intervention. I.e., to what degree the individually learned patches for controlling the grammar can be combined. To judge this, a synthetic dataset is proposed. These are evaluated by looking at the predictions of the language model. Validation is done on natural language (e.g. number and tense) and synthetic examples (also negation, active to passive, object swap). The authors find that the interventions (using activation patching) can be less reliable when combined (sequentially through a grammar-specific projection) than on their own. An abstract counterexample to the patches working together is also given. A proof is given for a bound on the error from stacking patches under strict conditions.

Critical strengths

None

Critical weaknesses

The paper is hard to follow. I could barely understand the abstract and the introduction (bar its first paragraph) when first seeing them. Terms are used without having been introduced, and the flow is confusing (see e.g. the start of the third paragraph of the introduction: “A finite counterexample shows why reuse needs testing.” Counterexample of what? Why finite? What is being reused? Why test?). The methodological chapter is not much better; it starts with a claim about OpClosure and some properties said to toggle things, except it is not clear what OpClosure is yet (it’s referenced in the introduction and abstract but would like to see it introduced properly), nor what it means to toggle something. Or at the start of 3.2. “Sequential reuse generates a domain of reached activations.” There is so much context missing that reading the paper becomes detective work.

There is also a worry about the novelty. Many previous works do patching, and datasets like CausalGym have grammatical settings. See e.g., the papers you cite (Todd et al. 2024) who apply grammatical patches for tense and number in combination with steering to pick a word, and (Klerings et al 2025) that compare tense and aspect, no direct comparisons are made to these methods. Two others that are not cited (Brinkmann et al, 2025 and Hendel et al 2023) that steer grammatical properties. Besides the linguistic grammar there are many other works on steering and patching, that do stack and consider impacts thereof, e.g. ReFT (Arora et al, 2024) and pyvene (Wu et al 2024) – the last of which you point out already in appendix K.

Questions for authors

Given how crowded this field is I'd like to understand better what your protocol specifically reveals that prior work does not do? (I.e. is there anything that changes that we thought otherwise about or tells us something new, and why did existing evaluations miss this, and could this not have been checked with existing datasets like CausalGym or adaptions thereof?)

Reviewer_ZSYvOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

NA

Critical strengths

NA

Critical weaknesses

Unfortunately, I think this article should be desk rejected. I found the paper to be unreadable. Even by the second sentence in the abstract, it is incomprehensible: "We formalize this requirement as complexity-bounded causal realization: rank-bounded interchange maps are learned from unary counterfactuals at a fixed layer and span, then evaluated under held-out composition and on their intervention closure, the states reach- able through valid intervention sequences."

Additionally, the over-formalization and poor explanation seems like LLM generated text to me.

Questions for authors

NA

Reviewer_cFrkOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents and formalizes complexity-bounded causal realization , a framework for evaluating whether neural interventions implementing individual linguistic operations work under composition. The interventions are learned from unary counterfactual examples as rank-bounded interchange maps, then frozen and applied sequentially. The paper discusses two metrics: endpoint correctness (whether the final reached state matches the target state) and closure-local correctness (whether the final reached state matches the intervention on the last decoded state). The paper provides a counterexample showing that perfect unary correctness does not necessarily imply compositional correctness and presents sufficient conditions for bounding composition error. The paper then presents OpClosure, a synthetic benchmark with exact counterfactual targets and composition at controlled depths. Experiments show that learned maps often lose accuracy under composition, including after a correctly decoded first step. Additional experiments look at modular reference models trained with composed targets and number-tense interventions in pretrained Language Models.

Critical strengths
  1. The two scores for endpoint correctness and closure-local correctness are useful to understand different aspects of composition under interventions: the correctness of the final reached state w.r.t the expected final state and the correctness of the final reached state given the previous decoded state.
Critical weaknesses
  1. It was very difficult for me to understand the paper with technical terms introduced without any definitions ("reuse" in line 45, "caused-motion introduction" in line 129, "echo-tail site" in line 334, etc.), and even complete sentences that used multiple such terms ("validation candidate-mass gate excludes CM there and admits it in instance 90904" in lines 334-335). The opening examples in the introduction motivate the problem on a high level but do not explain how these relate precisely to the "counterfactual training examples", "matched donor sentences" discussed in the text. Several details from Section 3 (the key formalization) and Section 5 (OpClosure and empirical evaluation) are also excluded from the main text and deferred to the appendix. As OpClosure is an important contribution of this work, difficulty understanding the development and evaluation on this suite limits usefulness beyond this work. The main text would greatly benefit from a worked example tracing donor selection, map fitting, and sequential application of maps.

Overall, I appreciate the interesting question that the paper addresses but the presentation substantially obstructs understanding and assessment of the work.

Questions for authors
  1. Could the authors present a clear and concrete example explaining the various technical terms they present in Section 3? Please identify base and source and their aligned positions, explain what reuse means for the learned maps, show the observational support, an intervention reached state, and its relationship to the intervention closure. For the same example, please also discuss the diagnostic continuations, observable signature, and two-stage sampling law. A walkthrough of Section 3 with this example would greatly improve the readability of the section.

  2. What are the intuitive takeaways from Proposition 1 and Theorem 1? Concretely, how do the theorem's assumptions and guarantees relate to the metrics and conclusions from the experiments?

  3. Could you explain OpClosure through some representative examples from the benchmark? Given a trained transformer, what does a researcher need to fit, what inputs and counterfactual targets are supplied by the benchmark, and how should the resulting scores be interpreted?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.