acceptodds
Under review as a conference paper at ICLR 2027

When Transformers Copy Instead of Compose: Circuit Competition under Shared Semantics

Abstract

Foundation models are often given the same semantic event realized in several modes at once: an image and its description, a sentence and its translation, a specification of a program and its implementation. The semantics are shared across views while the realization is not, so a model can form two competing circuits: one that computes the content in the target view and one that copies it from the source view. We study this phenomenon in detail with synthetic data generated from probabilistic context-free grammars (PCFGs). Specifically, we hold view difficulty fixed and dial the levels of shared structure: a controlled hierarchical setting where two views can share semantic structure down to a precisely controlled level , with exact Bayes-optimal ceilings determining how much information is available from each view at each level. Probing the residual stream at every layer, we find the following: First, fusion between the views happens at the deepest shared latent level (the fusion boundary), the level where the shared information first becomes sufficient for the target view. Second, the availability of a copying route weakens composition at semantic levels above : once the model can copy the shared content from the source view, its further composition in the target view weakens. Third, this weakening is reversible: removing the target's access to the source, or otherwise reducing the copying route's reliability, restores the missing computation, which coexists with the copying route rather than being lost. Finally, we observe a consistent pattern in LLaVA-1.5-7B, where blocking attention to a conflicting caption restores image-based answers, and discuss implications for fusion in multimodal models.

Primary area:
interpretability and explainable AI
Keywords:
information fusion, multimodal fusion, mechanistic interpretability, representation learning, circuit competition, copy heads, decodability probing, shared semantics, two-view pcfg, 2-pcfg, probabilistic context-free grammars, random hierarchy model, bayes-optimal ceilings

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_S71QOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper uses a two-view hierarchical grammar and Bayes-optimal ceilings to study how shared semantics affect Transformer representations. Decodability decreases above the shared boundary and partially recovers when cross-view attention is blocked. LLaVA experiments provide qualitatively consistent behavioral evidence.

Critical strengths
  1. The known generative process and Bayes-optimal ceilings distinguish information availability from representation recoverability and identify which semantic levels are affected.
  2. Blocking source access in the same trained model substantially improves higher-level decoding while leaving levels below the shared boundary largely unchanged. This selectivity is key evidence supporting the paper.
Critical weaknesses
  1. The causal claims exceed the evidence. Sections 3.3 and 9 define composition through recoverability, but the abstract and Section 6 describe “suppression” of composition. Appendix G states that higher-level variables add nothing to next-token prediction given the shared latent variables. Reduced training pressure therefore remains an alternative to competitive suppression. Figure 3c also does not establish a causal link between its loss increase and reduced higher-level decodability.
  2. The key recovery experiment lacks an independent-rule control. Figure 2a/Table 1 use split-rule construction and prediction, whereas Appendix F suggests that Figure 4 uses shared-rule construction and decoding. The latter permits local token overlap: at Ls=3L_s = 3 with bottom-level m=2m = 2, corresponding leaf pairs are identical with probability 1/21/2. Local copying therefore remains an alternative explanation for the recovery attributed to semantic retrieval across distinct realizations.
Questions for authors
  1. Please confirm that Figure 2a/Table 1 use split-rule construction, prediction readout, and 100k training steps. Does Figure 4 use shared-rule construction, decoding readout, and 200k steps? Please specify these settings in each figure caption.
  2. Does the recovery in Figure 4 persist under split-rule construction?
  3. Do MLP probes or probes at multiple positions after span closure show the same decline? Can activation patching establish whether target-side Z1Z_1 representations affect downstream computation?
4 comments
AuthorsOpenReview ↗

Response to Reviewer S71Q

Thank you for your careful review and for suggesting specific controls, which led to several additional experiments. We revised the paper following your advice and you may find answers to each point below. For reference, the new experiments we've added, are in Sections 5–7 and Appendices E, F, G and H.1.


  1. The causal claims exceed the evidence. Sections 3.3 and 9 define composition through recoverability, but the abstract and Section 6 describe "suppression" of composition. Appendix G (now Appendix J) states that higher-level variables add nothing to next-token prediction given the shared latent variables. Reduced training pressure therefore remains an alternative to competitive suppression. Figure 3c also does not establish a causal link between its loss increase and reduced higher-level decodability.

Whether the drop comes from reduced training pressure or from competitive suppression depends on how the views are constructed. We've updated Section 6 and Appendix G.4 to separate the two. When the two views use separate rule tables (split rules), reduced training pressure is what happens: the target never learns to compose above the fusion boundary, severing source access lowers composition and only a model trained without source access composes these levels (Table 10). We call this starving. When they use a common rule table (shared rules), the composition circuit is present in the same weights and composes once source access is severed, so the effect happens at inference. Freeing the attention that the heads spend on the source, while keeping the copied content in place, already recovers 82% and 66% of the gain at LS=1L_S=1 and 22 (Appendix G.3, Table 9). Since the attention weights then sum to more than one, this test probes the mechanism rather than something the model can do. We call this suppression. Most of the recovery sits at the last target position (all of it at LS=1L_S=1 and 83% at LS=2L_S=2), which has no next token and receives no gradient (Appendix G.5). Since this position is never trained, the difference there between the intact and the severed pass, which share the same weights, comes from source access at inference rather than from training pressure. At the earlier position, which is trained, the copy delays composition rather than replacing it. In both cases, weakened and restored composition refer to recoverability (Section 3.3).

We agree that Fig. 3c alone does not establish this link. It ranks the span heads, which read the source span of the next boundary node and serve prediction. Blocking them raises the loss but restores no more composition than random heads (Table 8). The causal link to composition comes from a second kind of heads, the copying heads, which read the source's composed version of the node that the target has just closed. Blocking the top two per model restores 45% and 41% of the severing gain at LS=1L_S=1 and 22. At LS=3L_S=3 it takes more copying heads. Blocking all cross-view attention in layers 2–4 restores 52–77%, with next-node prediction intact (Section 6, Fig. 4b, Tables 7 and 8).


  1. The key recovery experiment lacks an independent-rule control. Figure 2a/Table 1 use split-rule construction and prediction, whereas Appendix F (now Appendix I) suggests that Figure 4 uses shared-rule construction and decoding. The latter permits local token overlap: at Ls=3L_s=3 with bottom-level m=2m=2, corresponding leaf pairs are identical with probability 1/21/2. Local copying therefore remains an alternative explanation for the recovery attributed to semantic retrieval across distinct realizations.

We ran the main experiments under split rules as well (Appendix E). No target leaf can then be copied from the source as a token, yet the drop above the fusion boundary is present (Fig. 6b), so the drop does not come from local copying. Under shared rules, what the copying heads pass on is the source's composed node, not its token: their output decodes the closed boundary node at 93%, against 44% from the aligned token alone (Appendix F.2, Table 3). Blocking only the aligned offset in every head, where a token copy would be read, restores 18%, 20% and −2% of the gain at LS=1,2,3L_S=1,2,3 (Table 8). Local copying through the aligned position therefore accounts for at most a fifth of the recovery, and for none at LS=3L_S=3, your example. We discuss the recovery under split rules in our response to Q2 below.

  1. Please confirm that Figure 2a/Table 1 use split-rule construction, prediction readout, and 100k training steps. Does Figure 4 use shared-rule construction, decoding readout, and 200k steps? Please specify these settings in each figure caption.

Yes for Fig. 2a and Table 1: split rules, the prediction readout and 100k steps. Fig. 4 uses shared rules and the decoding readout, but 100k steps, not 200k. The 200k runs are the masked ρ experiments of Figs. 2b, 2c and 3a–c. We've added the construction, masking and number of training steps to the caption of every synthetic experiment. We also corrected Appendix I (formerly Appendix F), which wrongly listed 200k steps for the severance experiments.


  1. Does the recovery in Figure 4 persist under split-rule construction?

We ran Fig. 4 under split rules and the recovery does not persist. This points to a second way in which copying costs composition. Severing source access lowers composition instead of restoring it (Z1Z_1 from 67% to 47% at LS=2L_S=2), while a model trained without source access reaches 99% (Appendix E, Fig. 7c, Table 10). Under shared rules both views use the same composition circuit, which composes from the target's boundary node as it does from the source's. Under split rules the target's boundary node is in a format that this circuit cannot read: a probe for the boundary node trained on the source reads the target's node at ceiling under shared rules but at chance under split rules (Appendix G.4, Fig. 10). With the source's copy always available in training, the target never learns to compose above the fusion boundary. We've updated Section 6, the abstract and the contributions to state this scope.


  1. Do MLP probes or probes at multiple positions after span closure show the same decline? Can activation patching establish whether target-side Z1Z_1 representations affect downstream computation?

We repeated the probing with an MLP probe and at later positions, and both show the same decline (Appendix E, Fig. 6a). The MLP probe improves prediction only above the fusion boundary, by 5–7 percentage points of the ceiling. One to four leaves after its span closes, Z1Z_1 decodes at 60–93% in the intact model, against 99–100% when source access is severed.

We also ran activation patching, in which only selected heads see the source of another sample. Through the two top copying heads alone, the other sample's boundary node reaches 43% in the target, against 29% for two volume-matched heads, 15% at baseline and 72% when every head sees it (Appendix F.4, Table 5). In the other direction, we swap the target's own boundary node into the source's composition circuit, which under shared rules changes the composed parent as much as the source's node does (Appendix G.4, Fig. 10b). These tests show that target-side representations feed the composition of the levels above. We have not shown an effect of the target's own Z1Z_1 on the model's output, which the Limitations state (Section 9). Under next-token prediction, the levels above the boundary help the target predict its next boundary node only when the source cannot supply it, which would explain why a model trained without source access composes them. With the source available, they add nothing to the loss (Appendix J). Testing their effect on the output therefore needs either the severed model, where the target must rely on them, or an objective that depends on them directly, which we leave to future work (Section 8).

Reviewer_S71QOpenReview ↗

Thanks for rebuttal

Dear Authors,

Thank you for your explanation. I am still confused about the practical function of composition suppression. I think the results you shown cannot prove that copying actually affects downstream predictions in a meaningful way. However, most of my concerns arised in previous review feedbacks have been answered. So, I would like to change my grade to weak reject (from my perspective, the quality of this paper still needs to be refined).

Best.

AuthorsOpenReview ↗

Thank you for your follow-up

Thank you for reading our responses, for following up and for raising your score. We are glad that most of your concerns have been addressed.

On the practical function of composition suppression: next-token prediction is not affected, by construction. With the source present, the levels above the fusion boundary add nothing to the loss (Appendix J), which is exactly why the objective does not prevent copying. The cost is expected downstream: any zero-shot use of the representation that depends on a higher-level node of the grammar can only draw on what the target has composed. As an analogy, if a high-level node encoded the sentiment of a sentence, a model that copies it from another view, instead of composing it, would be unreliable whenever that view is missing, wrong or only partially informative. We tested this by giving the model a source from another sample, and the target takes on that sample's nodes (61% against 16% at baseline for Z1Z_1), while its own Z1Z_1 falls from 85% to 67% (Table 4). In the VLM experiments, the same happens at the output: when the caption contradicts the image, the models' answers follow the caption, and blocking the heads that read it restores image-consistent answers (Section 7, Table 14).

We see this as a foundational result for how language models are trained. Our results suggest that pretraining on next-token prediction rewards copying over composition whenever the context already contains the answer, and the loss gives no sign of this. This happens in multimodal data, and likely even more within a single modality, where repeated and paraphrased content is everywhere. If so, the curriculum of pretraining data (when the copying route becomes available) and objectives that put pressure on higher levels (e.g. through SFT) become levers for making models compose instead of copy, and our framework is the first that can measure the effect of each against exact ceilings.

We deliberately kept this paper to what we can show experimentally and avoided speculating beyond it. How training objectives, data recipes and architectures shape the competition between copying and composition, and how composition can be restored, is the subject of follow-up work that this framework makes possible (Section 8).

In light of this, we would be grateful if you would consider whether the current rating reflects the revised paper. Thank you again for your time and engagement.

Reviewer_vfP5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how shared semantics across two views affect a Transformer’s use of cross-view retrieval and within-view composition. The authors construct a controlled hierarchical grammar framework with adjustable sharing depth and masking correlation, using Bayes-optimal reference accuracies and linear probes to analyze representations at different semantic levels. They find that complementary evidence improves target representations, while redundant evidence can reduce the decodability of higher-level semantics above the deepest shared level. Attention analysis identifies a cross-view retrieval route, and blocking source access restores much of the reduced decodability without updating model weights. A supplementary LLaVA experiment shows that blocking access to a conflicting caption’s answer word restores image-based predictions. The work offers a perspective on competition between information-processing routes, although the connection between probe decodability, compositional computation, and practical multimodal capabilities remains incompletely established.

Critical strengths

The research question is meaningful, and the introduction establishes the motivation well. When different inputs contain shared semantics, a model may retrieve information across views, changing how it performs composition within the target view. The paper shifts the focus from which input the model relies on to how it obtains semantic information, and explains the connection to modality dominance and optimization imbalance. This perspective is useful for understanding how multimodal models use shared information.

The synthetic framework provides clear controls for studying shared semantics. The hierarchical grammar explicitly specifies shared content, view-specific content, and semantic relationships across levels. Bayes-optimal accuracy provides a reference for information availability, allowing the authors to examine the gap between what the inputs determine and what the model represents. This is a useful research tool for multimodal problems where the underlying semantic structure is difficult to observe directly.

The experiments follow a coherent progression and include meaningful intervention controls. The paper starts with changes in representations, examines cross-view attention, and then compares removing connections at inference with removing them throughout training. Random and attention-volume-matched controls also strengthen the analysis. Together, these experiments support a basic finding: cross-view access affects the decodability of higher-level target semantics, and this change can be reversed without updating model weights.

Critical weaknesses

The central measurement does not sufficiently establish the proposed interpretation of compositional computation. The paper primarily uses linear probe accuracy to measure weakened and restored composition. However, lower accuracy could also reflect changes in encoding, representational location, or linear separability. Bayes-optimal ceilings establish how much information the inputs contain, but do not directly establish whether the model performs the corresponding computation. Although the authors acknowledge this limitation, independent evidence connecting the observed representational changes to compositional computation and its functional consequences is still missing. This leaves uncertainty around the paper’s central mechanistic interpretation.

The experiments do not yet establish a sufficiently direct connection between the identified retrieval route and the suppression of higher-level representations. Ablating selected attention heads shows that the identified cross-view route is useful for next-token prediction, whereas the recovery of higher-level representations is primarily demonstrated by blocking all target-to-source attention. The experiments therefore do not directly establish that the identified route causes the reduced decodability. In addition, increasing ρ changes both the complementarity of the views’ information and the alignment of their observed positions. Its results do not fully separate the effects of reduced additional information from those of easier retrieval.

The synthetic experiments have limited coverage, leaving the stability and scope of the findings unclear. The experiments mainly use one fixed random grammar, a fixed hierarchy depth, and one small Transformer, with fixed input lengths and source–target positions. The five training seeds capture variation in initialization and training order, but not variation in the grammar or its structure. It is therefore difficult to determine whether the pattern around the fusion boundary reflects a broader computational phenomenon or depends substantially on this particular setup. This distinction is important given the paper’s aim of explaining route competition under shared semantics.

The real multimodal experiment provides limited support for the paper’s main mechanistic interpretation. The LLaVA experiment examines left–right judgments in a single model, with the answer word explicitly provided in the caption. Recovering performance close to the image-only condition requires blocking access to that word in half of the attention heads. This shows that the model can return to using the image when the direct textual answer becomes unavailable, but does not establish the hierarchical suppression of composition observed in the synthetic setting. The potential relevance to UMMs is clear, but the current experiments do not sufficiently establish how broadly this mechanism applies to real multimodal computation or when it causes practical capability limitations.

Questions for authors

Can the authors provide evidence beyond linear probes that higher-level compositional computation is affected? For example, an output task directly requiring higher-level semantics could be added within the existing synthetic framework, comparing performance with and without cross-view access. This would help determine whether changes in probe accuracy have functional consequences, or whether the current training objective simply does not require these higher-level semantics to remain explicitly decodable.

Does selectively blocking the identified cross-view retrieval route also restore the decodability of higher-level semantics? Comparing selective route ablation with blocking all cross-view attention, while reporting both higher-level representations and task outputs, would more directly test whether the identified route causes the observed suppression rather than merely contributing to prediction.

Under which minimal changes to the setup do the main findings remain consistent? In particular, do the reduction in decodability and its recovery persist across different random grammars and source–target relative positions? Figure 2a also uses independent rule tables, whereas the other main experiments use a shared rule table. Can the full sequence of findings—representational changes, route identification, and recovery—be reproduced under a consistent construction?

How can the proposed suppression of compositional computation be distinguished from general reliance on an explicit textual answer in a real multimodal model? For example, the caption could convey the answer indirectly through several relationships rather than explicitly containing the answer word. Examining behavior and intervention effects in this setting would help determine whether the phenomenon extends beyond direct answer retrieval and clarify its implications for cross-modal composition in UMMs.

3 comments
AuthorsOpenReview ↗

Response to Reviewer vfP5

Thank you for your careful review and the precise questions, which led to the main additions to the paper. We revised the paper following your advice and you may find answers to each point below. For reference, the new experiments we've added, are in Sections 5–7 and Appendices E, F, G and H.1.


The central measurement does not sufficiently establish the proposed interpretation of compositional computation. The paper primarily uses linear probe accuracy to measure weakened and restored composition. However, lower accuracy could also reflect changes in encoding, representational location, or linear separability. Bayes-optimal ceilings establish how much information the inputs contain, but do not directly establish whether the model performs the corresponding computation. Although the authors acknowledge this limitation, independent evidence connecting the observed representational changes to compositional computation and its functional consequences is still missing. This leaves uncertainty around the paper's central mechanistic interpretation.

We tested this beyond the linear probe (Appendices E and F.4). An MLP probe improves prediction only above the fusion boundary, by 5–7 percentage points of the ceiling, so Z1Z_1 stays at 68% of its ceiling at LS=2L_S=2 (Fig. 6a). Reading each level one to four leaves after its span closes, or at the best layer, gives the same picture. We also gave the model a source from another sample. When the two views use a common rule table (shared rules), the target's own Z1Z_1 falls from 85% to 67% and the target encodes the source's Z1Z_1 instead (61%, against 16% at baseline), although its own leaves are unchanged (Table 4). Patching only the two top copying heads (Section 5) with the source of another sample, brings the other sample's boundary node into the target at 43%, against 29% for two volume-matched heads and 15% at baseline (Table 5). The drop is therefore unlikely to be a matter of encoding, location or linear separability. What we do not show is an effect on the model's output, which the Limitations state. We return to this in our response to Q1 below.


The experiments do not yet establish a sufficiently direct connection between the identified retrieval route and the suppression of higher-level representations. Ablating selected attention heads shows that the identified cross-view route is useful for next-token prediction, whereas the recovery of higher-level representations is primarily demonstrated by blocking all target-to-source attention. The experiments therefore do not directly establish that the identified route causes the reduced decodability. In addition, increasing ρ changes both the complementarity of the views' information and the alignment of their observed positions. Its results do not fully separate the effects of reduced additional information from those of easier retrieval.

Following your question, we looked for the heads behind the recovery and found that the target reads the source through two kinds of heads (Section 5, Appendix F). The span heads, which Fig. 3c ranks, read the source span of the next boundary node and serve prediction. Blocking the top 16 breaks next-node prediction, but restores no more composition than 16 random heads (Table 8). The copying heads read the aligned position, where the source has already composed the node that the target has just closed. Blocking only the top two per model restores 45% and 41% of the gain from severing source access at LS=1L_S=1 and 22, against 16% and 6% for two volume-matched heads. At LS=3L_S=3 it takes more copying heads. Blocking all cross-view attention in layers 2–4 restores 52–77% (Section 6, Fig. 4b, Tables 7 and 8). Next-node prediction stays at 97% or more after every block except that of the span heads.

We agree that ρ changes both how much the source adds and how easy it is to retrieve. We removed the sentence in Section 3.5 that claimed ρ alone separates the two. The separation comes from two other results. Without masking, both views fully determine every latent level, so the source adds no information. Composition above the fusion boundary is still weaker (Z1Z_1 at 85% of a 100% ceiling at LS=2L_S=2), and under shared rules severing source access restores it (Section 6). At ρ=1, the target ends 9.7 points below the source although it sees strictly more input, so the gap cannot come from less additional information (Section 4).

AuthorsOpenReview ↗

Continued [1]

The synthetic experiments have limited coverage, leaving the stability and scope of the findings unclear. The experiments mainly use one fixed random grammar, a fixed hierarchy depth, and one small Transformer, with fixed input lengths and source–target positions. The five training seeds capture variation in initialization and training order, but not variation in the grammar or its structure. It is therefore difficult to determine whether the pattern around the fusion boundary reflects a broader computational phenomenon or depends substantially on this particular setup. This distinction is important given the paper's aim of explaining route competition under shared semantics.

We repeated the main experiments on two more random grammars, under both shared and split rules (Appendix E). In every grammar the fusion boundary is recovered completely, Z1Z_1 reaches 60–66% of its ceiling at LS=2L_S=2 and the two constructions stay within 6 percentage points of each other in every cell (Fig. 6b). The sign flip of Fig. 2c repeats at nearly the same size (Fig. 6c), and severing source access again raises decoding only above the fusion boundary (Fig. 7b). We also flipped the order of the two views, and the pattern of Fig. 2a holds. We kept the depth of the hierarchy, the model size and the layout of the views fixed. We also state in the Limitations that real multimodal inputs differ in their layout.


The real multimodal experiment provides limited support for the paper's main mechanistic interpretation. The LLaVA experiment examines left–right judgments in a single model, with the answer word explicitly provided in the caption. Recovering performance close to the image-only condition requires blocking access to that word in half of the attention heads. This shows that the model can return to using the image when the direct textual answer becomes unavailable, but does not establish the hierarchical suppression of composition observed in the synthetic setting. The potential relevance to UMMs is clear, but the current experiments do not sufficiently establish how broadly this mechanism applies to real multimodal computation or when it causes practical capability limitations.

We repeated the experiment in LLaVA-1.5-7B, Qwen2.5-VL-7B and Qwen3-VL-8B, with captions written in French and Spanish, so that they don't share the answer word with the English question (Section 7, Appendix H.1). Under conflict, most answers still follow the caption in every model. Blocking the whole caption in the heads that read it most, an eighth to a sixth of all heads, brings agreement with the image back to about its level without a caption (72–80%, against 73–83%), while volume-matched and random heads leave it at 24–56% (Table 14, Figs. 11 and 12). In the original experiment, which blocked only the answer word, this took half of all heads. Captions that convey the answer only indirectly give the same result, which we describe in our response to Q4 below. We agree that this does not show the hierarchical suppression of the synthetic setting. We read these results as consistent with the synthetic mechanism, not as a demonstration of it (Section 7).


Can the authors provide evidence beyond linear probes that higher-level compositional computation is affected? For example, an output task directly requiring higher-level semantics could be added within the existing synthetic framework, comparing performance with and without cross-view access. This would help determine whether changes in probe accuracy have functional consequences, or whether the current training objective simply does not require these higher-level semantics to remain explicitly decodable.

We describe the evidence beyond linear probes in our response to the first weakness above (MLP probes, later read positions, the conflict test and patching). We considered the output task you suggest. An output that requires a higher-level node puts direct pressure on that level, and we expect it to restore composition even when the source is available, as removing source access in training does (Table 10). It would therefore test a remedy rather than whether the competition arises under next-token prediction alone, which is the question of this paper. We list such top-down objectives as future work (Section 8).

We tested whether "the current training objective simply does not require these higher-level semantics" by training a model with the same objective but without source access. Under split rules, this model composes these levels, reaching 99% at Z1Z_1 at LS=2L_S=2 and 33, against 67% and 62% for the intact model (Table 10). The objective alone therefore does not explain the drop, which comes from the availability of the source.

AuthorsOpenReview ↗

Continued [2]

Does selectively blocking the identified cross-view retrieval route also restore the decodability of higher-level semantics? Comparing selective route ablation with blocking all cross-view attention, while reporting both higher-level representations and task outputs, would more directly test whether the identified route causes the observed suppression rather than merely contributing to prediction.

Yes, through the copying heads, rather than the span heads that Fig. 3c ranks (see our response to the second weakness above). We compared every block with severing all target-to-source attention and report both composition and next-node prediction (Fig. 4b, Tables 7 and 8). Blocking the two top copying heads restores composition while next-node prediction stays at 97% or more (Table 8). Blocking a copying head both removes the copied content and frees the attention the head spent on the source. We separated the two effects, and at LS=1L_S=1 and 22 they are of similar size (Section 6, Appendix G.3, Table 9).


Under which minimal changes to the setup do the main findings remain consistent? In particular, do the reduction in decodability and its recovery persist across different random grammars and source–target relative positions? Figure 2a also uses independent rule tables, whereas the other main experiments use a shared rule table. Can the full sequence of findings—representational changes, route identification, and recovery—be reproduced under a consistent construction?

We repeated the drop and its recovery on two more random grammars, and both hold (Appendix E, Figs. 6b, 6c and 7b). For the positions, we flipped the order of the two views and the pattern of Fig. 2a holds.

We also reran Fig. 2a under shared rules (Fig. 6b), so that the full sequence can be followed under one construction. Under shared rules it holds: the drop (Fig. 6b), the sign flip (Fig. 2c), the two kinds of heads (Fig. 3) and the recovery (Fig. 4). The sign flip is measured with masking, the copying heads and the recovery without it. With masking, the source has not composed the node at the aligned position, so no copying heads emerge and the suppression is spread over many heads (Appendices F.5 and G.2). Under split rules, the drop and both kinds of heads also hold (Fig. 6b, Table 6), but severing source access lowers composition instead of restoring it. Only a model trained without source access composes these levels (Fig. 7c, Table 10). The reason is that under shared rules both views use the same composition circuit, while under split rules the target's boundary node is in a format that this circuit cannot read (Section 6, Appendix G.4, Fig. 10). Copying therefore suppresses composition under shared rules and starves it under split rules. We've updated the abstract, the contributions and Section 6 to state this.


How can the proposed suppression of compositional computation be distinguished from general reliance on an explicit textual answer in a real multimodal model? For example, the caption could convey the answer indirectly through several relationships rather than explicitly containing the answer word. Examining behavior and intervention effects in this setting would help determine whether the phenomenon extends beyond direct answer retrieval and clarify its implications for cross-modal composition in UMMs.

We used relational captions, such as "The lamp is on the right. The dog is on the opposite side." (Appendix H.1). LLaVA-1.5 can't resolve these even without an image, so we used Qwen3-VL-8B, which follows all three wordings on a blank image. Under conflict, the only word in such a caption that names a side is the lamp's side, yet the model follows what the caption means in 55–84% of the examples. Blocking the heads that read these captions most (192 heads) brings agreement with the image from 16–43% back to 81–87%, its level without a caption, while volume-matched heads reach only 36–57% (Table 14, Fig. 11b). The phenomenon therefore extends beyond direct answer retrieval.

Reviewer_rbYaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper investigates how combining views of a single semantic concept affects model processing of that semantic concept. An image and its caption are a simple example of two views of a single semantic concept. Specifically, they look at the question of how access to a second view changes the way in which the semantic information is represented.

Assuming simple information on semantics is represented in each view, the authors look into different cases of how this simple information can be composed to answer higher-order (complex) questions:

  1. The information can be taken from a single view and composed to yield an answer.
  2. Fusion is when the information is found across different views and combined between them (e.g. some information comes from caption, some information comes from image).
  3. Copying is when the composed information is copied from a different view. To study this question, the authors use controlled PCFG data and compute formal ceilings for the amount of information available at each representation level.

The authors find that when the two views include complementary pieces of information on the semantic concept, the representation is improved; but when the views are redundant (same semantic information appears in both views) the model can copy information from one view instead of relying on composition from the second view (unclear why this is bad if both views contain the same amount of information, and this isn't discussed). The authors look into how each of these routes (copying vs fusing) is implemented in model internals, and draw a parallel to some empirical findings in an image-text VLM.

I think the main contribution of the paper is the controlled experimental setup described in section 3, which can be used in further theoretically-grounded multimodal studies. However, the current presentation is not clear enough (see critical weaknesses), and the paper needs a major revision before it is accepted.

Critical strengths

The controlled setup is a nice contribution. Using PCFG as data for studying the question of information composition across views is a very interesting approach. It can be helpful for further research on understanding multimodality and multimodal models. Such research has tended to be more empirical and less theoretically grounded, thus the approach suggested in this paper seems like it could be helpful.

Critical weaknesses

The writing is unclear and required a lot of effort to understand. I suspect heavy use of AI for writing and polishing, which makes the content very difficult to follow.

Below are some suggestions to the authors to improve the unclear presentation:

  1. Don't present concepts / jargon without immediately explaining what it means. e.g. in the abstract its unclear what the difficulty of a view is and how it's measured; what are the target / source views (which are heavily used in the introduction before it is properly defined); is L_S an actual model layer or a conceptual semantic hierarchical level? L_S is also presented in the introduction and heavily relied on without being explicitly explained.

  2. Write with linear flow (i.e. a->b,b->c,c->d; and not a->b, c->d, b->c). Lines 72-88 present the main question, expansion of this question and what it means, and then suddenly a reference to findings on "Section 5 localizes a mechanism that implements it" before jumping again to presenting the main question. This is too messy and unclear. Instead, the authors should first present the research question and motivation and expand on it, and only then move to the results and conclusions. The next paragraph, starting at line 89, is another example of this. Instead of starting with the main point (which I presume is the motivation behind using PCFGs as data), the authors start with a negative example of why image--text multimodal data is not a good test bed for their question.

  3. There are many sentences which are just too convoluted. e.g. "the evidence becomes redundant, through a distributed, span-level retrieval route that strengthens with retrievability" (this sentence is too complex, and that's even without mentioning that "span-level retrieval route" was not defined at any point prior to this sentence; see point 1). There are many such examples.

  4. The technical writing is often unclear as well.

    • Part 3.1 is actually fully clear (and should be the first point where the formal markers such as L_S are presented).

    • However, in 3.2, it is unclear from "We probe the residual stream at every layer to measure the information about the hierarchical target variables that has been computed at each stage" what the probes are actually trained to predict (what are "hierarchical target variables"?).

    • Another example is the sentence "The probe predicts the latent RHM node that has just closed at the current position" at line 216. But what if two or more latent RHM nodes close at the same position (e.g. "n" and "e" and "h" are closed at the position of the blue token "x" in the example in fig1; which is the true label here (n / e / h))?

    • Also, I had a question (which wasn't clear from the writing) regarding semantics and masking: if a token is masked in a view (e.g. the token "v" in the source view in fig1) doesn't it make the rest of the semantics ambiguous? (e.g. won't masking "v" in fig1 cause the shared representations of "e", "h", "l" to be potentially different under the orange view?)

  5. As part of the presentation, I would've expected to see an explicit motivation for the research question (ideally early on). However, the motivation remains unclear. I understand why this work is different from previous work, but what I'm really missing (from the introduction) is why the question is interesting / relevant at all.

  6. General heuristic: figure captions should not overflow between pages.

Beyond writing, the comparison to the experiment in a real VLM has its problems (I'm emphasizing this part since in my opinion this is the most interesting usage for the framework suggested by the authors):

  1. The caption view is a single token. This is (1) very different from real VLM training captions; (2) the single-token makes it extremely easy for a model to induce a token-copy head and just copy the token as-is, which is indicated by the 100% accuracy of this setup (line 447). Thus, I'm not convinced that the VLM setup (as presented in section 7) is equivalent to the controlled setup and its conclusion. In other words, there is no evidence to show that the model builds a shared semantic representation of the object, analyzes which side the object is on, and then answers; and that this process is affected by a wrong caption; rather, the likely explanation is the simpler "model copies token" explanation as-is).

  2. Additionally, the authors ignore a long line of works on "mechanism races" in-the-wild (both in LLMs and VLMs) that discuss these questions in actual non-toy settings (e.g. https://arxiv.org/abs/2410.02102, https://arxiv.org/abs/2507.13868, https://arxiv.org/abs/2507.01790). I therefore don't believe that their controlled setup parallels the VLM setup (but I would be very excited to see this part investigated more in-depth with more soundness!).

Questions for authors

Please refer to the weaknesses mentioned above

4 comments
AuthorsOpenReview ↗

Response to Reviewer rbYa

Thank you for your careful review and for the detailed suggestions on the writing. We revised the paper following your advice and you may find answers to each point below. For reference, the new experiments we've added, are in Sections 5–7 and Appendices E, F, G and H.1.


The authors find that when the two views include complementary pieces of information on the semantic concept, the representation is improved; but when the views are redundant (same semantic information appears in both views) the model can copy information from one view instead of relying on composition from the second view (unclear why this is bad if both views contain the same amount of information, and this isn't discussed).

Copying gives the same answer only as long as the source is there and agrees with the target. A target that has not composed the content itself, adopts the source's version of it. We now discuss this in the introduction: even when the views carry the same information, copying weakens composition in the target view (Section 1). We tested it by giving the model a source from another sample (Appendix F.4). When the two views use a common rule table (shared rules, Section 3.1), the target's own Z1Z_1 falls from 85% to 67% and the target encodes the source's Z1Z_1 instead (61%, against 16% at baseline), although its own leaves are unchanged (Table 4). With separate rule tables (split rules), it falls to 29% (Table 6). A VLM faces a similar situation when a caption contradicts the image.


The writing is unclear and required a lot of effort to understand. [...] Below are some suggestions to the authors to improve the unclear presentation: 1. Don't present concepts / jargon without immediately explaining what it means. [...] 2. Write with linear flow [...] 3. There are many sentences which are just too convoluted. [...] 4. The technical writing is often unclear as well. [...] 5. As part of the presentation, I would've expected to see an explicit motivation for the research question (ideally early on). [...] 6. General heuristic: figure captions should not overflow between pages.

We followed your suggestions throughout the paper. We rewrote the introduction in the order you suggest, with the motivation before the question and every term explained where it first appears. We also shortened the sentences, defined the two kinds of heads that read the source (span heads and copying heads) and removed the overflow caption in Figure 1.

On your technical questions:

  • However, in 3.2, it is unclear from "We probe the residual stream at every layer to measure the information about the hierarchical target variables that has been computed at each stage" what the probes are actually trained to predict (what are "hierarchical target variables"?).

    We fit one probe per depth of the target's hierarchy, which predicts which value the latent node takes at that depth. Section 3.2 now states this directly.

  • Another example is the sentence "The probe predicts the latent RHM node that has just closed at the current position" at line 216. But what if two or more latent RHM nodes close at the same position (e.g. "n" and "e" and "h" are closed at the position of the blue token "x" in the example in fig1; which is the true label here (n / e / h))?

    Each depth has its own probe and its own label. At "x", the Z3Z_3 probe predicts "n", the Z2Z_2 probe "e" and the Z1Z_1 probe "h". We've revised Section 3.3 to say "the latent RHM node at depth dd", which makes this clear.

  • Also, I had a question (which wasn't clear from the writing) regarding semantics and masking: if a token is masked in a view (e.g. the token "v" in the source view in fig1) doesn't it make the rest of the semantics ambiguous? (e.g. won't masking "v" in fig1 cause the shared representations of "e", "h", "l" to be potentially different under the orange view?)

    Masking hides leaves but not the latent nodes. The shared nodes ("e", "h" and "l") are the same in both views by construction and masking "v" only reduces the information that the source view can provide about them, which is what the Bayes ceilings measure (how much information the source still reveals). We've updated Section 3.5 to state this.

  1. The caption view is a single token. This is (1) very different from real VLM training captions; (2) the single-token makes it extremely easy for a model to induce a token-copy head and just copy the token as-is, which is indicated by the 100% accuracy of this setup (line 447). Thus, I'm not convinced that the VLM setup (as presented in section 7) is equivalent to the controlled setup and its conclusion. In other words, there is no evidence to show that the model builds a shared semantic representation of the object, analyzes which side the object is on, and then answers; and that this process is affected by a wrong caption; rather, the likely explanation is the simpler "model copies token" explanation as-is).

To test whether the model only copies the caption token, we repeated the experiment with captions that do not contain the answer word in English, in LLaVA-1.5-7B, as well as in Qwen2.5-VL-7B and Qwen3-VL-8B (Section 7, Appendix H.1). We used captions written in French and Spanish, so that they don't share the same word with the English question. Under conflict, most answers still follow the caption in every model: 80% and 71% in LLaVA and 58–64% in the Qwen models. Blocking the heads that read the caption most (an eighth to a sixth of all heads) brings agreement with the image back to about its level without a caption (72–80%, against 73–83%), while volume-matched and random heads leave it at 24–56% (Table 14, Fig. 11).

We also used relational captions, such as "The lamp is on the right. The dog is on the opposite side.". They give the same result in Qwen3-VL (LLaVA-1.5 can't resolve these even without an image). Under conflict, the only word in such a caption that names a side is the lamp's side, yet the model follows what the caption means in 55–84% of the examples. Blocking the heads that read these captions most, as above, brings agreement with the image back to 81–87%, its level without a caption (Table 14). The competition is therefore with what the caption means, not with a token. We agree that our captions remain short, unlike real training captions. Captions that realize the shared content through many tokens, with a known shared hierarchy, are the setting of the multimodal grammar we outline as future work (Appendix B). We read these results as consistent with the synthetic mechanism, not as a demonstration of it (Section 7).


  1. Additionally, the authors ignore a long line of works on "mechanism races" in-the-wild (both in LLMs and VLMs) that discuss these questions in actual non-toy settings (e.g. https://arxiv.org/abs/2410.02102, https://arxiv.org/abs/2507.13868, https://arxiv.org/abs/2507.01790). I therefore don't believe that their controlled setup parallels the VLM setup (but I would be very excited to see this part investigated more in-depth with more soundness!).

Thank you, these works are closely related. Lepori et al. trace contextualization errors in LLMs to races between processes that run in parallel across layers. Ortu et al. and Hua et al. find small sets of heads that decide which source wins when a VLM's sources of information conflict, which is the setting of our Section 7. Lepori et al. is now cited in Section 2 and Appendix J, alongside Hua et al. and Ortu et al. These works study which source wins in pretrained models, where the correct computation is unknown. Our setting adds exact ceilings, which tell us which computation is skipped and at which level.

Thank you for the response and for the effort to address my concerns.

My main remaining concern is the connection between the controlled setting and real VLMs. While the new experiments provide stronger evidence for competition between information sources, they do not establish that the specific mechanisms identified in the synthetic setting are also operating in VLMs. As I mentioned in my original review, I see the controlled experimental framework as the main contribution of the paper. I therefore think it is particularly important to clearly motivate what this framework allows us to investigate and what new insights it provides beyond the already established phenomenon of competition between information sources. I therefore maintain my original score.

AuthorsOpenReview ↗

Thank you for your follow-up

Thank you for the follow-up. We understand your scepticism. Going from a synthetic grammar to real models takes a leap of faith, and we felt it too. The conviction came only after repeated experiments showed the same phenomena in small and large models.

That copying competes with composition in real models is not in question. Induction heads, hailed as a key feature of foundation LMs, are a copying mechanism, and the works you cited (Lepori et al., Ortu et al., Hua et al.) show source competition in pretrained LLMs and VLMs. Our VLM results add to this: models follow captions that contain no answer word, and blocking the heads that read the caption restores what the image determines.

What is missing from the literature is a tool to measure how much composition is lost, and to derive scaling laws for copy vs. compose. In pretrained models this cannot be done, because the correct computation is unknown. Our framework provides exact ceilings at every level, together with control over how deep the shared semantics run, how complementary the views are, and how they are realized.

Beyond the framework, the second contribution of the paper, and in our opinion the most important, is a surprising result. Under shared semantics, next-token prediction pushes the model further into copying, and more is less: extra information can produce worse representations. The target sees strictly more input than the source, yet ends 9.7 points below it (Section 4). A model trained without access to the source composes the higher levels at 99%, against 67% for the model that has it (Table 10). Giving the model a second view of the same content makes it compose less.

We deliberately kept this paper to what we can show experimentally and avoided speculating beyond it. Scaling laws for copy vs. compose, how training objectives, bidirectionality, data recipes and architectures shape this competition, and how composition can be restored are the subject of follow-up work that this framework makes possible (Section 8).

Thank you again for your engagement.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank the reviewers for their thoughtful feedback. Below we address the shared concerns and the new experiments each one led to. Detailed answers follow in the reply to each review.

All three reviewers see value in the controlled setting. Reviewers rbYa and vfP5 describe the two-view grammar as a useful tool for multimodal research. Reviewers vfP5 and S71Q highlight the exact Bayes ceilings, which separate what the inputs determine from what the model represents. Reviewer S71Q also considers the selective recovery when the source is blocked to be strong support for our claims. Reviewer vfP5 finds the research question meaningful and well motivated, and mentions the coherent progression of the experiments and their intervention controls.

The concerns fall into four groups:

  • whether the drop in composition above the fusion boundary ZLSZ_{L_S} is an artifact of linear probes (Reviewers vfP5 and S71Q)
  • whether it holds beyond one grammar and one way of constructing the views (Reviewers vfP5 and S71Q)
  • whether the route identified in Section 5 is what causes it (Reviewers vfP5 and S71Q)
  • whether the VLM experiment shows more than copying a caption token (Reviewers rbYa and vfP5).

Reviewer rbYa also asks for a clearer presentation, which we revised throughout.

The drop is not a probing artifact. It remains with an MLP probe, which improves prediction at the levels above LSL_S by only 5–7 percentage points of the achievable (Bayes-optimal) accuracy compared to the linear probe and leaves the other levels unchanged (Appendix E, Fig. 6a). Reading the representation at different positions after the span closes or at the best layer instead of the last one, confirms the results of the paper with both linear and MLP probes.

The drop also holds across different grammars and view constructions. We repeated the experiments on two additional random grammars and under two constructions: one where the target has its own rule table, so that no token can be copied (split rules) and one where both views share it (shared rules). In every case the shared level is recovered completely, the levels above it fall short and, under shared rules, severing source access restores them. The pattern also holds when the order of the two views is flipped.

Following the questions of Reviewers vfP5 and S71Q, we looked into the route in more detail and found that it is made of two kinds of heads with different roles: one carries the fusion at the shared level and the other causes most of the drop. The heads that we rank by span concentration in Section 5 (span heads), read the source span of the next shared node, which the target cannot know yet. They serve prediction, which is why blocking them raises the loss (Fig. 3c). The heads that compete with composition (copying heads) read one position further, where the source has already composed the node that the target has just closed. They sit in the middle layers of the network (layers 2–4 out of 8) and blocking cross-view attention there, restores 77% of the composition (at LS=2L_S=2) while prediction stays intact.

Whether composition comes back when the copying route is blocked at inference, depends on the view construction. With shared rules, both views use the same composition circuit. So once the copying route is blocked, the target composes the higher levels itself. With split rules, the target's boundary node is in a format, that the shared composition circuit cannot read, so the target never learns to compose above LSL_S and only training without the route helps. In both cases, it is the availability of the shared information that keeps the target from fully composing the levels above LSL_S. Copying therefore suppresses composition under shared rules and starves it under split rules (Section 6).

Finally, to test whether the VLM result reduces to copying a single caption token, we used captions that do not contain the English answer word. When a caption written entirely in French or Spanish ("le chien est à droite") contradicts the image, LLaVA-1.5, Qwen2.5-VL and Qwen3-VL follow the caption in 58–80% of the cases, depending on the model (chance is 50%). The same holds in Qwen3-VL for relational captions, where the correct answer is only given relative to a second object ("The lamp is on the right. The dog is on the opposite side."). In both cases, blocking the heads that read the caption most brings agreement with the image back to about the level when no caption is given (72–87%), while blocking matched or random heads does not. The model therefore follows what the caption means, not a word it could copy.

The revised PDF is now uploaded. Following the advice of Reviewer rbYa, we rewrote the introduction and the definitions. We also extended Section 5 with the two kinds of heads, stated in Section 6 whether composition comes back once the copying route is blocked and added the new experiments to Appendices E–H, with an overview table at the start of the appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.