- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper uses a two-view hierarchical grammar and Bayes-optimal ceilings to study how shared semantics affect Transformer representations. Decodability decreases above the shared boundary and partially recovers when cross-view attention is blocked. LLaVA experiments provide qualitatively consistent behavioral evidence.
Critical strengths
- The known generative process and Bayes-optimal ceilings distinguish information availability from representation recoverability and identify which semantic levels are affected.
- Blocking source access in the same trained model substantially improves higher-level decoding while leaving levels below the shared boundary largely unchanged. This selectivity is key evidence supporting the paper.
Critical weaknesses
- The causal claims exceed the evidence. Sections 3.3 and 9 define composition through recoverability, but the abstract and Section 6 describe “suppression” of composition. Appendix G states that higher-level variables add nothing to next-token prediction given the shared latent variables. Reduced training pressure therefore remains an alternative to competitive suppression. Figure 3c also does not establish a causal link between its loss increase and reduced higher-level decodability.
- The key recovery experiment lacks an independent-rule control. Figure 2a/Table 1 use split-rule construction and prediction, whereas Appendix F suggests that Figure 4 uses shared-rule construction and decoding. The latter permits local token overlap: at with bottom-level , corresponding leaf pairs are identical with probability . Local copying therefore remains an alternative explanation for the recovery attributed to semantic retrieval across distinct realizations.
Questions for authors
- Please confirm that Figure 2a/Table 1 use split-rule construction, prediction readout, and 100k training steps. Does Figure 4 use shared-rule construction, decoding readout, and 200k steps? Please specify these settings in each figure caption.
- Does the recovery in Figure 4 persist under split-rule construction?
- Do MLP probes or probes at multiple positions after span closure show the same decline? Can activation patching establish whether target-side representations affect downstream computation?
4 comments
Continued
- Please confirm that Figure 2a/Table 1 use split-rule construction, prediction readout, and 100k training steps. Does Figure 4 use shared-rule construction, decoding readout, and 200k steps? Please specify these settings in each figure caption.
Yes for Fig. 2a and Table 1: split rules, the prediction readout and 100k steps. Fig. 4 uses shared rules and the decoding readout, but 100k steps, not 200k. The 200k runs are the masked ρ experiments of Figs. 2b, 2c and 3a–c. We've added the construction, masking and number of training steps to the caption of every synthetic experiment. We also corrected Appendix I (formerly Appendix F), which wrongly listed 200k steps for the severance experiments.
- Does the recovery in Figure 4 persist under split-rule construction?
We ran Fig. 4 under split rules and the recovery does not persist. This points to a second way in which copying costs composition. Severing source access lowers composition instead of restoring it ( from 67% to 47% at ), while a model trained without source access reaches 99% (Appendix E, Fig. 7c, Table 10). Under shared rules both views use the same composition circuit, which composes from the target's boundary node as it does from the source's. Under split rules the target's boundary node is in a format that this circuit cannot read: a probe for the boundary node trained on the source reads the target's node at ceiling under shared rules but at chance under split rules (Appendix G.4, Fig. 10). With the source's copy always available in training, the target never learns to compose above the fusion boundary. We've updated Section 6, the abstract and the contributions to state this scope.
- Do MLP probes or probes at multiple positions after span closure show the same decline? Can activation patching establish whether target-side representations affect downstream computation?
We repeated the probing with an MLP probe and at later positions, and both show the same decline (Appendix E, Fig. 6a). The MLP probe improves prediction only above the fusion boundary, by 5–7 percentage points of the ceiling. One to four leaves after its span closes, decodes at 60–93% in the intact model, against 99–100% when source access is severed.
We also ran activation patching, in which only selected heads see the source of another sample. Through the two top copying heads alone, the other sample's boundary node reaches 43% in the target, against 29% for two volume-matched heads, 15% at baseline and 72% when every head sees it (Appendix F.4, Table 5). In the other direction, we swap the target's own boundary node into the source's composition circuit, which under shared rules changes the composed parent as much as the source's node does (Appendix G.4, Fig. 10b). These tests show that target-side representations feed the composition of the levels above. We have not shown an effect of the target's own on the model's output, which the Limitations state (Section 9). Under next-token prediction, the levels above the boundary help the target predict its next boundary node only when the source cannot supply it, which would explain why a model trained without source access composes them. With the source available, they add nothing to the loss (Appendix J). Testing their effect on the output therefore needs either the severed model, where the target must rely on them, or an objective that depends on them directly, which we leave to future work (Section 8).
Thanks for rebuttal
Dear Authors,
Thank you for your explanation. I am still confused about the practical function of composition suppression. I think the results you shown cannot prove that copying actually affects downstream predictions in a meaningful way. However, most of my concerns arised in previous review feedbacks have been answered. So, I would like to change my grade to weak reject (from my perspective, the quality of this paper still needs to be refined).
Best.
Thank you for your follow-up
Thank you for reading our responses, for following up and for raising your score. We are glad that most of your concerns have been addressed.
On the practical function of composition suppression: next-token prediction is not affected, by construction. With the source present, the levels above the fusion boundary add nothing to the loss (Appendix J), which is exactly why the objective does not prevent copying. The cost is expected downstream: any zero-shot use of the representation that depends on a higher-level node of the grammar can only draw on what the target has composed. As an analogy, if a high-level node encoded the sentiment of a sentence, a model that copies it from another view, instead of composing it, would be unreliable whenever that view is missing, wrong or only partially informative. We tested this by giving the model a source from another sample, and the target takes on that sample's nodes (61% against 16% at baseline for ), while its own falls from 85% to 67% (Table 4). In the VLM experiments, the same happens at the output: when the caption contradicts the image, the models' answers follow the caption, and blocking the heads that read it restores image-consistent answers (Section 7, Table 14).
We see this as a foundational result for how language models are trained. Our results suggest that pretraining on next-token prediction rewards copying over composition whenever the context already contains the answer, and the loss gives no sign of this. This happens in multimodal data, and likely even more within a single modality, where repeated and paraphrased content is everywhere. If so, the curriculum of pretraining data (when the copying route becomes available) and objectives that put pressure on higher levels (e.g. through SFT) become levers for making models compose instead of copy, and our framework is the first that can measure the effect of each against exact ceilings.
We deliberately kept this paper to what we can show experimentally and avoided speculating beyond it. How training objectives, data recipes and architectures shape the competition between copying and composition, and how composition can be restored, is the subject of follow-up work that this framework makes possible (Section 8).
In light of this, we would be grateful if you would consider whether the current rating reflects the revised paper. Thank you again for your time and engagement.
Response to Reviewer S71Q
Thank you for your careful review and for suggesting specific controls, which led to several additional experiments. We revised the paper following your advice and you may find answers to each point below. For reference, the new experiments we've added, are in Sections 5–7 and Appendices E, F, G and H.1.
Whether the drop comes from reduced training pressure or from competitive suppression depends on how the views are constructed. We've updated Section 6 and Appendix G.4 to separate the two. When the two views use separate rule tables (split rules), reduced training pressure is what happens: the target never learns to compose above the fusion boundary, severing source access lowers composition and only a model trained without source access composes these levels (Table 10). We call this starving. When they use a common rule table (shared rules), the composition circuit is present in the same weights and composes once source access is severed, so the effect happens at inference. Freeing the attention that the heads spend on the source, while keeping the copied content in place, already recovers 82% and 66% of the gain at LS=1 and 2 (Appendix G.3, Table 9). Since the attention weights then sum to more than one, this test probes the mechanism rather than something the model can do. We call this suppression. Most of the recovery sits at the last target position (all of it at LS=1 and 83% at LS=2), which has no next token and receives no gradient (Appendix G.5). Since this position is never trained, the difference there between the intact and the severed pass, which share the same weights, comes from source access at inference rather than from training pressure. At the earlier position, which is trained, the copy delays composition rather than replacing it. In both cases, weakened and restored composition refer to recoverability (Section 3.3).
We agree that Fig. 3c alone does not establish this link. It ranks the span heads, which read the source span of the next boundary node and serve prediction. Blocking them raises the loss but restores no more composition than random heads (Table 8). The causal link to composition comes from a second kind of heads, the copying heads, which read the source's composed version of the node that the target has just closed. Blocking the top two per model restores 45% and 41% of the severing gain at LS=1 and 2. At LS=3 it takes more copying heads. Blocking all cross-view attention in layers 2–4 restores 52–77%, with next-node prediction intact (Section 6, Fig. 4b, Tables 7 and 8).
We ran the main experiments under split rules as well (Appendix E). No target leaf can then be copied from the source as a token, yet the drop above the fusion boundary is present (Fig. 6b), so the drop does not come from local copying. Under shared rules, what the copying heads pass on is the source's composed node, not its token: their output decodes the closed boundary node at 93%, against 44% from the aligned token alone (Appendix F.2, Table 3). Blocking only the aligned offset in every head, where a token copy would be read, restores 18%, 20% and −2% of the gain at LS=1,2,3 (Table 8). Local copying through the aligned position therefore accounts for at most a fifth of the recovery, and for none at LS=3, your example. We discuss the recovery under split rules in our response to Q2 below.