- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors explore when states between transformers can be stitched together using the identity tranformation. They mainly address this question in the context of one and two layer transformers on toy tasks, but they include an experiment on Pythia 70B. They find that without scaffolding during training, the models are not stitchable. They find this at scale as well, ultimately concluding that the "experiments identify conditions under which the identity map becomes usable across models and show that this compatibility is scaffold-induced and coordinate-relative."
Critical strengths
- understanding model stitching is an important precursor for making good brain-computer interfaces.
- the authors interpret their models to better understand the raw interoperability
Critical weaknesses
- The authors mainly consider one and two layer models, which are known to be limited computational models due to lack of layer depth (Biran et al. 2024; Yang et al. 2024; Wang et al. 2024; Grant et al. 2025; Li, Guo & Andreas 2025). It is difficult to discern what to take away from such small-scale experiments that can be applied to models at scale.
- There is a lack of theoretical rigor. The results in the paper are almost entirely empirical. There exists a lot of theoretical work showing what types of representational invariances could exist and be exploited for representational alignment (Williams et al. 2021). Much of the work in the reviewed paper could likely be explained in the context of existing frameworks.
- Unclear or missing methodological detail:
- The paper states that there are "two primary controlled regimes and one compositional bridge task" pg 4 line 206. Then it introduces the "first primary regime" pg 4 line 206 without ever mentioning a "second" regime. Is the second regime the "main regime" from pg 4 line 211?
- What is a bridge task?
- The answer to the motivating question is not very surprising. Given past work on model stitching, it would have been a surprising result if the identity operation was highly successful without constraining the training. It is not surprising that the raw transfer only worked when the models were constrained to a specific coordinate system. And, if I understood correctly, "worked" is generous, given that the recovery of the transfers only ever reached about ~0.75 at best. This work reads like the authors tried to paint negative results in a positive light.
Questions for authors
- The paper states that there are "two primary controlled regimes and one compositional bridge task" pg 4 line 206. Then it introduces the "first primary regime" pg 4 line 206 without ever mentioning a "second" regime. Is the second regime the "main regime" from pg 4 line 211?
- What is a bridge task?
- Why should a general audience have predicted that the identity transform would work?
- What new capabilities, understanding, or efficiencies have been unlocked by the results of this paper?
Response to Reviewer 8Hf9
We thank the reviewer for the detailed feedback. We agree that unconstrained identity-map transfer should not generally be expected. Our contribution is to characterize the conditions under which raw interoperability becomes possible and its limitations.
Scientific insight and theory. Our experiments distinguish functional equivalence, compatibility under a known coordinate transformation, and direct identity-map interoperability. Exact function-preserving gauge transformations alter raw transfer without changing model behavior, while applying the known coordinate correspondence restores compatibility. Scaffold interventions further show that shared output coordinates support substantially more transfer than shared input coordinates under the tested conditions (advantages 0.644 versus 0.003). These controlled findings complement existing representational-invariance theory rather than establishing a new general theorem.
Beyond the shared-decoder intuition. Shared decoder coordinates make partial compatibility plausible but do not guarantee equivalent downstream causal effects. The full donor and decoder-matched surrogate have identical immediate fixed-decoder projections, yet achieve recovery of 0.746 and 0.489, respectively, after downstream recipient computation. Thus, matching decoder-visible information preserves substantial but incomplete transfer, demonstrating that immediate decoder agreement does not fully determine downstream compatibility.
Scale and recovery. We agree that shallow models limit generalization and that recovery around 0.75 is partial rather than complete. Nevertheless, recovery of 0.760 versus a matched corrupted-donor control of 0.076 demonstrates substantial causal transfer under the controlled shared-scaffold condition. The eight-block, approximately 1.6M-parameter experiment retains fixed content-addressing routing, while independently pretrained Pythia-70M models show no reliable cross-seed identity-map transfer at the tested interfaces. We therefore do not claim general LLM interoperability or practical efficiency gains.
Terminology. Associative retrieval is the second primary controlled regime. The "bridge task" refers to held-out two-hop compositional retrieval: given Ea→Eb and Eb→Ec, the model predicts Ec from Ea. Complete two-hop compositions are withheld during training, while their constituent atomic relations remain represented.
Motivation and significance. We do not assume identity-map transfer should have been predicted in general. Our question is which controlled training conditions make direct transfer possible despite independent optimization. The findings identify coordinate constraints supporting transfer, distinguish decoder-visible agreement from downstream causal interchangeability, and demonstrate that functional equivalence does not guarantee identity-map compatibility. These results provide a causal characterization of alignment-free interfaces, not a claim of immediate new capabilities or efficiencies.