- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper investigates whether hidden states transferred between different models through stitching represent answers that have already been computed or intermediate computational states that can support further computation. The authors design a learnable linear mapping that injects the hidden states of a larger Donor Model into a specified layer of a smaller Recipient Model, and train the Stitching Map using the cross-entropy loss on the first token of the correct answer. Experiments are conducted across multiple model families and tasks. The paper shows that Cross-Model Stitching mainly improves performance by transferring answers already computed by the Donor, with no clear evidence that it provides effective intermediate states that enable the Recipient to continue the computation.
Critical strengths
1、The research question addressed in this paper is highly valuable and has significant implications for cross-model information transfer.
2、The experimental design is very comprehensive, including various mechanistic analyses and control experiments, while also covering multiple models and tasks.
3、 The paper provides in-depth analyses of certain mechanisms. For example, the analysis of First-token Conferral and Full-answer Conferral. The paper not only observes a significant performance gap between the two but also investigates this gap through different mapping functions, training objectives, and Hidden States from different computational stages of the Donor, providing valuable experimental findings for understanding the limitations of cross-model information transfer.
Critical weaknesses
1、The paper's layout is not visually appealing, and the visual presentation and organization of information are not very good.
2、The First-token Cross-Entropy training objective may affect the conclusions about Computation Transfer, as this loss function inherently tends to optimize the transfer of answer information.
3、The sensitivity analysis of Stitching Layer selection is insufficient. The paper lacks a systematic analysis of different combinations of Donor Layers and Recipient Layers.
4、The paper lacks a systematic analysis of Stitching performance on the full test set. For example, can Stitching still provide benefits for questions that neither the Donor nor the Recipient can answer correctly? Could Stitching lead to performance degradation on questions that the Recipient could originally answer correctly?
Questions for authors
1、Could the authors consider using an Intermediate-state Alignment or Computation Continuation Objective to train the Stitching Map and compare it with the current Answer-supervised Objective?
2、Could the authors further analyze the impact of Stitching Layer selection on transfer performance? For example, systematically comparing First-token Conferral and Full-answer Conferral across different combinations of Donor Layers and Recipient Layers to verify whether the main experimental conclusions depend on specific Layer Pairs.
3、Could the authors further report the Accuracy of Stitching on the full test set and analyze the performance on different types of samples? In particular, can Stitching provide additional benefits for questions that neither the Donor nor the Recipient can answer correctly? Could Stitching lead to performance degradation on questions that the Recipient could originally answer correctly?
4、Could the authors further improve the paper's layout and presentation of experimental results? For example, presenting some key ablation experiments and control results together to help readers more intuitively understand the relationships between different experiments and how they support the main conclusions.
4 comments
Answer 4 and Additional Clarification
Q4. Could the authors further improve the paper's layout and presentation of experimental results?
We submitted the revised manuscript, where we made substantial revisions to its structure. We re-organized the sections to create a clear narrative arc:
Finding that the stitch confers -> Finding that it confers due to the transfer of a precomputed answer -> Direct comparisons to prior work and potential implications for the circuit universality hypothesis -> Demonstrating the impacts of that transfer of an answer through the full-answer conferral gap, and what it suggests about the generalizability of cross-model stitching.
We also flag the implications of the arguments we make more frequently so as to better convey our points.
Finally, you mention as a weakness that "The First-token Cross-Entropy training objective may affect the conclusions about Computation Transfer," which we wanted to address here.
Our task-supervised map is rewarded for anything that improves the recipient's prediction, whether through answer transfer, computational transfer, or both. The map will transfer whatever gives the recipient the best ability to output the correct answer, which can include a combination of a precomputed answer and computation. Despite this, we find that precomputed answer content captures more or less all of the first-token conferral on Gemma, as shown by the probe bottleneck stitch (Section 5). This finding gives us substantial evidence that the two models' circuits are not aligned enough for computational transfer.
We also have sufficient evidence against the idea that the map never learns computational transfer because it never needs to, since the benefit from answer transfer outweighs anything the recipient would gain from computational transfer. First, our SVD truncation experiment showed that retaining only the top 16 singular directions preserves 99% of the task map's original conferral, while the remaining 2,288 directions on their own recover only around 17% of full-map conferral. This means that the task map has the dimensional capacity to transfer computation to supplement the answer encoding, but does not learn to. We also know that the task map can effectively interact with every part of the recipient's transformer accessible to it, given how it has to encode the answer to correctly interact with every component of the recipient's remaining layers. Additionally, since the donor eventually gets every first token in the unsolvable bin correct, we know there is a correct chain of donor computation that starts at the donor stitch site and is visible to the stitch. Together, these facts mean that the donor's stitch site contains both a precomputed answer encoding and the start of a chain of successful computation, and that the task map can reach the recipient with it. If the task map learned a consistent encoding for even a single step of that chain, it could use its remaining 2,288 dimensions to stitch that step into the recipient and improve the recipient's output through computation, yet it does not. You could also make the claim that answer transfer does so well on Gemma and Qwen that it never needs to learn computational transfer, but computational transfer remains absent on tasks where the map confers as little as a third of what the donor solves, meaning the stitch would have a lot to gain if it could use transferred computation to resolve those remaining errors, yet it does not.
Ultimately, though, the finding that successful stitching can be explained largely by answer transfer maintains the paper's primary implication for interpreting stitching results. This connection is made much more explicit in the revised version of our paper (specifically section 6.2).
response
Thank you for the detailed responses and additional analyses. I appreciate your efforts to address my concerns. However, several concerns remain insufficiently resolved. In particular, I am not convinced that the negative results can be attributed to limited computational transfer rather than limitations of the training objectives or stitching interface. I will therefore maintain my original score.
Response to Remaining Concerns
Thank you for taking the time to read our responses.
In terms of computation being limited by the stitching interface, our paper tests full-answer transfer through a free vector allowed to optimize for each question at the same graft site, which achieves 100% full-answer accuracy on a Gemma subsample. Additionally, our Section 7 evaluates the same stitching protocol on several stronger maps, which show little to no full-answer improvement over a basic linear stitch.
In terms of computation being limited by the training objective, Section 7 also tests intermediate-state alignment, distillation, and whole-answer objectives, all of which stay well below the donor's own full-answer rate. If concerns about the objective remain, would you be able to describe them? Since it is a negative result, there will always be possible reasons other than unaligned circuits for why computation does not transfer (such as the hypothetical we described in our follow-up comment to Reviewer sAPs), but we think what we provide is still fairly substantial evidence towards unaligned circuits.
Finally, this contribution is secondary to our primary claim that the success of prior stitching work cannot be taken as evidence of aligned circuits. For this argument, even if computational transfer were limited by the interface or loss function, that same bottleneck would apply to prior stitching work using similar maps and objectives to those covered in our paper, so our main conclusion would still hold.
Reviewer e9tC Answer 1-3
Hello, thank you for your constructive feedback. Answering each question:
Q1. Could the authors consider using an Intermediate-state Alignment or Computation Continuation Objective to train the Stitching Map and compare it with the current Answer-supervised Objective?
We compare both kinds of objective against the first-token objective in Section 7. Our reconstructive map represents Intermediate-state Alignment, since the map is trained only to match the recipient's own activations at the graft site, and we find that it confers far less than the task map. We additionally sweep across alignment values in our first-token map, which is trained with an added weighted term that rewards similarity to the recipient's own state, and find that, as this alignment term increases, first-token and full-answer conferral decrease, which is consistent with our findings.
For computation continuation, we use distillation to train the map to match the donor's full next-token distribution at every answer position. We also have our whole-answer objective which trains the map with cross-entropy on the full gold answer. Not much of the second or third answer digits are present at the donor site, making computational transfer especially attractive. While they raise full-answer conferral to 0.19 and 0.23, they still sit well below the donor's own 0.42, and further ablations in section 7 reveal that the slight gain comes from improved answer readout of the second and third digits.
Q2. Could the authors further analyze the impact of Stitching Layer selection on transfer performance?
We added this comparison as the appendix section "Layer pairs," where we retrain the first-token task map at every combination of four donor and four recipient layers for Gemma and Qwen. Full-answer conferral stays low at every pair (0.092 to 0.165 on Gemma and 0.023 to 0.070 on Qwen), so the limit on full-answer conferral is not a product of our layer choice.
Q3. Could the authors further report the Accuracy of Stitching on the full test set and analyze the performance on different types of samples?
We added this as the appendix section "The full test set," which splits all 2,000 evaluation problems by which model natively gets the first answer token right. Over the full set, the stitch raises first-token accuracy from 0.59 to 0.89 (Gemma) and from 0.63 to 0.80 (Qwen).
On questions neither model got correct, the stitch reaches 0.59 and 0.57 on the first token, against about 0.10 for the shuffled-donor and constant controls. This fits within our model since the answer is often already present at the donor stitch site even when the donor's final output is wrong, a read which is further supported by the fact that a linear probe fit to the donor stitch site performs similarly well and succeeds on largely the same problems. The full-answer gain is much smaller (about 0.06).
On questions just the recipient gets correct, stitching does degrade accuracy to 0.76 (Gemma) and 0.54 (Qwen), with most of the loss coming from the recipient outputting the donor's wrong first token. On questions both models get correct, it falls only to 0.95 and 0.87.