- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates whether sparse attention-head patching can repair numerical comparison failures when numbers are written in different formats. The authors study Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct, transferring prompt-final attention-head outputs from a correct donor prompt into a failing target prompt.
Aligned-donor patching improves consistency in both models. However, when the donor operands are swapped, the target tends to follow the donor's answer rather than its own correct answer. A follow-up experiment in Llama, using different answer labels for donor and target, suggests that the transferred information is associated with the selected option position instead of the literal answer token.
The paper also studies the native role of these heads through knockout and operand-level interventions. These experiments find some causal effects on answer margins but the effects are substantially stronger for same-format comparisons. Additional experiments across numerical formats fail their predefined competence criteria.
Overall, the results support a limited answer-position transfer effect, but do not establish a general numerical comparison mechanism or a circuit that is robust across number formats.
Critical strengths
- Transparent reporting of negative results and limitations.
I appreciate that the authors report experiments that did not support their initial hypotheses, rather than presenting only the successful interventions.
- The relabeled-option experiment is a useful control.
The experiment in Section 6 is interesting because it distinguishes transferring an answer position from simply copying a literal answer token. By using (1)/(2) labels for the donor and (A)/(B) labels for the target, the authors show that the patched model tends to select the corresponding donor position despite the different output labels.
Critical weaknesses
- The main contribution is quite narrow and its novelty relative to existing work is unclear.
My biggest concern is that the paper does not establish the numerical comparison mechanism that motivates the investigation.
The aligned-donor experiments initially look promising but the swapped-donor results suggest that the patched heads mainly transfer the donor's already-selected answer. The relabeled-option experiment makes this more precise by showing that the transferred information is associated with the selected option position rather than the literal answer label.
While this is an interesting observation similar distinctions between answer selection, option position and output-symbol binding have already been studied in the mechanistic interpretability literature. The additional novelty here seems to come primarily from applying these interventions to a particular numerical comparison task.
The later experiments also weaken the numerical interpretation. In Section 7, the native knockout fails the preregistered selectivity test and the follow-up in Section 8 reports a much stronger margin effect on digits/digits comparisons than on mixed-notation comparisons. The knockout does not reduce same-format accuracy in the reported replication, despite the substantial change in margins. This makes the functional role of the heads difficult to interpret.
The broader format experiments do not establish generalization either, as most conditions fail the required competence thresholds before the causal interventions can be performed.
I appreciate that the authors acknowledge these limitations. However, after narrowing the conclusions appropriately, I am not convinced that the remaining result provides a sufficiently new insight for ICLR.
- Important methodological details are missing from the manuscript, making the main results difficult to assess independently.
I found the experimental presentation difficult to follow. Several sections refer to stored JSON files, experiment attempts and available records rather than explaining the experimental design in a self-contained way. Appendix A mostly lists artifact paths, which makes it harder to understand the methodology without reconstructing it from the accompanying files.
Most importantly, I could not find a sufficiently detailed explanation of how the six Llama heads and fourteen Qwen heads were originally selected. The paper specifies the final head sets but the discovery procedure, selection criteria, and separation between discovery and confirmation data are not clearly established. Since these head sets are central to the main claims, this information is important for evaluating the sufficiency results.
I also would have liked clearer descriptions of the prompt templates, numerical-pair generation procedure, and the different presentation conditions. The differences in tokenization and execution settings across experiments (for example the standalone versus chat-format BOS handling) further complicate interpretation.
The small evaluation populations are another limitation. Several central results use only 16 numerical pairs and the relabeled-option experiment acknowledges a mismatch between the preregistered inferential unit and its presentation-level decision threshold. Some of the earlier donor results also rely on summary files without trial-level records.
Finally, the reproducibility and limitations sections contain unfinished sentences, which makes the manuscript feel insufficiently polished.
Taken together, these issues make it difficult to evaluate the robustness and generality of the findings.
Questions for authors
- Head selection and reproducibility.
Could the authors clarify how the six Llama heads and fourteen Qwen heads were selected?
- Specificity to numerical comparison.
Given that swapped-donor patching appears to transfer the donor's selected answer, what evidence suggests that these heads are specifically involved in numerical comparison or cross-notation processing? Would the same transfer effect occur for non-numerical binary-choice tasks, such as word-length or alphabetical-order comparisons?
- Generalization of the relabeled-option result.
Can the authors replicate the cross-label experiment in Qwen or with additional answer-label pairs? I would also be interested in whether the effect survives under the original chat-format rendering, since the reported chat-format attempt stopped before patching.