- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper examines whether functional labels assigned to attention heads, such as induction, retrieval, and truthfulness, remain meaningful across models, training stages, and interventions. Through comparisons across models, analyses of pretraining trajectories, and head-level interventions, the paper argues that these labels capture certain coarse structural and task-related regularities, but do not reliably identify specific heads, establish exclusive functions, or guarantee performance improvements through amplification.
Critical strengths
The paper investigates an interesting and relevant problem in mechanistic interpretability: what can legitimately be inferred from a functional label assigned to an attention head? Examining whether labels established using a particular model, detector, and intervention remain meaningful in other settings is a valuable research direction. The distinction between coarse structural regularities and the identities of individual heads is potentially useful for interpreting and reusing head-level analyses. Bringing together questions about distribution, attribution, formation, and control could also provide a useful framework for studying these issues systematically.
Critical weaknesses
- The organization and formulation of the research questions need substantial improvement.
Although the paper addresses several interesting questions, their motivation and precise formulation are not sufficiently developed. The relationship between each research question, its experimental setup, and the resulting conclusion is often difficult to follow. In particular, the paper should clearly establish what property is expected to transfer, how that transfer is measured, and what evidence would support the corresponding claim. Organizing each section around an explicit question, experimental rationale, and conclusion would make the argument easier to assess.
- The model coverage is limited relative to the breadth of the conclusions.
The main model comparison includes Qwen2.5-7B, Llama-2-7B, Llama-3.1-8B, Mistral-7B, and Pythia-6.9B. This provides useful coverage across model families, but the models are concentrated in a narrow parameter range of approximately 7–8B. Since the paper’s central contribution is an empirical characterization of the transferability of head-level labels, the dependence of its findings on model scale remains insufficiently explored. Comparisons across substantially different model sizes would strengthen the conclusions. Alternatively, the paper should more explicitly delimit its claims to the evaluated settings. The scope of conclusions drawn from the two Pythia pretraining trajectories should likewise be distinguished from conclusions supported by the broader model comparison.
- Too much essential evidence is deferred to the appendix.
The main paper does not present enough of the evidence needed to assess its central claims, with too many important result tables relegated to the appendix. The authors should select the most informative tables and incorporate them into the main text, particularly those supporting the principal comparisons across models and the intervention results. Each table should be directly connected to a research question and accompanied by a clear interpretation. The appendix is appropriate for exhaustive numerical results, additional breakdowns, and implementation details, but the main paper should contain enough evidence for readers to evaluate its core arguments.
- The pretraining configuration underlying Figure 3 is insufficiently clear.
Reporting training steps alone does not provide enough context to interpret the timing of head formation or compare the training trajectories. The paper should specify the effective global batch size, sequence length, and corresponding number of training tokens, and clarify which training settings are shared across the trajectories. Providing cumulative training tokens alongside optimization steps would make the results easier to interpret. This is especially relevant to the claim that detector strength saturates thousands of steps before top-head membership stabilizes: the reader needs sufficient training context to understand the magnitude and significance of that separation.
- The experimental choices require clearer justification.
Beyond the missing configuration details, the paper does not sufficiently explain why its experimental settings provide appropriate tests of the stated hypotheses. For each research question, the authors should justify the chosen detectors, head-selection criteria, interventions, and evaluation measures, and explain how the resulting observations support the corresponding interpretation. Detector overlap, the causal contribution of a selected head set, and the effect of output amplification address different questions and warrant different conclusions. Without a clearer connection between the experimental design and the claims, it is difficult to assess whether the broader interpretations are adequately supported.
Questions for authors
None, it is just not a conference-level paper.