- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether attention surrogates that fit native attention can also predict responses to positional interventions. It compares Cubic Field and Rotary Field while keeping tokens and upstream states fixed. On several Qwen heads, Rotary Field fits native attention better but predicts intervention effects less accurately. The main findings are replicated on another dataset, with additional experiments examining the scope of the results.
Critical strengths
Freezing upstream states and anchoring predictions to native teacher scores cleanly separates positional-response error from native replacement error. Null gives the normalized risk a meaningful reference. Meanwhile, the main Qwen reversals persist across seeds, disjoint corpus evaluations, and gap geometries. Paired document inference and the disclosure of negative controls, small output effects, and unestablished Cubic-over-Null benefits make the bounded empirical claim credible.
Critical weaknesses
- The contribution is limited. Reconstruction does not ensure interventional fidelity is already established by Dyer et al. [1], and attention proxies already undergo causal replacement checks in Hayes et al. [2]. The controlled positional-response protocol and replicated Qwen counterexamples are useful additions, but the paper does not establish which conclusions about existing attention-explanation methods should change as a result. Its main comparison concerns two purpose-built surrogate families. Thus, the demonstrated contribution is largely a specialized empirical case study rather than a substantially new understanding of surrogate faithfulness.
- The reasons for the failures remain unclear. The paper shows that better native fit can lead to worse intervention predictions, but does not clearly explain why this happens or which heads are likely to fail. The frequency analyses do not resolve this issue, limiting the additional insight beyond demonstrating the mismatch.
[1] Interventionally Consistent Surrogates for Complex Simulation Models. NeurIPS 2024.
[2] Explaining Attention with Program Synthesis. arXiv 2026.
Questions for authors
- Does the ranking reversal persist when both native fit and intervention fidelity are evaluated using attention KL?
- Is there evidence explaining why the identified Qwen heads fail while the other tested heads do not?
- Could simply scaling down Rotary Field’s predicted changes match Cubic Field’s performance, with the scale chosen independently of the test data?
- What do the Table 5 results establish about the claim that the mismatch is “not confined to RoPE”, given that no target meets the combined selection-failure criterion?
2 comments
Authors’ Response to Reviewer NLsd (continued)
How far does the T5 result go?
No T5 target met the combined selection-failure rule (0/15). At cutoff 64 and gap 128, the mean native-policy risk exceeded no change by 0.211 and exceeded the bucket-aware surrogate by 0.418. Only seven material heads supported both comparisons, below the required 16. We read this as a localized discrepancy in a model with additive positional bias.
The revised abstract states that none of the 15 T5 targets met the combined criterion (p. 1). The Introduction distinguishes this local T5 discrepancy from the replicated Qwen finding (p. 2). See Section 7, Fig. 4 and Appendix K, Tables 23–24 (pp. 49–50).
What is new relative to Dyer and Hayes?
Dyer et al. show that observationally fitted simulator surrogates can misjudge policy interventions. Hayes et al. select executable attention programs by agreement with held-out attention maps, and they evaluate head replacement through perplexity and task accuracy. Neither study tests whether selecting a surrogate by native attention fit preserves the ranking of predictions for a position shift isolated to one frozen head.
As described in Sections 2–3, we hold tokens, causal masks and upstream states fixed. We add each predicted change to the same native teacher scores (Section 3.2, p. 4), so the zero-change baseline preserves the head’s native attention. On the same three Qwen heads across two corpora, with fresh fits, native KL selected the surrogate family whose predicted score change had a higher error than the no-change baseline (Table 1 and Appendix B). For a method used to explain positional sensitivity, the natural test is whether its predicted change improves on leaving the head at its native attention.
Reference
- Joel Dyer, Nicholas Bishop, Yorgos Felekis, Fabio Massimo Zennaro, Anisoara Calinescu, Theodoros Damoulas, Michael Wooldridge. "Interventionally Consistent Surrogates for Complex Simulation Models." NeurIPS (2024).
- Amiri Hayes, Belinda Z Li, Jacob Andreas. "Explaining Attention with Program Synthesis." arXiv (2026).
Authors’ Response to Reviewer NLsd
We thank Reviewer NLsd for the thoughtful feedback. This comment addresses the first three questions. We address the T5 results and the comparison with prior work in a second, continued comment.
Yes, with simultaneous interval support on L21H13 and L25H12. We reanalyzed 96 archived head–method records from 32 Qwen heads on the same 512 WikiText-103 documents (cutoff 128, gap 64). Cubic (C) and Rotary (R) are fixed three-seed ensembles. For shifted KL, we add each predicted change to the teacher’s native scores. A zero change (0) preserves the native attention.
Table 29 in Appendix M gives the differences in 10−3 nats per valid query row. Positive native C−R favors Rotary. Positive shifted R−C and R−0 favor Cubic and no change, respectively, because lower KL is better. Brackets are simultaneous 95% intervals for all nine contrasts, based on 10,000 paired document-bootstrap draws.
Only the native interval for L24H10 crosses zero. Shifted KL favors Cubic and no change over Rotary on all three heads. By point estimate, 8 of 32 heads reverse under matched KL and 3 of 32 under matched MSE (Appendix M, Tables 29–31, pp. 55–57).
Figure 3 and Appendix I.1–I.2 (pp. 35–36) identify one source of the squared error. On the three historical failure heads, Rotary predicted changes 1.15–1.25 times the teacher’s response norm, with directional cosines of 0.371–0.475. On three selected success controls, its cosines were 0.789–0.883. Expanding the squared error in Eq. 6 shows why this matters. A prediction does worse than zero change when its squared norm exceeds twice its inner product with the teacher’s change. All three failure heads satisfy this condition (Table 16).
We also tested a possible frequency-based explanation. On L24H10, fixing the frequencies to the teacher’s in a separately fitted surrogate reduced the risk from 1.632 to 1.412. This is still above 1, the value for no change (Table 16). Across 32 OpenWebText heads, the native frequency-discrepancy statistic had a rank correlation of −0.107 with Rotary response risk. It also fell below our prespecified threshold for separating the failure heads (AUC 0.759 against a threshold of 0.80; Appendix F, Table 10).
It could. A scale of zero gives the no-change baseline. At cutoff 128 and gap 64, zero had lower point risk than Cubic on two of the three historical heads in the separate WikiText cohort. Cubic had lower point risk than zero on all three original OpenWebText heads (Tables 5 and 16, pp. 21 and 30).
To test a nonzero scale independently, we examined the disjoint 128-document calibration archive. It retains only the squared prediction error and the teacher energy. These two quantities do not determine the prediction norm and the teacher–prediction dot product that a scale fit needs. The independent comparison we can report is therefore the zero-scale endpoint.