acceptodds
Under review as a conference paper at ICLR 2027

WHAT DO VISUAL-HEAD DISCOVERY SCORES ACTU- ALLY FIND? ATTENTION SOURCE, NECESSITY, AND RANDOM-CONTROL PITFALLS IN VISION-LANGUAGE MODELS

Abstract

Attention heads that look at the image region a question refers to are increasingly used to steer, ground and interpret vision-language models (VLMs) and to reduce their hallucinations. They are mostly found by ranking heads with an attentionpattern score and validated by steering or masking them. We ask how the choices inside such a discovery score shape the heads it returns, and which of the two validations identifies them. On Qwen3-VL-8B-Instruct we vary the query type, the discovery corpus and the token positions whose attention is read (the attention source) in a full 2 × 3 × 2 design, and test all 12 head-sets for steerability (CERS) and necessity (NCS) against layer-matched random head-sets. (1) The attention source and the query type reshape the selected heads far more than the corpus: changing the source keeps a median of 49 of the top-100 heads, the query 55 and the corpus 84, while two halves of the same data share 96–99. (2) Necessity follows the attention source: head-sets discovered from query-span attention are more necessary than last-token ones (difference 0.083, p=6×10−10), and the two referring + query-span sets lie 17–19 standard deviations above 250 layer-matched random sets (Holm-adjusted p=0.048). The referring + query-span effect replicates across several recent VLMs. (3) Steerability is not specific to the discovered heads: no discovered head-set steers better than its layer-matched random sets (rank p ≥ 0.14), and random sets drawn from the same or from any layer span the discovered sets’ scores. Necessity therefore separates the discovery settings, while steering at this strength does not. We turn these findings into a protocol for reporting visual-head discoveries.

Primary area:
interpretability and explainable AI
Keywords:
Interpretable AI

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Fu4vOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper examines what attention-based visual-head discovery scores select (Section 3.1). It crosses query type, attention source, and discovery corpus in a 2 x 3 x 2 design on Qwen3-VL-8B, then distinguishes steerability (CERS, Eq. 3) from necessity (NCS, Section 3.3 and Figure 8). Changing attention source or query changes the top-100 heads substantially more than changing corpus, relative to a strong split-half stability ceiling (Tables 1 and 20). Query-span discoveries show stronger necessity, whereas none of the 12 discovered sets achieves significant steering superiority under the layer-matched random-set rank test at the default strength (Table 3; p >= 0.14). Validation includes attention-matched controls, masking-location and head-count analyses, a separately audited region-contrast grid, and additional models and benchmarks (Tables 9, 13, 14, 16, 19, and 24). Section 6 turns these findings into reporting recommendations.

Critical strengths
  1. The controlled discovery comparison makes an otherwise ambiguous methodological choice consequential. The crossed design separates attention source, query, and corpus instead of attributing differences between two native discovery pipelines to a single factor (Sections 3.1 and 5.1). Median overlaps of 49/100 for source changes and 55/100 for query changes, versus 84/100 for corpus changes and 96-99/100 within-setting split halves, show that these are not merely sampling fluctuations (Tables 1 and 20; Appendix B.1). The row-restricted masking analysis further links the query-span sets' necessity to question/option processing rather than the final prediction row (Table 5). This is useful evidence that discovery specifications must accompany functional interpretations, not just implementation details.

  2. Necessity receives unusually substantive negative controls, with a cleaner confirmatory test. The two referring/query-span sets exceed all 250 layer-matched random sets, with Holm-adjusted significance across 12 settings (Tables 3 and 10). Image-heavy heads and several generic-damage controls do not reproduce their NCS (Appendix A.4, Table 12). Importantly, the authors acknowledge that the mismatched-image arm does not itself cancel generic damage. The separate equal-sized-cell masking contrast addresses this limitation more directly: all eight query-span/output-token sets score 0.181-0.381 against approximately zero random means, while last-token effects are at most 0.007 (Table 16). Together, these checks support a real distinction between the tested discovery settings, beyond high image attention alone.

Critical weaknesses
  1. The steering result is convincing at the specified intervention strength but does not yet support a broader judgment about steering-based validation. The default bias is beta = 10,000 (Eq. 2), while the pilot already shows substantially different behavior at beta = 3: referring/query-span CERS reaches 0.869 and 0.789, exceeding every sampled control (Appendix A.10, Table 18). Only five random sets and four settings are tested there, so the pilot cannot establish specificity either. Section 6 appropriately qualifies the conclusion, but whether the observed random-control problem primarily reflects extreme intervention strength remains unresolved. This matters because steerability is one of the paper's central contrasts. A properly controlled strength sweep is a fixable experimental gap; the strong-steering result itself remains informative.

  2. The broader validation is less secure than the central experiment, and NCS exposition needs clarification. Tables 7-8 and 23-24 use one random draw per set, precisely the practice criticized for steering in Section 5.2; item-level intervals do not quantify variation across random head-sets. The out-of-domain audit identifies weak oracle performance and above-chance mismatched arms. Appendix C reports that calibration brings four of five problematic mismatched arms near chance, but evaluates head-sets with the uncalibrated pipeline (Table 22). These extensions are therefore suggestive rather than equally strong replication. Separately, Section 3.3 displays the grid estimator, while Table 3 uses matched/mismatched single-image NCS. Figure 8 supplies that definition, and Appendix A.3 analyzes its arms, but Section 3.3 does not clearly introduce both estimators; Appendix E.1 still refers to "Eq.??". These fixable limitations motivate soundness 2 and presentation 2 without outweighing the controlled discovery and clean-grid results supporting weak acceptance.

Questions for authors
  1. [Clarification] Can the Table 18 strength sweep be expanded to enough layer-matched random draws for corrected inference, with strength selection separated from evaluation? Would the conclusions about steering specificity hold at moderate strength (Appendix A.10)?

  2. [Clarification] Do the cross-model necessity separations survive repeated random controls and evaluation restricted to benchmarks passing the audit? Appendix C identifies which datasets drive the pooled separation; could quantitative per-dataset effect sizes and uncertainty establish whether the effect is broad or concentrated (Tables 22-24)?

  3. [Clarification] Could Section 3.3 explicitly distinguish its grid estimator from the matched/mismatched estimator defined in Figure 8 and mapped to experiments in Table 28? Please confirm that the 0.083 factor contrast uses the latter throughout, so its interpretation does not rely on within-item region masking cancellation (Section 5.2).

3 comments

part 1/2

Thank you for the detailed review. You named two gaps as fixable: the steering strength and the single-draw tables. Both are now closed with data. We follow the order of your review.

W1. Is the random-control problem a product of extreme steering strength?

"whether the observed random-control problem primarily reflects extreme intervention strength remains unresolved."

It is resolved: yes, and we can show the mechanism.

Sweep. All 12 settings, 20 random sets each, N=200:

Strength β\beta131010010410^4
Discovered sets above all 20 random sets (of 12)99210

Mechanism. At β=104\beta=10^4 the edit takes all attention away from text. A steered head puts 0.00 on text and 0.999 on the target cell, whether it was discovered or random. Every head-set becomes the same hard pointer.

Test of the mechanism. We steered at β=104\beta=10^4 but kept each head's text attention fixed (four settings, 20 random sets, N=200). Discovered sets then beat all 20 random sets in 4 of 4 cases (original edit: 0 of 4). The spread of random sets falls from SD 0.22 to 0.08. A boost-only edit, which also removes text attention, fails again (0 of 4). A suppress-only edit, which keeps it, works (4 of 4).

The strong-steering result stays in the paper, as you suggested, because that is the strength prior work reports. The paper now adds the explanation and the fix.

W2. Broader validation and NCS exposition

(a) Single random draws in Tables 7–8 and 23–24.

"Tables 7-8 and 23-24 use one random draw per set…"

Rerun with 20 random sets per head-set on Qwen3-VL-8B and Qwen3-VL-4B. The necessity result holds on both (Q2). Gemma-4, Ovis1.5 and InternVL3.5 are not rerun yet and stay labelled as single-draw. Qwen3.5-9B is removed, because no fair control exists when top-100 covers 78% of its heads.

(b) Calibration.

"…evaluates head-sets with the uncalibrated pipeline (Table 22)."

Done. Every new run is evaluated with and without calibration, and on the audit-passing benchmarks alone. The result is the same in all four cases (Q2).

(c) NCS exposition.

"Section 3.3 does not clearly introduce both estimators; Appendix E.1 still refers to "Eq.??"."

Done. Sec. 3.3 now gives NCS-MCQ and NCS-grid as two named equations, and the reference in Appendix E.1 is fixed.

Q1. Can the sweep be expanded to enough random draws for corrected inference, with strength selection separated from evaluation? Would the conclusions hold at moderate strength?

Yes. Selection and evaluation used different data.

  1. β=3\beta=3 was fixed from the old pilot, before any new run.
  2. The test used 120 new items that share no photo with any earlier item.
  3. The test family was fixed in advance: the four query-span settings, Holm-corrected over the four.
Random sets per settingQuery-span sets above all random setsHolm pp, one-sidedHolm pp, two-sided
100 (planned first)4 of 40.0400.079
250 (planned extension)4 of 40.0160.032

With 100 sets, a two-sided Holm test over four settings cannot reach 0.05 even when every random set is beaten. That is why the extension to 250 was planned. On the same items and sets at β=104\beta=10^4, 0 of 4 beat their random sets.

Moderate strength. The original conclusion holds from about β=10\beta=10 upward: 2 of 12 sets beat all 20 random sets at β=10\beta=10, 1 of 12 at β=100\beta=100, 0 of 12 at 10410^4. It reverses at β≤3\beta \le 3 (9 of 12).

part 2/2

Q2. Do the cross-model necessity separations survive repeated random controls and audit-passing benchmarks? Are the effects broad or concentrated?

They survive. 20 random sets per head-set:

ModelBenchmarksCalibratedQuery-span and output-token sets above all 20Their NCSRandom mean
Qwen3-VL-8Ball eightno4 of 40.14–0.220.02
Qwen3-VL-8Baudit-passingyes4 of 40.23–0.360.03
Qwen3-VL-4Ball eightno5 of 50.14–0.210.01–0.02
Qwen3-VL-4Baudit-passingyes5 of 50.13–0.270.02–0.03

Audit-passing means CountBenchQA, MMStar and ImageNet. This rule was fixed before the run.

Broad, with one large contributor. Per benchmark, on Qwen3-VL-8B:

  • The two referring + query-span sets beat all 20 random sets on 8 of 8 and 7 of 8 benchmarks.
  • The two ordinal + query-span sets do so on 6 of 8.
  • CountBenchQA gives the largest share of the pooled difference (39–62%), then MMStar (11–19%).

So the effect is present on most benchmarks, and the pooled number is lifted by CountBenchQA. The new per-benchmark table in Appendix C gives every value with its interval.

Last-token sets stay small out of domain: 0.007 and 0.047, against 0.14–0.22 for query-span sets. One of them (co-ref-last) is above its random sets, mostly because of CountBenchQA. The paper now says that the last-token null holds on ImageNet and is approximate out of domain.

Q3. Could Section 3.3 distinguish the two estimators? Please confirm that the 0.083 contrast uses the matched/mismatched one throughout.

Confirmed. The 0.083 contrast uses NCS-MCQ throughout. Recomputed from the saved outputs: 0.083 [0.057, 0.109], p=6×10−10p = 6\times10^{-10} two-sided, paired over the 400 items. It uses no region masking, so it does not rely on cancellation inside items. Its controls are random sets (about 0), damage-built sets (at most 0.025) and the text-solvable arm (Table 12).

The grid test gives the same contrast on its own: 0.283 against 0.004 (Table 6). Sec. 3.3 now defines both tests, and Table 28 names the test for every result.

Both gaps you named are now closed, and the steering result has a mechanism behind it. We hope this strengthens your support for the paper.

Dear Reviewer Fu4v,

Thank you again for your constructive review. The discussion period ends tomorrow, so we wanted to check whether our response answered your questions.

Both gaps you named are addressed with data. (1) Strength sweep, with strength selection separated from evaluation: β=3\beta=3 was fixed in advance, and on new items all four query-span sets beat all 250 random sets; at β=104\beta=10^4 none does. (2) Repeated random draws: out of domain on Qwen3-VL-8B and Qwen3-VL-4B, with 20 random sets per head-set and with and without calibration, every query-span set beats all of them on necessity. We also found the mechanism: at the answer position, the default edit leaves steered heads with no attention on text, and keeping it restores the separation.

If these results strengthen your view of the paper, we would be grateful if you would consider reflecting that in your score.

Reviewer_8e7hOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how question type, attention source, and discovery data affect which visual heads are found in Qwen3-VL-8B. It tests the heads with steering and masking against random controls. Heads found from query span attention show larger masking effects. At the default steering strength, the selected heads show no clear advantage over random heads.

Critical strengths

The paper makes a useful distinction between heads that can change an answer and heads that the model needs. It tests all 12 settings against 20 to 250 layer-matched random sets.

Critical weaknesses
  1. The main text mixes two NCS definitions. Eq(4) and Fig~1 mask one region of a grid. Tables 3 and 4 instead mask all image attention in the selected heads and compare probability drops on matched and mismatched images. These tests measure different effects. The appendix describes both, but the main text should define them separately and state which results use each.

  2. Most random controls outside the main design use a single draw. Tables 7, 8 and 24 compare each selected set with one random draw. Table 27 uses one random set per model, matched only to ORD-LAST. These comparisons do not measure variation across random sets. The control rule also needs clarification for Qwen3.5 9B. If the pool contains only its 128 full attention heads, selecting 100 leaves just 28, so a separate control set of 100 heads cannot exclude all selected heads. The authors should report the head pool and explain how these controls were built. https://huggingface.co/Qwen/Qwen3.5-9B/blob/main/config.json

  3. The manuscript needs further revision in its writing, presentation of results, and methodological descriptions. There are also smaller presentation errors, including a broken reference (“§??” in line 153) and a stray “?” in line 40. I think that the manuscript needs a more thorough consistency check and proofreading.

Questions for authors
  1. Please define the ImageNet MCQ test used for Table 3 in the main text. How does it relate to Equation (4)?
  2. How were the random controls built for Qwen3.5 9B? Did they exclude the selected heads? Were linear attention layers included?
  3. Do the steering results hold at weaker strengths with at least 20 random sets? Do the cross-model results hold with 20 random draws per head-set?
5 comments

part 1/2

Thank you for the precise review. Below we show that the main results never relied on single draws or on one score, we give the reruns you asked for, and we answer the Qwen3.5-9B question exactly. We follow the order of your review.

W1. Two NCS definitions

"Eq(4) and Fig 1 mask one region of a grid. Tables 3 and 4 instead mask all image attention in the selected heads… These tests measure different effects."

They are two separate tests, and the paper runs both on purpose. Both are in the main text, and they give the same answer:

TestWhat is maskedTablesQuery-span vs last-token sets
NCS-MCQ (Figure 8)The whole image, for the head-set3, 4, 5, 9–15, 19; out of domain: 7, 8, 23, 240.080 vs -0.003
NCS-grid (Eq. 4)One cell of a 2×2 grid6, 16; Figures 1, 40.283 vs 0.004

So no conclusion depends on which test is used. To make the split obvious, Sec. 3.3 now gives two named equations, every caption names its test, and Table 28 has a column for it.

W2. Single random draws, and Qwen3.5-9B

"Most random controls outside the main design use a single draw… If the pool contains only its 128 full attention heads, selecting 100 leaves just 28."

Single draws. The 12-setting design behind the paper's three main results uses 20 to 250 random sets. Single draws are only in the extension tables, and the paper lists that as a limitation. We have now rerun those tables with 20 random sets per head-set. The necessity results hold (see Q3).

One number changed, and it changed in favour of the discovered heads. On the pointing suite (Table 26), the old single draw reached 73% of ORD-LAST's steering effect. With 20 random sets, the random average is -10%, and ORD-LAST beats all 20. The paper now reports the 20-set result.

Qwen3.5-9B. Your arithmetic is right: 100 of 128 heads leaves 28. A fair random control of the same size cannot exist on this model, so we removed it from the paper's claims. No main result used it. Exact details are under Q2.

W3. Writing and consistency

"a broken reference ("§??" in line 153) and a stray "?" in line 40… a more thorough consistency check and proofreading."

All fixed: line 153 (now Appendix E.4), line 40, and "Eq. ??" in Appendix E.1. We also checked every table against the saved outputs. That corrected two ranges in the Figure 7 caption, four recipes in Table 29, and the items per task in Table 24 (40, not 80).

Q1. Define the ImageNet MCQ test used for Table 3. How does it relate to Equation (4)?

The test. One ImageNet photo. The question "Which of the following is shown in this image?" with four options in random order. We read the probability of the correct letter cc. The head-set SS is masked from all image tokens.

NCSMCQ=[P(c∣matched)−P(c∣matched,S masked)]−[P(c∣mismatched)−P(c∣mismatched,S masked)]\mathrm{NCS}_{\mathrm{MCQ}} = \big[P(c \mid \text{matched}) - P(c \mid \text{matched}, S\ \text{masked})\big] - \big[P(c \mid \text{mismatched}) - P(c \mid \text{mismatched}, S\ \text{masked})\big]

The matched photo shows the correct class. The mismatched photo shows a class that is not among the options.

Relation to Eq. 4. Both ask the same question: does the model need SS to answer from the image? Eq. 4 hides one grid cell and compares the target cell with another cell, so generic damage cancels inside each item. The MCQ test cannot cancel damage that way, so the paper controls it directly: random sets score about 0, and head-sets built to cause damage score at most 0.025 against 0.125 (Table 12).

Q2. How were the random controls built for Qwen3.5-9B? Did they exclude the selected heads? Were linear attention layers included?

  • How built: each random head is drawn from the same layer as a selected head, from the heads not selected. If a layer has too few heads left, the draw is capped at what remains.
  • Excluded the selected heads? Yes. Overlap was zero.
  • Linear-attention layers? Not included. Selection and controls use only the 8 full-attention layers (8 × 16 = 128 heads). The linear-attention layers have no attention matrix, so masking and steering cannot act there.

The result: with 100 heads selected, the controls had only 24–28 heads. That comparison is not fair, so the Qwen3.5-9B rows are no longer used. Table 8 now lists each model's head pool and the share that top-100 covers.

part 2/2

Q3. Do the steering results hold at weaker strengths with at least 20 random sets? Do the cross-model results hold with 20 random draws per head-set?

Steering, 20 random sets, all 12 settings:

Strength β\beta131010010410^4
Discovered sets above all 20 random sets (of 12)99210

The paper's result at the default strength holds (0 of 12). Below β=10\beta=10 the picture reverses: discovered sets beat random. A confirmatory test at β=3\beta=3 on 120 new items confirms it: all four query-span sets beat all 250 random sets (Holm-adjusted p=0.032p = 0.032, two-sided).

We also found why. At β=104\beta=10^4 the edit removes all text attention from every steered head (0.00 on text, 0.999 on the target cell). When text attention is kept, discovered sets beat all 20 random sets at β=104\beta=10^4 as well (4 of 4 tested). The paper now states: steering specificity depends on the edit.

Cross-model necessity, 20 random sets per head-set:

Model and dataQuery-span and output-token sets above all 20 random setsTheir NCSRandom mean
Qwen3-VL-8B, eight benchmarks4 of 40.14–0.220.02
Qwen3-VL-4B, eight benchmarks5 of 50.14–0.210.01–0.02
Qwen3-VL-4B, ImageNet (Table 19)referring + query-span: yes0.0540.002
Qwen2-VL-7B, ImageNet (Table 19)referring + query-span: yes0.0290.002

The first two rows hold with and without calibration, and on the audit-passing benchmarks alone. On all eight benchmarks, last-token sets are at least three times smaller (0.047 or less), although one of them (co-ref-last) is above its random sets.

Gemma-4, Ovis1.5 and InternVL3.5 are not rerun yet, because our fast evaluation code is specific to Qwen models. Their rows stay labelled as single-draw.

Every soundness point in your review is now answered with data. We would be grateful if you would reconsider your assessment in that light.

Dear Reviewer 8e7h,

The discussion period ends tomorrow, so we wanted to check whether our response addressed your concerns.

In short: (1) Qwen3.5-9B: your reading was correct. With 100 of its 128 heads selected, the controls had only 24–28 heads, so we removed the model from all claims. (2) Out of domain, we reran with 20 random sets per head-set on Qwen3-VL-8B and Qwen3-VL-4B. All 4 query-span sets on 8B and all 5 query-span and output-token sets on 4B beat all 20 on necessity, with and without calibration. The three other models remain single-draw and are labelled as such. (3) Steering at weaker strength is now tested with 20 random sets on all 12 settings and 250 on the four query-span settings. (4) The two NCS scores are defined separately in Sec. 3.3, and every table names its score.

If anything is still unclear, we would be glad to answer before the deadline. If our response resolves your concerns, we would be grateful if you would consider updating your score.

Reviewer_2fjiOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies how reliable attention head selection is when using attention-based discovery methods. To investigate this, the authors introduce two metrics, steerability and necessity, and systematically examine the effects of token choice, question formulation, and dataset. They find that using a default steering strength does not outperform random selection and that attention source and question type have a larger impact than the dataset.

Critical strengths
  • The paper is a very controlled experimental setup with a reasonable parameter grid and multiple random baselines that strengthen the results.
  • The paper asks an important question: do the heads identified by attention scores actually help the model understand images? This matters because researchers use these heads to explain and control the model’s answers.
Critical weaknesses
  • My main concern is that the evidence needed to interpret the central steering result is incomplete. Although the authors appropriately restrict their main finding to the default intervention strength, Table 18 shows substantial separation between discovered and random heads at weaker steering. Furthermore, the necessity score is not consistently specified: Figure 1 and Eq. (4) define a within-image region contrast, whereas Table 3 and Figure 8 use matched/mismatched-image masking. These experiments have different control assumptions, and Appendix A.4 acknowledges that the latter cannot cancel generic damage. Each headline necessity claim therefore needs to be explicitly linked to its score and supporting controls.
  • Park et al. (2026) already tested different attention sources and checked whether the selected heads affect the model’s performance. The main additions here seem to be comparing steering with necessity and using stronger controls. This is less new than some wording suggests, and I question whether it is enough for acceptance. A deeper analysis of what causes the steering effects, particularly in random heads, could make the contribution stronger.
  • The writing is very dense and often reads like a highly compressed enumeration of facts, making the argument difficult to follow. There are several citation and cross-reference errors, for example in lines 040 and 153. Outside the “Related Work” section, several in-text citations interrupt the sentence structure and reduce readability; see lines 163, 168, and 185 for examples.
Questions for authors
  • Does the weaker-steering advantage in Table 18 persist with more random controls, and would it change the main conclusion?
  • Which necessity score supports each main result, and how does each control for damage?
3 comments

Thank you for the review. Your two soundness concerns are answered by results that are in the main text of the submitted paper. We point to them below, and we add the experiment you asked for. We follow the order of your review.

W1. Steering at weaker strength

"the evidence needed to interpret the central steering result is incomplete… Table 18 shows substantial separation between discovered and random heads at weaker steering."

Two facts about the submitted paper first:

  • Its steering claim is for the default strength only. The abstract says "steering at this strength does not."
  • Table 18 could not show a real separation. It has 5 random sets, so its smallest possible pp is 1/6. The paper says this in Appendix A.10.

We have now run the full test: all 12 settings, 20 random sets each, the same sets at every strength.

Strength β\beta131010010410^4
Discovered sets above all 20 random sets (of 12)99210

Then a confirmatory test: β=3\beta=3 fixed in advance, 120 new items, 250 random sets.

Query-span setCERSRandom sets: mean (highest of 250)Random sets at or above
cm-ref-qsp0.802-0.152 (0.400)0
co-ref-qsp0.7160.067 (0.532)0
co-ord-qsp0.635-0.060 (0.345)0
cm-ord-qsp0.5360.065 (0.527)0

Holm-adjusted p=0.032p = 0.032, two-sided. On the same items and random sets at β=104\beta=10^4, none of the four beats its random sets.

Both results stand together: discovered heads steer far above random at low strength, and the default strength hides it.

W2. Two necessity scores

"Figure 1 and Eq. (4) define a within-image region contrast, whereas Table 3 and Figure 8 use matched/mismatched-image masking… Each headline necessity claim therefore needs to be explicitly linked to its score and supporting controls."

These are two separate tests, not one score defined twice. Both are in the main text, and they agree.

  • NCS-MCQ (Table 3, Figure 8): mask the head-set from the whole image. Score = drop when the answer is in the image, minus drop when it is not.
  • NCS-grid (Eq. 4, Table 6): hide one cell of a 2×2 grid. Score = P(correct) with another cell hidden, minus P(correct) with the target cell hidden. Both arms hide a cell of the same size, so generic damage cancels inside each item.

Each headline claim, its score and its controls:

ClaimScoreControl for damageWhere
Query-span sets are more necessary than last-token sets (0.083, p=6×10−10p = 6\times10^{-10})MCQRandom sets score ≈ 0Table 3
Same contrast, damage cancelled inside each itemgridSame-size cell in both arms; 100 random setsTable 6: 0.283 vs 0.004
Referring + query-span sets beat randomMCQ250 random sets (17–19 SD); damage-built sets ≤ 0.025; text-solvable arm 0.112 vs ≤ 0.010Tables 10, 12
Necessity acts at the question rowsMCQSame heads, only the masked rows change: 0.140 vs 0.000Table 5
Necessity holds out of domainMCQ design20 random sets per head-setNew: 0.14–0.22 vs 0.02

Appendix A.4 says the MCQ mismatched arm cannot cancel damage. That is exactly why the paper adds the controls in column 3, and repeats the main contrast with the grid test, where damage cancels by design.

To make the two tests impossible to confuse, Sec. 3.3 now gives them as two named equations, and every table names its score.

W3. Novelty relative to Park et al. (2026)

"Park et al. (2026) already tested different attention sources… The main additions here seem to be comparing steering with necessity and using stronger controls."

Park et al. is concurrent work (arXiv, 27 Aug 2026), and it does not contain our main results.

  1. No last token. They compare input tokens with output tokens. They never test the last prompt token, which is what Gaze Heads use. Our main necessity result is exactly that contrast: query-span heads are needed, last-token heads are not.
  2. No steering. They validate by masking only. We test steering and masking on the same heads, with the same readout.
  3. No explanation of steering. We show that the default steering edit removes all text attention, and that discovered heads separate from random ones once this is fixed (W4).
  4. No query or corpus comparison. We show that the query type changes the head-set as much as the attention source (55 vs 49 of 100 heads kept), against a noise ceiling of 96–99.
  5. No location of the effect. Masking only at the question and option rows gives the full effect (0.140). Masking only at the last row gives none (0.000).
  6. One random baseline. They report one, with no spread. We use 20 to 250 random sets.

One sentence in our Sec. 1 said that Park et al. do not vary their score. They do (input vs output tokens), and we corrected it.

W4. What causes steering in random heads

"A deeper analysis of what causes the steering effects, particularly in random heads, could make the contribution stronger."

We found the cause: the boost term of Eq. 2.

At β=104\beta=10^4 a steered head puts 0.00 of its attention on text and 0.999 on the target cell. This is the same for discovered and random heads. Without steering, these heads put 0.84–0.98 on text. So every head-set turns into the same hard pointer, and any set can move the answer.

We tested this with four edits at β=104\beta=10^4 (four settings, 20 random sets each, N=200):

EditText attention kept?Discovered sets above all 20 random setsSpread of random sets (SD)
Original Eq. 2No (0.00)0 of 40.22
Boost onlyNo (0.00)0 of 40.22
Suppress onlyYes (0.98)4 of 40.04
Redirect that keeps text attentionYes (0.95)4 of 40.08

When text attention is kept, cm-ref-qsp reaches 0.813 against a random mean of -0.100.

The paper had already ruled out depth (random sets in the same layers differ by up to 1.1) and image attention (R2=0.03R^2 = 0.03).

W5. Writing and references

"The writing is very dense… citation and cross-reference errors, for example in lines 040 and 153… see lines 163, 168, and 185."

All fixed: lines 040 and 153, "Eq. ??" in Appendix E.1, and the citations at lines 163, 168 and 185. Sec. 3.3 and Sec. 5.2 are rewritten in short paragraphs that start with their claim.

Q1. Does the weaker-steering advantage persist with more random controls, and would it change the main conclusion?

It persists: 9 of 12 sets beat all 20 random sets at β=1\beta=1 and β=3\beta=3, and all four query-span sets beat all 250 on new items.

It sharpens the conclusion. The paper said the default strength does not separate discovered from random heads. That still holds on this test (0 of 12). We now add why, and that the separation is clear once the edit keeps text attention. The revised claim is: steering specificity depends on the edit.

Q2. Which necessity score supports each main result, and how does each control for damage?

See the table under W2. In one line: the main contrast is shown with both tests; the grid test cancels damage inside each item; the MCQ test uses random sets, damage-built sets and a text-solvable arm.

Every soundness point in your review is now answered with data, most of it from the submitted paper. We would be grateful if you would reconsider your assessment in that light.

Dear Reviewer 2fji,

The discussion period ends tomorrow, so we wanted to check whether our response addressed your concerns.

In short: (1) The steering-strength test you asked for is done. With 20 random sets, 9 of 12 discovered sets beat all of them at β=3\beta=3, and all four query-span sets beat all 250 random sets on new items (two-sided, Holm-adjusted over the four: p=0.032p=0.032). At the default β=104\beta=10^4, none does. (2) We found why: at the answer position, the default edit leaves a steered head with no attention on text (0.00), for discovered and random heads alike. With an edit that keeps text attention fixed, all four tested sets beat all 20 random sets at β=104\beta=10^4. (3) The two NCS scores are now defined as two named equations, and the main necessity result holds under both (0.080 vs −0.003; 0.283 vs 0.004).

If anything is still unclear, we would be glad to answer before the deadline. If our response resolves your concerns, we would be grateful if you would consider updating your score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers and the AC

We thank the reviewers. Three questions decide this paper. For each one, the answer is already in the submitted paper, and we have now confirmed it with the experiments the reviewers asked for. Changes in the revised PDF are in blue.

1. Is the necessity result sound, given two NCS scores?

Yes. The paper does not use one score in two inconsistent ways. It runs two separate tests, both in the main text, and they give the same answer.

TestWhereQuery-span setsLast-token sets
NCS-MCQ: mask the head-set from the whole imageTable 30.080-0.003
NCS-grid (Eq. 4): hide the target cell vs another cellTable 60.2830.004

In the grid test, both arms mask a cell of the same size, so generic damage cancels inside each item. For the MCQ test, the paper rules out generic damage directly: head-sets built to cause damage score at most 0.025, against 0.125 for the discovered set (Table 12).

So the necessity result does not depend on which score one prefers. To make this impossible to miss, Sec. 3.3 now gives the two scores as two named equations, and every table names its score.

2. Does the steering result hold at weaker strength?

The paper made its claim for the default strength only, and said so in the abstract ("steering at this strength does not"). The reviewers asked what happens below it. We ran that test on all 12 settings with 20 random sets each.

Strength β\beta131010010410^4 (default)
Discovered sets above all 20 random sets (of 12)99210

A confirmatory test at β=3\beta=3, fixed in advance, on 120 new items: all four query-span sets beat all 250 random sets (Holm-adjusted p=0.032p = 0.032, two-sided). On the same items at β=104\beta=10^4, none does.

We also found the reason. At β=104\beta=10^4 the steering edit takes all attention away from text: a steered head puts 0.00 on text and 0.999 on the target cell, whether it was discovered or random. When we keep text attention fixed, discovered sets beat all 20 random sets at β=104\beta=10^4 too (4 of 4 tested; before: 0 of 4).

This makes the paper's message stronger, not weaker. The default steering edit used in prior work cannot tell discovered heads from random ones. We now show why, and how to fix it.

3. Are the random controls strong enough?

The 12-setting design behind the paper's three main results uses 20 to 250 random sets per setting. Single draws appear only in the extension tables, and the paper lists this as a limitation. We have now rerun those tables with 20 random sets.

  • Out of domain, Qwen3-VL-8B: all 4 query-span sets beat all 20 random sets on necessity (0.14–0.22 against 0.02).
  • Qwen3-VL-4B: all 5 query-span and output-token sets beat all 20 (0.14–0.21 against 0.01–0.02).
  • Both hold with and without calibration, and on the audit-passing benchmarks alone.

Updates to the paper

  • Steering claim. It now reads: steering specificity depends on the edit. At the default strength the verdict even changes with the task (0 of 12 sets beat random on the ImageNet grid; 3 of 4 do out of domain), which is one more reason not to trust it.
  • Pointing suite. The single-draw "73%" figure is replaced by 20 random sets. ORD-LAST beats all 20 (0.138 against a random mean of -0.014).
  • Qwen3.5-9B. Removed. Top-100 covers 78% of its attention heads, so no fair random control exists. No main claim used this model.
  • Statistics. Table 3 reports one-sided rank pp, now stated (two-sided Holm for the two referring + query-span sets: 0.096). For those two sets Table 3 uses the smaller of two saved runs (0.116 and 0.126; the other run gives 0.162 and 0.142), so its values are conservative.
  • Presentation. All broken references and citations are fixed. Sec. 3.3 and Sec. 5.2 are rewritten in short paragraphs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.