acceptodds
Under review as a conference paper at ICLR 2027

Localizing and Disrupting Entity Identification in Vision-Language Models

Abstract

Shown a photograph of a famous person, brand, or landmark, a vision–language model (VLM) can report facts that no pixel contains: it first recognizes the entity and then retrieves what it has memorized about it. The same step lets memory override the image. Which components of a VLM support recognition of familiar entities? We answer this question with two tools: familiarity tiers, derived from the model’s own behavior, which separate entities it knows robustly, knows weakly, or does not know; and an exhaustive sweep of attention-layer knockouts, scored against perceptual control questions on the same images. Across four open VLMs (Qwen2.5-VL, Qwen3-VL, Gemma-4, and LLaVA-1.5), recognition loss exhibits prominent peaks in mid-decoder attention layers, with selectivity relative to perceptual controls varying across models. In Qwen2.5-VL and LLaVA-1.5, where the band is one or two layers wide, ablating it substantially lowers recognition accuracy while having smaller effects on answers to the perceptual control questions: an entity-recognition gate. In Gemma-4 and Qwen3-VL the band spans five layers, and ablating it also changes a substantial share of the control answers. An in-depth analysis of Qwen2.5-VL shows that its gate generalizes across faces, logos, landmarks, and artworks; that ablating it lowers identification of highly-known-tier edited faces from 58% to 15% while reports of the same images’ attributes are unchanged and general VQA accuracy drops by 1–3 points; and that ablating five of its attention heads lowers logo recognition from 96% to 3%. Finally, the gate influences behavior when memory and image disagree: ablating it reduces memory-driven identification errors on VLMBias by up to 24 points, and on people edited to wear a doctor’s coat it shifts the model’s answer from their real occupation toward “doctor”.

Primary area:
interpretability and explainable AI
Keywords:
vision-language models, mechanistic interpretability, entity recognition, knowledge recall, attention head ablation, causal intervention, visual grounding, language priors, bias mitigation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_va2TOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates which components of vision-language models support the identification of familiar entities. The authors define familiarity tiers using recognition under image perturbations, then scan attention-layer windows by replacing head outputs with their mean over the input sequence. Experiments cover four models and four entity domains: faces, logos, landmarks, and artworks. The authors find that interventions in middle decoder layers substantially reduce identity responses, with varying effects on perceptual controls. Further analysis of Qwen2.5-VL identifies five attention heads whose joint ablation sharply reduces logo recognition. The paper also examines how the intervention changes responses when visual evidence conflicts with memorized knowledge.

Critical strengths
  1. The experiments cover four model families and several entity domains. The systematic layer sweeps provide a useful account of where identity responses are sensitive to intervention. The authors also report the less selective results in Qwen3-VL and Gemma-4, which helps establish the limits of the findings. (Sections 3 and 4; Figure 3.)
  2. The edited-face experiment provides a useful control on the same images. In the highly known tier, identity preservation falls from 58.3% to 14.9%, while 99.0% of attribute responses remain unchanged. This helps rule out complete model failure as the explanation for the recognition drop. (Section 4.2; Appendix F; Figure 9.)
Critical weaknesses
  1. The reference lines in Figure 3 overlap the annotations. The vertical dotted lines run through the layer labels at the top of the panels. Please move the labels or shorten the lines so that they do not intersect. This is a minor presentation issue; the methodological concerns below determine my assessment.

  2. The recognition measure conflates identification with willingness to answer. The main probes require a known or unknown prefix, and the parser treats responses that do not begin with known as unknown. A decrease in the reported recognition rate can therefore reflect changes in confidence, abstention, or output compliance as well as entity identification. Appendix D describes some of these response changes, but does not establish whether the model has lost the ability to distinguish the entity. The direct naming experiment on edited faces helps address dependence on the prescribed format, although it still allows an unknown response. The discussion acknowledges this limitation, but it remains central to the proposed recognition-gate interpretation. (Section 2; Appendices B, D, and J.)

  3. The localization may depend on the intervention itself. Mean replacement changes each selected head's output at every input position, including both visual and text positions. It also retains an input-dependent average contribution. This operation can alter information routing beyond the processing of entity identity. A recognition-loss peak could therefore identify layers that are particularly sensitive to this replacement procedure. I did not find a comparison establishing that the same localization holds under a different intervention. Without that check, the functional interpretation of the selected window remains uncertain. (Section 2, Equation 2; Appendix B.)

  4. The controls are too coarse to establish the claimed selectivity. Most control probes concern color or the presence of text and circles. Preserving these answers does not rule out damage to the finer visual features needed to distinguish faces, logos, or landmarks. The edited-face controls mainly test color reporting and leave this possibility open. The evidence also varies considerably across models: control responses change on 13% to 31% of examples in Gemma-4 and 35% to 46% in Qwen3-VL. The authors acknowledge these differences, but the results provide limited support for a shared selective gate across all four models. (Section 4.1; Figure 3; Appendices D and F.)

  5. The five-head result lacks independent validation of selectivity. The heads are selected according to the logo recognition loss caused by individual ablations, and their joint effect is reported on the same logo collection. The paper does not describe a separate selection and evaluation split. More importantly, Figure 4 reports recognition performance without corresponding perceptual-control results for this five-head intervention. The controls for ablating the entire layer window cannot establish the selectivity of the five selected heads. The drop from 96.1% to 3.1% demonstrates strong dependence of logo responses on this set, but its selectivity and transfer to other entities remain untested. (Section 4.2; Figure 4; Appendix G.)

Questions for authors
  1. On the existing entity images, does the intervention still cause a large accuracy drop in a forced-choice identification task that does not allow abstention? Please randomize the answer order. This would help determine how much of the reported effect comes from impaired entity discrimination rather than a change in the threshold for answering.

  2. Does the same layer window emerge under an alternative intervention that keeps visual and text positions separate? Evidence that the localization persists would help address the possibility that the current result depends on sequence-wide mean replacement.

  3. Does the intervention preserve fine-grained visual discrimination when no entity name is required, for example in an image-matching task? Such a control would help distinguish disruption of identity-related visual processing from disruption of knowledge retrieval or identity reporting.

  4. Were the five heads selected and evaluated on separate samples? With the head set fixed, please report recognition loss and perceptual-control changes on images excluded from selection. Any existing evaluation in other entity domains would also help assess whether this is a stable functional component.

1 comment

Thank you for highlighting the systematic sweeps, edited-face controls, and reporting of differences across models.

Experiments 1–5 are described in our general comment.

Addressing the weaknesses:

  1. We will move the layer labels above the panels and shorten the reference lines so they no longer overlap.

  2. We agree that the format and option to abstain complicate the interpretation; While appendix F does not fully resolve this, we are working on forced-choice identification without abstention. We will post the results once ready and verified. For now, we describe the effect as a loss of explicit naming, without assuming loss of identification ability.

  3. In the complete sweeps (Exp 2), zero ablation at all positions and mean ablation at text positions both recover layers 16–17 on faces and logos. Zero ablation there gives 100% naming loss with 13% control change on faces and 5% on logos. Final-position interventions peak later for faces, and connection blocking can peak elsewhere.

  4. We added face matching using different photographs, visually similar negatives, and no names. Accuracy is 88.2% intact, 88.7% with five-head ablation, and 84.9% with layer ablation (Exp 1). This harder control supports the naming result, especially for the five heads; we also report the larger layer-ablation drop for highly known people. Model differences remain at matched widths (Exp 5). We will limit selectivity claims to the tested models and abilities, without claiming a shared selective gate.

  5. On 235 new brands excluded from selection, the fixed heads give 100% naming loss among 160 initially correct cases, with 1.7% control change (Exp 3). The largest loss among 25 random sets with the same layer allocation is 15.6%. We also test the same heads on faces and landmarks, with direct visual controls, and add face matching in Exp 1.

Addressing the questions:

  1. These experiments are still in progress, and we will post the results once ready and verified.

  2. Text-only mean ablation recovers layers 16–17, with naming loss / control change of 85% / 12% on faces and 100% / 4% on logos. Visual-only ablation at that window gives only 5% and 2% naming loss, respectively. Final-position interventions peak at layers 19–20 for faces. Exp 2 therefore supports the same window under text-only intervention while showing that the positions we target matter.

  3. Five-head matching accuracy is 88.7%, compared with 88.2% intact. Layer ablation gives 84.9%, with a larger drop for highly known people. Exp 1 gives the setup and results. This supports a separation between naming and matching, most clearly for the five heads.

  4. The submission used the same logos for selection and evaluation. We now test the original heads on new brands and other domains without reselecting them. Naming loss / control change is 91% / 7.7% on landmarks and 58% / 10.8% on faces. Exp 3 also reports a separate selection/evaluation split and comparisons with random head sets.

Reviewer_qFRCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the internal components that support identification of specific entities in vision-language models. The authors assign entities to familiarity tiers based on recognition under masking, cropping, and downsampling. They then mean-ablate consecutive attention layers and compare recognition loss with changes in answers to perceptual control questions on the same images. The experiments cover four models and four entity domains: faces, logos, landmarks, and artworks.

The strongest results come from Qwen2.5-VL-7B. Ablating layers 16–17 substantially reduces entity naming while largely preserving some visual attribute judgments. Ablating just five attention heads within these layers reduces recognition from 96.1% to 3.1% on a set of logos. The paper also examines how this intervention changes responses when visual evidence conflicts with prior knowledge.

Critical strengths
  1. The research question is valuable, and the experiments use direct causal interventions. The relationship between identifying specific entities and judging visual attributes is worth investigating. Compared with analyses of attention patterns or hidden representations alone, systematic ablations provide more direct evidence that recognition behavior depends on particular components.

  2. The behavioral dissociation in Qwen2.5-VL is compelling. On edited face images, identity preservation in the highly known tier falls from 58.3% to 14.9%, while the authors report unchanged attribute answers on 99.0% of the edited images. The free-form naming prompt in Appendix F also provides evidence that the phenomenon is not limited to a single structured output format.

  3. The authors evaluate capability losses and report unfavorable results. Table 3 includes large-scale general capability evaluations with confidence intervals for paired differences. The main text explicitly acknowledges substantial changes in perceptual control answers for Gemma-4 and Qwen3-VL. The appendix also reports declines on counting tasks, helping clarify the intervention’s limitations.

Critical weaknesses

1. The mechanistic interpretation of the “entity-recognition gate” remains ambiguous.

The experiments directly measure whether the model reports the correct entity name; they do not independently establish whether it can still distinguish the depicted entity. The main probe counts recognition as successful only when the model outputs known followed by a correct name. Consequently, an inability to recognize the entity, reduced confidence leading to an unknown response despite retained identity information, and a failure to generate the name all count as recognition failures. For example, a model that no longer names a person but can still select the correct name from several candidates differs meaningfully from one that cannot identify the person at all. The current metric does not distinguish these outcomes. Appendix F removes the structured known/unknown output requirement but still permits unknown, leaving this ambiguity unresolved.

Two controls would help. First, provide candidate entity names and require the model to select one by returning only its option index. This would reduce the influence of abstention and free-form name generation. If selection performance remains relatively strong after ablation while free-form naming deteriorates, the original results would not establish an equally large loss of entity discrimination. Second, supply the correct entity name in the prompt and ask factual questions about it, testing whether knowledge retrieval remains intact when visual identification is bypassed. Impaired image–name matching alongside relatively preserved factual question answering given the name would more strongly support a disruption of visual identity recognition rather than subsequent general knowledge retrieval.

The authors acknowledge these unresolved mechanisms in the conclusion. This does not undermine the evidence that the identified components contribute to entity naming, but it limits the stronger interpretation that they gate recognition or memory access. These controls would not fully identify the mechanism on their own, but they would provide more discriminating evidence.

2. Robustness of the localization to the intervention method has not been established.

Equation (2) replaces each head’s output at every input position with its mean over the current input sequence. This removes variation across positions while retaining an input-dependent contribution at each position. Its effects may therefore depend on the proportion of visual tokens, prompt length, and sequence composition. The intervention can reveal component dependencies, but using it alone leaves open how much the localization depends on this particular replacement operation.

The paper does not report localization results under alternatives such as zero ablation or mean replacement using independent reference inputs. It also does not separately test interventions at visual positions, text positions, and the final input position. Different interventions need not produce effects of identical magnitude, but persistence of the main localization under reasonable alternatives would strengthen the conclusion.

3. The five-head result lacks independent validation and an assessment of selectivity at the head level.

Appendix G describes scoring individual heads in layers 16–17, selecting the five that cause the largest recognition losses, and evaluating their joint ablation. However, it does not specify whether head selection and evaluation use separate sets of entities, leaving potential selection bias in the effect estimate unresolved. Figure 4 also primarily compares recognition rates without reporting the effects of these five heads on perceptual controls or general capabilities. Selectivity observed for an intervention on whole layers cannot automatically be attributed to the selected subset of heads.

Figure 4 already includes a random five-head ablation control. Reporting the distribution across multiple random head sets, validating the selected heads on held-out entities, and evaluating their effects in other entity domains and perceptual tasks would make this result more convincing.

4. The relationship between variation in intervention effects and model structure remains unclear.

Selectivity is clearest in Qwen2.5-VL, while the other models show substantial differences. In Table 7, the selected Qwen3-VL window produces an average recognition loss of 69% alongside changes in 42% of control answers; the control change rate for Gemma-4 is also substantial at 25%. For LLaVA, ablating layer 15 produces only 8% recognition loss on logos, much less than in the other domains.

The authors acknowledge these differences in the abstract and Section 4.1, but their origins remain unexplained. Are intervention effects and their selectivity systematically related to parameter count, network depth, or the way visual information enters the decoder? Empirically clarifying these relationships could help explain how different architectures organize entity recognition and visual perception. It could also inform architecture design and the choice of architecture for tasks that depend on entity knowledge versus those that emphasize visual evidence.

5. The mechanistic contribution relative to recent work on knowledge priors needs clarification.

The authors cite relevant prior work and distinguish their contribution primarily through differences in the task studied. Ortu et al. identify a small set of attention heads that mediate conflicts between visual evidence and prior knowledge. Lietzow et al. further report that ablating a small set of heads substantially changes answers driven by prior knowledge while having much smaller effects on answers grounded in visual evidence. These findings are closely related to the mechanistic interpretation advanced here. See Ortu et al., “When Seeing Overrides Knowing: Disentangling Knowledge Conflicts in Vision-Language Models,” and Lietzow et al., “Vision-Default, Prior-Override: Causal Mechanisms of Perception-Knowledge Conflict in Vision-Language Models.”

The focus on specific entity identification, familiarity tiers, and matched perceptual controls provides distinct empirical value. What remains unclear is whether the localized components exhibit selectivity for entity recognition beyond the previously identified mechanisms supporting knowledge priors. Comparing component overlap in a shared model or testing the components across tasks would help clarify this relationship and the paper’s mechanistic contribution.

Questions for authors
  1. Does ablation still substantially reduce performance on entity selection or image–name matching tasks that do not permit unknown responses?
  2. When the correct entity name is supplied in the prompt, is retrieval of related facts preserved? Could this control help distinguish disruption of visual identity recognition from disruption of knowledge retrieval?
  3. Does the localization to layers 16–17 persist under alternative replacement methods or intervention positions? Have the selected five heads been validated on held-out entities, with recognition impaired and perceptual controls relatively preserved?
  4. How do the identified heads relate to those reported in prior work on conflicts between visual evidence and knowledge priors? Could component overlap or interventions across tasks clarify their similarities and differences?
1 comment

Thank you for recognizing the causal evidence, the edited-face results, and our reporting of the method’s limitations. We especially appreciate the suggested experiments and feel that it will strengthen the paper.

Experiments 1–5 are described in our general comment.

Addressing the weaknesses:

  1. We agree: failing to name an entity does not by itself show that the model can no longer identify it. We are working on forced-choice selection without “unknown” and factual QA with the correct name supplied, and will post the results once verified. For now, the demonstrated effect is on explicit naming. The matching test (Exp 1) shows that visual discrimination can remain largely intact, but does not settle the mechanism.

  2. The full sweeps in Exp 2 show that zero ablation at all positions and mean ablation at text positions recover layers 16–17 on faces and logos. Visual-only effects there are small, and final-position or connection-blocking interventions can peak elsewhere. We have not tested mean replacement using independent reference inputs, so sensitivity to prompt composition remains open.

  3. In the paper we used the same logos for selection and evaluation. We now test the fixed heads on 235 new brands: all 160 initially correct names are lost, with 1.7% control change. For 25 random sets with the same layer allocation, naming loss averages 1.7% ± 3.8% and reaches at most 15.6%. Exp 3 also reports transfer and split-based validation. We directly tested five-head visual controls and matching; general-capability benchmarks for those heads remain untested.

  4. At matched one- and two-layer widths (Exp 5), the naming–control gaps remain larger in Qwen2.5-VL and LLaVA on faces and landmarks. Window width alone therefore does not explain the pattern. These four models cannot tell us whether depth, size, training, or visual integration causes the differences. We will state that limit clearly.

  5. Exp 4 now compares both head overlap and behavior on naming and conflict tasks. Lietzow’s promoting heads and the Ortu-style prior-favoring heads also affect naming, so we do not claim that the two tasks rely on separate mechanisms. Our added evidence concerns heads selected through ordinary naming: their effect holds on new identities and other domains, with direct visual controls and largely preserved face matching. The comparison also shows that the effects depend on the ablation method.

Addressing the questions:

  1. Selection without “unknown”** We present an image with candidate names and require only the selected option index. There is no “unknown” option. We randomize answer order and keep the candidates identical across intact, layer-ablated, and five-head-ablated conditions. These checks are still in progress; we will report accuracy and malformed responses separately once the results ready and verified.

  2. We will report intact accuracy and paired changes as soon as the results are verified.

  3. Zero ablation at all positions and mean ablation at text positions both recover layers 16–17 on faces and logos (Exp 2). The fixed five heads give 100% naming loss with 1.7% control change on new brands (Exp 3). The general comment gives both comparisons, including the effects of other position choices.

  4. Our set shares one head with Lietzow’s promoting set, none with the Ortu-style prior-favoring set, and one with the Ortu-style visual-favoring set. Exp 4 shows that some of these sets also affect naming, with different effects under the two ablation methods. On VLMBias, changes are modest and occur in both directions. We take this as evidence of some shared effects, without claiming separate circuits or a fixed order of processing.

Reviewer_fsHKOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper focuses on studying the layers and heads of a VLM that enable the identification of familiar entities, such as people or brands. The main contribution is the discovery of specific attention-layer windows where intervention damages entity identification, while having a low impact on responses to low-level perceptual questions about the same images. To achieve this, the authors replace the outputs of the attention heads with their mean value across the sequence and evaluate four models, categorizing entities based on the robustness with which each model recognizes them. In the case of Qwen2.5-VL, the study goes further, identifying five attention heads that account for a large portion of logo recognition. They also show that the intervention can diminish the influence of memory when it conflicts with the visual input.

Critical strengths
  1. I find interesting to attempt to separate the identification of an entity from other responses regarding the same image. The results for Qwen2.5-VL show a significant disparity between the decline in identification and the changes in the evaluated attributes. The experiment involving edited faces is particularly useful, since it allows for the assessment of both aspects using the same images.

  2. The study covers four models and four types of entities. Categorizing entities based on whether the model still recognizes them after cropping or masking also helps in examining how the effect changes according to the robustness of the recognition.

  3. The finding of specific heads in Qwen2.5-VL that reduces logo recognition from 96.1% to 3.1% when removed, while removing five random heads from the same layers results in 87.8%.

  4. General benchmarks and control tests performed on the same images are also useful for verifying that the intervention does not cause a complete collapse of the model, though they do not completely resolve the concerns I detail below.

Critical weaknesses
  1. I am not convinced that network modules in charge of identification have been isolated. Identifying a color or detecting a circle requires far less capability than distinguishing a face or a logo. It’s quite possible for a VLM to keep those basic responses while losing the fine-grained visual capacity needed to identify particular entities. Moreover, the minor drops in other benchmark scores do not discard significant damage in other high-level tasks, identification might just be the most sensitive task among those evaluated.

  2. I am also not convinced by the conceptual justification for the main intervention scheme. Cohen et al. already employ the classic knockout technique to study entity identification, while Lietzow et al. use zero ablation to evaluate the dependence on memorized knowledge versus visual evidence. In the paper, the authors replace the outputs of the attention heads with their mean over the sequence, which is completely valid, but they do not compare this approach to these well-known alternatives from the literature. While retaining an input-dependent component clarifies the nature of the replacement, it does not show that this method is adequate to isolate the identification process.

  3. It is difficult to see a clear advancement relative to related work. Cohen et al. already used interventions to study where an entity is identified and how knowledge about it is retrieved. Ortu et al. and Lietzow et al. identify small sets of attention heads whose manipulation alters the relative weight given to memory versus visual input. While these studies are cited, the paper fails to discuss what specific findings show an advance over their work.

  4. The results for Qwen3-VL demonstrate the limits of this interpretation, as the selected band spans five layers and alters approximately 42% of the control responses, compared to a 69% loss in recognition. The chosen criterion favors this, even though both metrics are high. Even though the authors acknowledge that they might be removing more general visual information, these results suggest that they failed to isolate an identification gate within this model, which limits the generalizability of the finding among different architectures.

  5. Very minor terminology point: the term attention knockout is tipically used to the process of blocking attention connections, like in Geva et al. and Cohen et al. The intervention proposed here is called the same, but instead replaces each head’s outputs with their mean over the input sequence. The operation is clearly defined, but calling it, for example, mean ablation would avoid confusion with those earlier methods.

Questions for authors
  1. Why did you choose elements like the dominant color or the presence of circles to evaluate an intervention that changes so many heads and positions? These features can be detectable even when more complex visual capabilities of VLMs are damaged. A better strategy could have been comparing if two photographs show the same person, without asking for their name. This can help determine whether the intervention makes it difficult to distinguish identities, as opposed to primarily hindering the retrieval or recall of names.

  2. In Appendix C, you mention that the dataset contains text-only version of logos. I noticed that the prompt specifies not to rely exclusively on the text, but this instruction does not reveal how the model arrives to the answer. Does the five-head result hold true for logos that do not contain the brand name in written form? I am interested in determining how much of the decline from 96.1% to 3.1% is because of a loss of brand recognition, and how much might be due to difficulties in reading the text.

3 comments

Thank you for highlighting the same-image comparisons, the breadth of the evaluation, and the five-head result.

Experiments 1–5 are described in our general comment.

Addressing the weaknesses:

  1. We agree that the original controls, including the edited-face attributes, do not fully answer this. We ran the face-matching test you suggested: on 372 pairs, accuracy is 88.2% intact, 88.7% with five-head ablation, and 84.9% with layer ablation. This gives stronger evidence that naming loss is not simply a failure to distinguish faces. The result is clearest for the five heads; layer ablation has a larger effect on highly known people. Exp 1 in the general comment gives the details. We will add it to the paper and limit preservation claims to the abilities tested.

  2. Mean replacement removes differences between positions while keeping the head’s average contribution. It is the closest constant replacement in squared-error terms, which explains our choice but does not prove selectivity. Rudman et al. (2026, Appendix D) use the same kind of replacement. We now compare zero ablation, connection blocking, and interventions at different positions (Exp 2). Zero ablation and text-only mean ablation both recover layers 16–17 on faces and logos; other position choices can move the peak.

  3. We will make the specific contribution clearer. Cohen et al. study where entities are identified and how facts about them are retrieved. Their intervention analyses focus on LLaVA-style models, although their behavioral evaluation includes other architectures. Our intervention study covers Qwen2.5-VL, Qwen3-VL, Gemma-4, and LLaVA-1.5, across four domains and 2,152 entity images. We measure naming and visual-control effects together across decoder windows and familiarity tiers, showing both where they separate and where they do not.

Relative to Ortu and Lietzow, our heads are selected through ordinary logo naming, without a knowledge–image conflict. The new tests show that the fixed set transfers to new brands and other domains (Exp 3) and largely preserves face matching (Exp 1). We also directly compare the head sets on naming and conflict tasks (Exp 4). Prior sets can affect naming too, so we will frame the advance around these measured effects and their generalization, rather than claim that we have established a separate recognition circuit.

  1. We agree: 69% naming loss with 42% control change does not establish a selective gate as in the other models. Exp 5 compares models at the same one- and two-layer widths; Qwen3-VL and Gemma-4 still show much smaller naming–control gaps on faces and landmarks. We will make clear throughout the paper that some regions affect both naming and visual controls. The strongest evidence for selectivity is in Qwen2.5-VL.

  2. Thank you for pointing out the possible confusion. Rudman et al. (2026, Section 4.1 and Appendix D) also call sequence-mean head-output replacement “attention head knockouts,” so the term has precedent. We will make it clear in the paper that it dose not refer to blocking attention connections, like in Geva et al. and Cohen et al.

Addressing the questions:

  1. Color and shape gave us simple controls that could be used across domains without requiring entity knowledge. We agree that they do not test finer visual discrimination. We have now added your suggested matching task, using different photographs of the same person and visually similar people as negative pairs. Exp 1 reports the results: matching stays largely intact, especially under five-head ablation, despite the substantial naming loss.

  2. We tested naming and literal transcription separately. We grouped logos using the intact model’s transcriptions and the dataset’s text-variant labels, so these are automatically assigned groups rather than manually verified categories.

Logo groupInitially correct namesNaming loss: layersNaming loss: five heads
Classified as text-free40100%95%
Containing the readable brand name235100%99%

In the text-bearing group, the model can still transcribe the brand string in 97% of cases under either intervention. The naming loss therefore cannot be explained solely by an inability to read the brand name. We will report the grouping method and avoid extending this result to all OCR tasks.

Reviewer_fsHKOpenReview ↗

Thanks for the response

Thanks for addressing most of my concerns. I'm still interested in discussing the novelty of the identified heads. In Exp 4, ablating Lietzow's heads also negatively affects naming people from photos. Exp 1 shows that ablating your five heads has almost no effect on the model's ability to tell if two photos are from the same person, but that test is not reported for Lietzow's heads. Is there a result that suggests your heads serve a different function?

Thank you for following up and for the suggestion. We have now tested Lietzow’s 21 prior-promoting heads on the same photo-matching task (Exp 1) and held-out logo/control tasks (Exp 3), extending the comparison in Exp 4.

Using the same sequence-mean ablation at all prompt positions for both sets, we obtain:

MeasureOur five headsLietzow’s 21 heads
Face naming loss, as reported in Exp 455%62%
Photo-matching accuracy; intact: 88.2%88.7%84.7%
Held-out logo naming loss100%100%
Held-out logo control change1.7%4.0%

Both sets strongly affect naming while largely preserving photo matching. Our set affects the tested visual controls less, although it also contains fewer heads: five versus 21. Exp 4 shows another difference in their response to ablation: with final-token zeroing, face naming loss is 8% for our heads and 57% for Lietzow’s.

Regarding the overall discussion on novelty, our contribution does not depend on these heads having entirely different functions. The paper primarily focuses on entity recognition, familiarity tiers, and matched perceptual controls. We also show that the fixed five-head set generalizes to new identities and other domains without reselecting the heads. These findings add empirical value even if some of the underlying functions are shared. We will add the new comparisons and clarify this scope and specific contribution in the paper.

Comments on the paper

4 comments

Thank you for the helpful reviews. We are addressing your comments with nine additional experiments. We summarize five completed experiments here because they address concerns shared by several reviewers. The individual replies cover the other experiments and their status.

The main findings are:

  • The main layer window also appears with zero ablation. Replacing outputs with zero finds the same layers as our mean replacement. Which token positions we target still matters.
  • A harder visual control supports the five-head result. The model remains able to match different photographs of the same person, even though its ability to name people drops substantially.
  • The five-head effect holds on new identities. The original heads strongly affect naming on new brands, faces, and landmarks, with much smaller changes to the visual controls.
  • The comparisons help define the contribution and its limits. Heads from prior work can also affect naming, and the separation between naming and visual controls is much weaker in some models.

Unless noted otherwise, the experiments use Qwen2.5-VL-7B-Instruct. We compare the intact model with either all attention heads in layers 16–17 ablated (layer ablation) or the five heads selected in the paper ablated (five-head ablation). Our main method replaces each selected head’s output at every input position with its average across the input sequence. In the tables, naming loss is the percentage of initially correct names that are wrong or no longer given after ablation. Control change is the percentage of answers to visual questions that change; a changed answer is not necessarily wrong.

Exp 1 — Can the model still distinguish people without naming them?

Requested by fsHK (W1/Q1) and va2T (W4/Q3).

Color and circle detection may be too easy to reveal damage to face discrimination. We therefore tested the stronger control the reviewers suggested: whether two different photographs show the same person, without asking for a name.

We found second photographs for 186 of the 600 identities and built 372 balanced pairs: 186 same-person pairs and 186 different-person pairs. Positive pairs used genuinely different photographs; negative pairs used visually similar people. We randomized image order and asked only for “same” or “different.”

Familiarity tierPairsIntact accuracyLayer ablationFive-head ablation
All37288.2%84.9%88.7%
Highly known4297.6%88.1%97.6%
Known8689.5%84.9%93.0%
Unknown24486.1%84.4%85.7%

The tiers reflect recognition reliability under the paper’s image perturbations.

Five-head ablation leaves overall matching accuracy essentially unchanged: 88.7% versus 88.2% intact. Layer ablation lowers it to 84.9%, a drop of about three percentage points. The naming loss therefore coexists with largely preserved face matching, especially for the five heads.

The overall scores do hide some differences. With five-head ablation, same-person accuracy changes from 92.5% to 88.7%, while different-person accuracy changes from 83.9% to 88.7%. Layer ablation has a larger effect on highly known people: 97.6% to 88.1%. Thus, largely preserved overall performance does not mean every subgroup is unaffected.

Five-head ablation changes seven correct answers to wrong ones and nine in the opposite direction; layer ablation changes 20 and eight, respectively. Malformed responses are 0% throughout.

\quad \quad

Exp 2 — Does the same layer window appear with other ablations?

Requested by fsHK (W2), qFRC (W2/Q3), and va2T (W3/Q2).

We swept all 27 adjacent-layer pairs on faces and logos, comparing mean replacement with zero ablation, interventions at different token positions, and blocking text-to-image attention connections.

At layers 16–17, the results are:

InterventionFaces: naming loss / control changeLogos: naming loss / control change
Mean replacement at all prompt positions85% / 13%100% / 4%
Zero ablation at all positions100% / 13%100% / 5%
Mean replacement at text positions85% / 12%100% / 4%
Mean replacement at visual positions5% / 2%2% / 0%

The first three methods all identify layers 16–17 as the strongest window, ranked by naming loss minus control change among the 26 windows excluding layer 0, in both domains. This shows that the location of the main effect does not depend on using the sequence mean.

The positions we intervene on do matter. Mean and zero ablation at only the final position peak at layers 19–20 for faces and 16–17 for logos. Blocking text-to-image connections peaks at layers 13–14 for faces and has weaker effects on logos. Final-position zeroing and connection blocking also continue during generation. We chose mean replacement to remove differences between positions while retaining the head’s average output. Among constant replacements, it makes the smallest squared change to those outputs.

\quad

Exp 3 — Do the five heads work on identities excluded from selection?

Requested by qFRC (W3/Q3) and va2T (W5/Q4).

In the paper we used the same logo collection to select and evaluate the heads. We have now tested the original five heads on 235 new brands excluded from selection, including 160 that the intact model names correctly. We also tested faces and landmarks without reselecting the heads.

EvaluationInitially correct namesFive-head naming lossControl changeRandom-set naming loss, mean ± SD
New brands160100%1.7%1.7% ± 3.8%
Landmarks7691%7.7%6.8% ± 8.3%
Faces11358%10.8%3.6% ± 2.7%

The fixed heads strongly affect naming on new identities, while visual controls change much less. For new brands, we compared against 25 random five-head sets, each with four heads from layer 16 and one from layer 17, excluding the selected heads. The largest random-set naming loss was 15.6%. Faces and landmarks each used five random sets with the same layer allocation.

We also split the original logos into selection and evaluation halves. Selecting five heads within layers 16–17 on the first half recovered four of the paper’s five heads. On the other half, the reselected set caused 99% naming loss among 121 initially correct names, compared with 0%–0.8% for five random sets.

The split test checks reselection within the chosen layers; the new-brand test checks the original heads on independent identities. Visual controls and face matching were measured under the five-head intervention.

Exp 4 — How do our heads compare with those from prior work?

Requested by qFRC (W5/Q4); also addresses fsHK (W3).

We tested whether heads from prior work on image–knowledge conflicts also affect ordinary naming, and whether our naming-selected heads affect answers to those conflicts.

We used Lietzow et al.’s published Qwen2.5-VL sets and adapted Ortu et al.’s last-token logit-lens selection to that model. On 449 VLMBias items, the Ortu-style procedure selected 20 heads favoring familiar knowledge and 20 favoring the image. This is an adaptation, not a reproduction of their original experiments.

Our five-head set shares one head with Lietzow’s prior-promoting set, none with the Ortu-style prior-favoring set, and one with the Ortu-style visual-favoring set.

On a separate set of 105 faces named correctly by the intact model, we applied the same two ablation methods to every head set:

Head setHeadsNaming loss: mean at all prompt positionsNaming loss: zero at the final position
Our set555%8%
Lietzow prior-promoting2162%57%
Lietzow prior-suppressing81%0%
Ortu-style prior-favoring2030%6%
Ortu-style visual-favoring2068% malformed responses5%

Some prior head sets also affect naming, with different effects under the two ablations. Across five random sets of each size, mean-ablation naming loss averaged 4% ± 7% for five heads and 5% ± 4% for 20 heads. Two of five random 21-head sets and two of five random 29-head sets produced malformed output, which we report separately.

We also tested the sets on the 449 conflict items. Here, “prior” means an answer consistent with familiar knowledge; “visual” means an answer supported by the image.

Head setMean ablation: prior / visual answersFinal-position zero: prior / visual answers
Intact61% / 34%61% / 34%
Our five heads59% / 37%61% / 36%
Lietzow prior-promoting58% / 37%58% / 34%
Ortu-style prior-favoring59% / 36%60% / 36%
Ortu-style visual-favoring43% / 32%*61% / 34%
Random five-head sets, mean of five sets61% / 35%62% / 34%

The Ortu-style visual-favoring set produced 26% malformed responses under mean ablation. Its lower prior-answer rate therefore does not show better visual grounding.

The intact model gave 275 prior answers, 154 visual answers, and 20 other answers. With mean ablation of our five heads, 24 prior answers became visual and 12 visual answers became prior. The net shift toward the image is small, and answers change in both directions. This run’s baseline agreed with the submission’s run on 378 of 449 items; all paired comparisons use the baseline from this experiment.

\quad \quad

Exp 5 — Do model differences remain when we ablate the same number of layers?

Addresses W4 of all three reviewers.

We reanalyzed the saved sweeps using one- and two-layer windows for every model. This checks whether the model differences simply come from comparing interventions of different widths.

The table shows selected windows for faces and landmarks. Each entry gives naming loss / control change.

WidthModelFacesLandmarks
1 layerQwen2.5-VLL16: 65% / 15%L19: 31% / 12%
1 layerLLaVA-1.5L1: 98% / 28%L6: 100% / 26%
1 layerGemma-4L11: 27% / 14%L16: 25% / 16%
1 layerQwen3-VLL29: 11% / 2%L1: 8% / 3%
2 layersQwen2.5-VLL16–17: 88% / 21%L16–17: 91% / 21%
2 layersLLaVA-1.5L4–5: 98% / 29%L6–7: 100% / 31%
2 layersGemma-4L20–21: 32% / 14%L15–16: 41% / 27%
2 layersQwen3-VLL29–30: 11% / 1%L14–15: 22% / 13%

The model differences remain at matched widths. With two layers, Qwen2.5-VL and LLaVA have larger gaps between naming loss and control change than Gemma-4 and Qwen3-VL on both domains. LLaVA’s controls still change in 29%–31% of cases, so a large gap should not be read as no effect on perception.

Equal layer counts still differ as a fraction of model depth. With these four models, we cannot separate the effects of depth, size, training, or visual integration, or establish a selective gate shared by all four.

Exp 6 — Identifying faces from a closed list of names

Addresses qFRC (W1/Q1) and va2T (W2/Q1).

Following the reviewers’ suggestion, we tested whether the model can still identify a person after ablation when candidate names are provided.

We kept the original identification prompt and added 100 candidate names, asking the model to choose one. Each image had its own list, with roughly one-third of the candidates from each familiarity tier and the correct name at a random position. We compared intact Qwen2.5-VL with the paper’s mean ablation of attention-head outputs in L16–17.

Familiarity tierImagesIntact accuracyL16–17 ablation
Highly known4497.7%90.9%
Known10088.0%84.0%
Unknown45622.1%14.0%

Note that even in the “unknown” tier, intact accuracy reaches 22.1% (over 20 times the 1% chance level) which indicate that choosing from supplied names is an easier task than producing a name without those cues. While we do see that there is a drop after the ablation (especially in the highly known tier) for this task, the accuracy is indeed much higher than for the original task. This control makes our interpretation more precise. We will distinguish the task of recognition that is stated in the paper (open-ended naming) from identification with candidate names and clarify that the familiarity tiers describe the original probe. We thank the reviewers for suggesting this experiment, it indeed better frame the results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.