- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether frozen Large Vision-Language Models (LVLMs) already encode spatial evidence for visual change localization, despite their limited ability to explicitly localize changes through textual coordinate generation. The authors propose a training-free framework that identifies change-informative attention heads using two criteria: (1) Attention Sum Deviation, which measures deviations from a semantically unchanged self-reference baseline, and (2) Attention Spatial Consistency, which measures the consistency of spatial responses under image-order reversal.
The selected attention heads are used to extract bidirectional deviation maps, which are aggregated and refined through Semantic-guided Attention Diffusion (SAD) using frozen vision-encoder features. The resulting change response maps are further refined into segmentation masks using SAM or MedSAM proposals.
The authors evaluate the framework across ten LVLMs and diverse benchmarks, including ChangeVPR, MVTecAD, and Brain AD. Experimental results show substantial improvements over existing LVLM-based localization baselines and competitive performance against dedicated localization-only methods, without additional model fine-tuning.
Critical strengths
S1. A compelling training-free approach to change localization using frozen LVLMs
- The paper presents an interesting approach to extracting change-localization signals from the internal attention representations of frozen LVLMs. Rather than relying on additional grounding supervision or fine-tuning, the authors identify a small subset of attention heads that respond to visual changes and exhibit spatial consistency under image-order reversal. This extends attention-based visual grounding to the more challenging setting of relational change understanding, where the localization targets must be inferred by comparing two images rather than being explicitly specified. The ability to expose such spatial evidence without modifying the underlying LVLM is a meaningful contribution toward integrating change description and localization within a unified framework.
S2. Extensive empirical validation across LVLMs and diverse domains
- The paper provides strong empirical evidence for the practical effectiveness of the proposed framework. The evaluation covers ten LVLMs with different architectures and parameter scales, as well as substantially different visual domains, including natural scene changes, industrial anomalies, and brain MRI. The method consistently improves upon the corresponding LVLM-based localization baselines and achieves competitive performance against dedicated localization-only methods in several settings. Furthermore, the ablation studies examine the contributions of the head selection criteria, selection strategies, SAD, SAM refinement, and context-aware prompting. These experiments provide useful evidence that the proposed framework is broadly applicable rather than being tailored to a single model or benchmark.
Critical weaknesses
W1. Insufficient justification of Attention Sum Deviation as a criterion for identifying change-responsive heads
-
Although the proposed head selection criteria demonstrate promising empirical results, the motivation behind Criterion 1 requires further justification. Specifically, Attention Sum Deviation measures the signed difference in total attention mass assigned to an image. For example, attention maps [0.30, 0.10] and [0.10, 0.30] yield a deviation sum of zero despite attending to different spatial locations. Thus, potentially useful heads may be discarded before Criterion 2 can assess their spatial consistency.
-
Furthermore, due to softmax normalization over both images and text tokens, attention mass can change without a corresponding change in the target image's attention logits. For example, decreasing the attention logits of the second image alone can increase the attention mass assigned to the first image. Hence, a large deviation does not necessarily indicate sensitivity to semantic changes.
-
The role of the self-augmented baseline also remains insufficiently explained. In particular, Gaussian perturbations do not necessarily represent real-world pseudo-changes such as illumination, seasonal, or viewpoint variations. While the reported ablations support the empirical effectiveness of the proposed criterion, they do not fully establish whether it specifically suppresses pseudo-changes or captures more general context-dependent attention shifts. This distinction matters because identifying inherently change-responsive heads is central to the paper's contribution.
W2. Limited analysis of cross-domain head selection and dependence on task-specific prompts
-
Although the proposed method demonstrates promising cross-domain performance, the generalizability of the selected attention heads remains insufficiently examined. The heads are selected on VL-CMU-CD and fixed across all evaluation domains. Although Appendix C.2 compares head selection on VL-CMU-CD and PSCD, both are natural-scene datasets. This does not establish whether head selection on industrial or medical image pairs would yield similarly informative heads or improve performance in those domains.
-
The anomaly detection prompts in Appendix B.3 explicitly specify category-specific defect types that closely match those in the evaluation benchmarks. Given the substantial performance improvements from context-aware prompting in Table 6, it is difficult to disentangle the contribution of transferable attention heads from that of highly tailored prompts.
Questions for authors
Q1. Justification of Attention Sum Deviation (Related to W1; Sections 4.1-4.2)
-
Could the authors further justify using the signed sum of attention deviations for the initial head filtering? As discussed in W1, this measure can overlook substantial spatial redistribution when positive and negative residuals cancel out. Since Criterion 2 is applied only to heads retained by Criterion 1, potentially informative heads excluded at the first stage cannot be recovered.
-
Have the authors considered alternative measures, such as the L1 or L2 norm of the element-wise deviation map, for the initial candidate selection? Comparing these alternatives in terms of head selection, GT alignment, and final localization performance would help clarify whether signed attention-mass deviation is the most appropriate criterion.
-
As a minor presentation clarification, could the authors also specify how the Head Index in Figure 2 maps to layer and head indices, particularly whether heads are flattened across layers?
Q2. Effect of baseline subtraction and photometric perturbations (Related to W1; Section 4.1 and Appendix D.1)
-
The proposed method uses Gaussian noise to construct the unchanged self-reference baseline. However, Gaussian noise does not necessarily reflect real-world pseudo-changes, such as illumination, seasonal, or viewpoint variations. Could the authors clarify why Gaussian noise was chosen and investigate whether other perturbation types affect head selection and localization performance?
-
Table 12 shows only a modest improvement from Gaussian perturbations (60.9 to 61.8 in average F1 for InternVL3-14B). Could the authors further disentangle the effects of context-aware prompting, unchanged self-reference subtraction, and photometric perturbation? For instance, a controlled comparison varying prompt specificity and baseline subtraction, together with different baseline perturbations where applicable, would be informative.
-
Additionally, evaluating image pairs containing only pseudo-changes, without semantic changes, would help determine whether the proposed baseline actually suppresses nuisance-induced responses rather than general context-dependent attention variations.
Q3. Generality of head selection across domains (Related to W2; Section 4.1 and Appendix C.2)
-
The authors motivate selecting heads on VL-CMU-CD by noting that changed instances appear in only one image per pair, thereby avoiding potential confounding effects when changes occur in both images. Could the authors elaborate on the specific difficulties that arise in the latter setting and explain how they affect the proposed head selection criteria?
-
Although Appendix C.2 compares head selection using VL-CMU-CD and PSCD, both datasets involve natural scene changes. Could the authors investigate head selection using substantially different domains, such as industrial or medical imagery, and compare the resulting heads and localization performance against VL-CMU-CD-based selection?
-
Such experiments could use independent source datasets containing suitable change pairs, without relying on target test annotations, to preserve the validity of the evaluation. This would help clarify whether the selected heads are domain-invariant or whether domain-specific selection provides additional benefits.
Q4. Dependence on category-specific anomaly prompts (Related to W2; Appendix B.3 and Table 6)
-
The prompts for MVTecAD explicitly list category-specific defect types that closely match the benchmark's anomaly taxonomy (e.g., cracks, cuts, holes, and print marks for hazelnuts). Could the authors clarify how these prompts were constructed? Specifically, were they designed a priori based on general domain knowledge, informed by benchmark-specific defect labels or annotations, or refined using validation or test performance?
-
Given the substantial improvement from context-aware prompting in Table 6, could the authors compare three prompting strategies: (i) generic anomaly prompts, (ii) category-aware but defect-agnostic prompts, and (iii) defect-aware prompts explicitly listing anomaly types, while keeping the selected heads and other inference settings fixed? This comparison, ideally including anomaly types not explicitly mentioned in the prompts, would help clarify how much the proposed method relies on benchmark-specific semantic priors.
6 comments
Response to Reviewer jreq (2/3)
[W1&Q2-1] Effect of Photometric Perturbation.
We select perturbations based on prior studies on robustness and scene change detection [1, 2], which consider 15 types of common corruptions. Following the reviewer’s constructive suggestion, we further evaluate brightness & contrast (illumination), snow (seasonal variation), and crop & resize (viewpoint change). As shown below, head selection remains consistent across all perturbations except snow—yielding the same Top-3 heads—with Gaussian noise achieving the best performance. Overall, the results suggest that augmentation during head filtering helps identify a consistent set of change-informative heads, while its effect during inference is generally modest.
\hline \text{Results} & \text{None} & \text{Gaussian noise} & \text{Brightness \\& contrast} & \text{Snow} & \text{Crop \\& resize} \\\\ \hline \text{Selected Top-3 heads} & \begin{array}{c} \text{L37H34 · L38H38} \\\\ \text{L38H32} \end{array} & \begin{array}{c} \text{L37H34 · L38H38} \\\\ \text{L38H35} \end{array} & \begin{array}{c} \text{L38H38 · L37H34} \\\\ \text{L38H35} \end{array} & \begin{array}{c} \text{L38H32 · L36H21} \\\\ \text{L38H35} \end{array} & \begin{array}{c} \text{L37H34 · L38H38} \\\\ \text{L38H35} \end{array} \\\\ \hline \text{F1-Score (head filtering only)} & 60.9 & 61.6 & 61.6 & 52.2 & 61.6 \\\\ \text{F1-Score (inference only)} & 60.9 & 61.4 & 59.7 & 51.0 & 58.5 \\\\ \text{F1-Score (both)} & 60.9 & 61.8 & 61.0 & 52.1 & 61.7 \\\\ \hline \end{array}$$ [1] Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (ICLR 2019) [2] Differencing-based Self-supervised Pretraining for Scene Change Detection (CoLLAs 2022) --- **[W1&Q2-2] Effects of Prompting, Baseline Subtraction, and Perturbation.** We provide a fine-grained analysis of prompting, baseline subtraction, and perturbations. Following the original protocol, we use InternVL3-14B and select the Top-3 heads using the VL-CMU-CD training set. As shown in the tables below, the context-aware prompt has the largest effect on performance, whereas the choice of baseline perturbation leads to modest variations. These results suggest that baseline perturbation serves as an auxiliary step for refining attention signals passing through change-informative heads, while the semantic definition of changes in the input prompt plays a more essential role. $$\begin{gathered} \begin{array}{lcc} \hline \text{Baseline perturbation} & \text{w/o subtraction} & \text{w/ subtraction} \\\\ \hline \text{None} & 23.5 & 24.5 \\\\ \text{Gaussian noise} & 23.5 & 25.5 \\\\ \text{Brightness \\& contrast} & 23.5 & 24.6 \\\\ \text{Snow} & 23.5 & 25.4 \\\\ \text{Crop \\& resize} & 23.5 & 24.7 \\\\ \hline \end{array} \\\\ \text{<Generic prompt>} \end{gathered}$$ $$\begin{gathered} \begin{array}{lcc} \hline \text{Baseline perturbation} & \text{w/o subtraction} & \text{w/ subtraction} \\\\ \hline \text{None} & 56.6 & 60.8 \\\\ \text{Gaussian noise} & 56.6 & 61.8 \\\\ \text{Brightness \\& contrast} & 56.6 & 61.0 \\\\ \text{Snow} & 56.6 & 59.1 \\\\ \text{Crop \\& resize} & 56.6 & 61.7 \\\\ \hline \end{array} \\\\ \text{<Context-aware prompt>} \end{gathered}$$ --- **[W1&Q2-3] Evaluation on Image Pairs Containing Only Pseudo-Changes.** We construct 300 pseudo-change image pairs from PSCD panoramas, each with an all-zero GT change mask, and evaluate the false positive (FP) rates of the methods. We report two metrics: FP pixels, the average percentage of pixels incorrectly predicted as changed, and FP images, the percentage of image pairs containing at least one predicted change pixel. Although our method yields a higher FP pixel rate than RSCD, it achieves a lower rate than the training-free method GeSCF, while attaining the lowest FP image rate. $$\begin{array}{lcc} \hline \text{Methods} & \text{FP pixels (\\%)} \downarrow & \text{FP images (\\%)} \downarrow \\\\ \hline \text{GeSCF} & 3.85 & 84.4 \\\\ \text{RSCD} & \mathbf{1.86} & 81.5 \\\\ \text{Ours (InternVL3-14B)} & 3.32 & \mathbf{80.1} \\\\ \hline \end{array}$$Response to Reviewer jreq (3/3)
[W2&Q3-1] Confounding Effects When Changes Occur in Both Images.
We thank the reviewer for raising this point. Due to the softmax normalization, concurrent changes in the other image may weaken the attention response to the target image and affect Criterion 1 rankings. For example, as shown below, L38H35 ranks within the top-100 on VL-CMU-CD and proceeds to Criterion 2. When also contains a change as in PSCD, its average attention mass on decreases from 0.117 to 0.049, and its average attention sum deviation decreases from 0.109 to 0.040. Finally, its Criterion 1 rank drops from 19th to 118th, causing the head to be filtered out before Criterion 2 on PSCD, consistent with Appendix C.2. We generally observe this weakening across heads when changes occur in the other image, but the decline is more pronounced for some heads, including L38H35. Token interactions and dataset biases may contribute to this variation, but understanding why certain heads are more strongly affected remains an interesting direction for future work.
\hline \text{Results (L38H35)} & \text{VL-CMU-CD} & \text{PSCD} \\\\ \hline \text{Avg. attention mass on $I_a$} & 0.117 & 0.049 \\\\ \text{Avg. attention sum deviation} & 0.109 & 0.040 \\\\ \text{Criterion 1 rank} & \text{19th} & \text{118th} \\\\ \hline \end{array}$$ --- **[W2&Q3-2] Generality of Head Selection Across Domains.** Following the reviewer’s insightful suggestion, we additionally used VisA [3] (industrial anomalies) and RESC [4] (retinal anomalies) to filter attention heads in InternVL3-14B. As described in the table below, VL-CMU-CD yields the best performance across all five datasets. Notably, despite their closer domain match, VisA and RESC do not improve performance on MVTecAD and Brain AD, respectively. These results suggest that domain similarity does not necessarily translate into improved change localization performance. One possible explanation is that change-informative heads capture the general concept of *difference* rather than responding only to domain-specific changes. We hypothesize that VL-CMU-CD or PSCD provides a clearer signal for identifying these heads—potentially because its natural images are more familiar to the LVLM. $$\begin{array}{lccccc} \hline \text{Filtering dataset} & \text{SF-XL} & \text{St Lucia} & \text{Nordland} & \text{MVTecAD} & \text{Brain AD} \\\\ \hline \text{VL-CMU-CD} & \mathbf{69.7} & \mathbf{71.7} & \mathbf{48.4} & \mathbf{40.8} & \mathbf{78.3} \\\\ \text{VisA} & 45.7 & 58.5 & 26.1 & 29.1 & 68.0 \\\\ \text{RESC} & 50.6 & 57.5 & 27.9 & 30.9 & 34.1 \\\\ \hline \end{array}$$ [3] SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation (ECCV 2022) [4] BMAD: Benchmarks for Medical Anomaly Detection (CVPRW 2024) --- **[W2&Q4-1] Rationale for Anomaly Prompts.** As described in Appendix B.3, our prompt template is adapted from C3-Bench [5]. We retain its original instructions while simplifying the wording for brevity—maintaining consistency with prior LVLM-based change captioning work without tuning prompts based on validation or test performance. The rationale for specifying context is that anomaly inspection is typically defined by product types or object categories: defects in pharmaceutical pills (e.g., discoloration, contamination, cracks, or faulty imprints) differ substantially from those in metal grids (e.g., bent strands, broken wires, or metal contamination) or leather products (e.g., scratches or color variations). [5] C3-Bench: A Context-aware Change Captioning Benchmark (ECCV 2026) --- **[W2&Q4-2] Effects of Varying Anomaly Prompts.** We thank the reviewer for this insightful suggestion. We evaluate four prompt types with varying levels of anomaly-related information. As shown in the table below, adding general anomaly information substantially improves F1-Score on both datasets (e.g., *Describe the visible abnormality from image 1 to image 2 in one concise sentence*). Category information further improves performance on MVTecAD but has a smaller effect on Brain AD. Lastly, providing specific defect information yields only marginal gains. These results suggest that our framework does not require extensive prior knowledge of the target domain or benchmark—supporting its practical applicability. $$\begin{array}{lcc} \hline \text{Prompt type} & \text{MVTecAD} & \text{Brain AD} \\\\ \hline \text{Fully Generic} & 18.8 \pm 0.7 & 23.8 \pm 2.7 \\\\ \text{Generic Anomaly} & 33.0 \pm 0.3 & 77.0 \pm 0.9 \\\\ \text{Category-Aware} & 39.7 \pm 0.4 & \mathbf{77.6 \pm 1.5} \\\\ \text{Defect-Aware} & \mathbf{40.7 \pm 0.3} & 77.3 \pm 1.8 \\\\ \hline \end{array}$$
Response to Reviewer jreq (1/3)
We deeply appreciate the reviewer’s insightful comments and careful review of our manuscript. We respond to each comment below, grouping related weaknesses and questions. In the revised manuscript, we highlight our major revisions in blue.
[W1&Q1-1] Further Justification for Attention Sum Deviation.
We greatly appreciate the reviewer’s thoughtful comments! To examine whether positive and negative attention deviations cancel out under the signed sum, we investigate heads for which the signed sum is small relative to the absolute deviation (L1 or L2). Specifically, we compute the cancellation ratio (CR) as one minus the ratio of the magnitude of the mean signed sum to the mean absolute deviation, and identify heads with a CR greater than 0.5.
As shown in the table, the reviewer’s concern is reflected in the Criterion 1 candidates: the L1 and L2 norms include 15 and 11 cancellation-affected heads, respectively, whereas the signed sum includes only one. This difference is expected because the signed sum allows positive and negative deviations to cancel, while L1 and L2 retain their magnitudes regardless of sign. However, no cancellation-affected heads remain after Criterion 2. All three variants also yield similar GT alignment scores, select the same top-3 heads, and achieve identical localization performance. Thus, although L1 and L2 initially admit cancellation-affected heads, the spatial consistency requirement in Criterion 2 filters them out, serving as an additional safeguard for robust change localization. We have included this valuable finding in Appendix C.6.
\hline \text{Results} & \text{Signed Sum} & \text{L1 Norm} & \text{L2 Norm} \\\\ \hline \text{Cancellation-affected Heads after Criterion 1} & 1/74 & 15/79 & 11/79 \\\\ \text{Cancellation-affected Heads after Criterion 2} & 0/33 & 0/23 & 0/26 \\\\ \text{GT Alignment} & 0.78 & 0.77 & 0.78 \\\\ \text{Selected Top-3 Heads} & \text{L37H34, L38H38, L38H35} & \text{Same} & \text{Same} \\\\ \text{F1-Score} & 61.8 & \text{Same} & \text{Same} \\\\ \hline \end{array}$$ --- **[W1&Q1-2] Regarding Softmax Normalization.** We are pleased to address this insightful point raised by the reviewer. First, softmax-normalized attention enables comparisons of attention distributions across different forward passes on a common probability scale, providing a concrete view of how much and where an LVLM focuses under a given instruction (L158–L159). Additionally, the table below compares the attention mass assigned to $I_a$ and the pre-softmax logit scores for each input segment. To facilitate comparison, we align the text logit scores across the two forward passes by adding a constant to all logits in one pass, which does not affect the softmax distribution. When $I_b$ replaces $I_a’$, the relative logit scores increase for both image segments, but more so for $I_a$ than for the second image (+2.07 vs. +1.28). This result indicates that, when a change occurs in $I_a$, the logit scores increase for both image segments, but the larger increase for $I_a$ contributes to its higher attention mass; 50% of the logit increase is concentrated within GT regions, despite their accounting for only 6.7% of all pixels in VL-CMU-CD. $$\begin{array}{lccc} \hline \text{Types (L37H34)} & \text{Input } (I_a, I_b) & \text{Input } (I_a, I_a') & \Delta \\\\ \hline \text{Attention mass } (I_a) & 0.16 & 0.03 & +0.13 \\\\ \text{First image logit score } (I_a) & 7.34 & 5.27 & +2.07 \\\\ \text{Second image logit score } (I_b \text{ or } I_a') & 5.74 & 4.46 & +1.28 \\\\ \text{Text logit score} & 9.12 & 9.12 & 0.00 \\\\ \hline \end{array}$$ --- **[W1&Q1-3] Clarification of Head Index** We thank the reviewer for this helpful suggestion. In Figure 2, heads are flattened in layer-major order: the head index is computed as $l \times H + h$, where $l$ is the zero-based layer index, $h$ is the zero-based head index within that layer, and $H$ is the number of heads per layer. We have clarified this in the figure.