- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Aiming to investigate the problem of mismatched VLM answers-text rationales, and early internal read-outs cannot reflect the final output. This paper introduces AnswerMap, a training-free/task-agnostic/black-box visual rationale constructed from the output head, which bypasses the reliance on discrete text tokens, providing a new lens on VLM interpretability.
Critical strengths
-
The method requires no training and relies only on output token probabilities, making it easy to apply across different VLM architectures without adapting to their internal visual token representations.
-
Evaluation across models with useful controls. The experiments cover multiple model sizes and two model families, and assess both spatial agreement and the effects of deleting selected regions. Matched random deletion, blurred masking, and blank or swapped image controls help characterize the method’s behavior, even though they do not fully establish faithfulness to the original decision process.
Critical weaknesses
My main concern is whether the paper sufficiently demonstrates faithfulness to the original model’s decision process.
-
Mismatch between probing and original answering. AnswerMap probes the model using cropped images and reformulated questions, rather than directly examining how the original answer was produced. Since both the visual input and the question are modified, these probing responses may rely on different evidence from the original answer. The connection between the resulting map and the original decision therefore requires stronger justification.
-
Image dependence does not establish faithfulness. Ensuring that a map depends on visual input does not necessarily mean that it identifies the evidence responsible for the original answer. The current results may demonstrate localization of semantically relevant content rather than faithful attribution of the model’s decision. Additional controls are needed to distinguish these interpretations.
Questions for authors
-
Compare AnswerMap with a baseline that directly probes individual image cells using the same Yes/No prompt. Although more expensive, this comparison would quantify the spatial information lost through the row and column factorization. Comparisons with coarse grids, sliding windows, or hierarchical cropping under the same query budget would also help establish whether AnswerMap offers advantages over simpler ways of allocating computation.
-
Apply the same deletion test to regions identified by another VLM or an independent grounding method. If these regions produce similar answer change rates, the results would suggest that AnswerMap primarily identifies generally relevant semantic content, rather than evidence specific to the original model’s decision.
I'm willing to consider revising the rating if my concerns are carefully addressed.
4 comments
Rebuttal
Cross-model control (Q2). The reviewer also asks for regions identified by another VLM. We ran this with InternVL3.5-8B as the second VLM. We argue that this control cannot isolate faithfulness, because its result depends on two things at once: whether the map is faithful, and how often the two VLMs give the same answer to the same question. The detector control above does not have this problem. A detector returns a box for the named object. It does not answer the question the VLM is asked (is the object in the image?), so it cannot give the same wrong answer as the VLM. A second VLM is asked the same question, and when it gives the same wrong answer, its map marks the evidence for that same wrong answer. This happens often: InternVL gives the same wrong yes in 189 of Qwen's 223 wrong cases, where about 25 would be shared if the two VLMs made their wrong answers independently. Deleting the region of InternVL's AnswerMap changes Qwen's wrong yes 64.1% of the time, against 69.5% for Qwen's own AnswerMap (gap 5.4 points, 95% CI -0.4 to 11.2). On TextVQA the two rates are 50.1% and 53.4%, and the two VLMs give the same answer on three in four of the questions. The two VLMs often give the same answer, so close rates are expected here, and we therefore rely on the detector control above to isolate faithfulness.
To accommodate this important control in the paper, we will extend Test 2 to cover both the VLM's correct answers, which are already in the paper, and its wrong answers. We believe Test 2 then evaluates faithfulness more comprehensively.
Per-cell probe and other ways to spend the query budget (Q1). The row and column factorization loses nothing on the two tests: probing each of the 64 cells alone with the same Yes/No prompt uses four times the queries and scores lower on both. AnswerMap also scores above a coarse grid, sliding windows and hierarchical cropping at the same query budget. All four were run on the examples of Tables 2 and 3.
| Method | Queries | RefCOCOg, agreement (AUC) | TextVQA, answers changed (%) |
|---|---|---|---|
| AnswerMap | 16 | 0.853 | 53.4 |
| Per-cell probe, 8x8 cells, each shown alone | 64 | 0.734 | 42.8 |
| Coarse grid, 4x4 cells, each shown alone | 16 | 0.783 | 12.0 |
| Sliding windows, 16 half-size windows | 16 | 0.768 | 28.3 |
| Hierarchical cropping, three rounds of four crops | 12 | 0.624 | 20.4 |
| Random cells | 0.488 | 8.4 |
The per-cell probe is AnswerMap without the factorization, and it is 0.119 AUC behind on agreement and 10.6 points behind on deletion (95% CI 5.4 to 15.8). The coarse grid, the sliding windows and the hierarchical cropping are also below AnswerMap on both tests. AnswerMap therefore has an advantage over each of these simpler ways to spend the queries, and the comparison will be added to the ablation study (Section 6).
Official Comment by XX3X
I thank the authors for the comprehensive rebuttal, which addresses my concerns regarding the baseline comparisons and stronger empirical support for the proposed method.
However, I remain concerned about whether the results sufficiently establish faithfulness to the original model's decision process. While the deletion experiments demonstrate that the identified regions influence the original predictions, such behavioral sensitivity does not necessarily establish that the probing procedure faithfully captures the evidence underlying the original inference.
Therefore, I'd like to keep my original score.
Follow-up
We thank the reviewer for the reply, and we are glad that the rebuttal addressed the concerns about the baseline comparisons and the empirical support.
We would like to clarify that AnswerMap is, in essence, an interpretable approximation of the VLM's decision process rather than a direct reading of that process, and the two tests measure the error of this approximation. Because the exact path of a decision inside the model is hard to interpret, AnswerMap builds a second, independent path to the same answer, one that is interpretable, through a different input and output. We see this as faithfulness by proxy, a new lens for interpreting a model. The error of the approximation is the divergence between the two paths. Test 1 measures this divergence on the answer and Test 2 on the evidence, with controls that rule out random regions and semantic matching (Tables 2, 3 and 6). When an interpretable path reaches the same answer from the same evidence as the original inference, this suggests that it reflects how that answer was reached, and the results of both tests and their controls support this for AnswerMap. Seen from this perspective, we believe AnswerMap is a first step in a new direction for interpreting VLMs. We will reflect this framing in the final version of the paper.
Rebuttal
We thank the reviewer for the constructive comments. Below are the new experiments they led to: a semantic-matching control (W2, Q2), and a per-cell probe with three other ways to spend the same query budget (Q1).
Link between the band queries and the original answer (W1). The two tests check this connection, and both compare the map with the answer the VLM gives on the original image with the original question. The intuition is that AnswerMap is a second, interpretable path to the answer. If this path agrees with standard inference, that suggests it reflects how the original answer was reached.
While the band queries use a different input, the deletion results show that the original answer depends on the region they identify. We treat this as evidence that the band queries capture the relevant evidence, and not an unrelated signal.
Semantic-matching control (W2, Q2). The reviewer asks whether regions identified by an independent grounding method would give similar answer change rates, which would mean that AnswerMap identifies generally relevant semantic content and not evidence specific to the VLM's decision. The results below show that AnswerMap follows the VLM and not the semantics of the query.
An independent grounding method only matches the query to the image, with no access to the VLM. We call such a method a semantic matcher. When the VLM answers correctly, the part of the image it answered from is where the queried object is, and a semantic matcher marks that part too. On correct answers a faithful map and a semantic matcher are therefore expected to give similar answer change rates, so correct answers alone cannot separate the two. However, the VLM's wrong answers can provide this separation. The clearest case is a yes to an object that is not in the image. Here the VLM cannot have answered from the queried object, because there is no such object in the image, so a faithful map and a semantic matcher mark different regions. If AnswerMap were only a semantic matcher, deleting its region in this case would change the wrong answer no more often than deleting the matcher's region. If AnswerMap is faithful, deleting its region changes the wrong answer more often. The table below shows this test on POPE, the benchmark of Table 6, which asks whether a named object is in the image. We delete the region marked by AnswerMap, by the semantic matcher (the best box of the detector GroundingDINO for the named object) or at random, and count how often the VLM's yes changes. The test is run on two VLMs, each explained by its own AnswerMap, on the same questions: 800 where the named object is in the image and 4,500 where it is not. Each column holds the questions that its VLM answers yes to.
All entries are the share of answers changed (%). On the correct answers, deleting the detector's box changes nearly as many answers as deleting AnswerMap's region, for both VLMs (35.3% against 38.2%, and 30.3% against 31.5%), as expected. On the wrong answers, deleting AnswerMap's region changes the wrong yes more often than deleting the detector's box, for both VLMs: 69.5% against 57.4% for Qwen3-VL-4B (gap 12.1 points, 95% CI 5.4 to 18.8), and 58.0% against 48.4% for InternVL3.5-8B (gap 9.6 points, 95% CI 4.5 to 14.6). The same detector that nearly matches AnswerMap on correct answers falls behind on wrong answers. On TextVQA, the benchmark of Table 3, the detector's box changes 27.5% of the answers against 53.4% for AnswerMap.
Conclusion. On the VLM's wrong answers AnswerMap follows the VLM and the semantic matcher falls behind, so AnswerMap identifies evidence specific to the VLM's decision, and not only generally relevant semantic content.