acceptodds
Under review as a conference paper at ICLR 2027

Says Who? We Read and We Write the Decisions of Vision-Language Models

Abstract

Vision-language models (VLMs) combine visual evidence with instructions, yet it is hard to see how they reach a decision or which parts of the picture drive it. We introduce SimonSays, a benchmark of controlled visual counterfactuals for studying authority attribution, evaluation awareness and sandbagging, and V-Lens, a pair of cheap tools that read and attribute a VLM's decision at the answer position. V-Read applies a closed-form linear shortcut from each layer to the final state, and V-Write applies integrated gradients to image-token states from the picture's own mean state. Our contribution is not the maps themselves but placing them at the answer position of a VLM, testing them against matched image edits, and using them on safety-relevant behaviour. V-Read fits in 7 to 20 minutes, against 2.6 to 8.9 hours for the Jacobian lens, reads the answer earlier than the logit lens on all four models we test, and is the best reader on authority tasks more often than any other. V-Write localises COCO objects as well as KernelSHAP when both are scored on the same regions, at about 1% of its runtime, and its top-ranked image tokens remove more of the answer than those of any compared method on all four models. With SimonSays, V-Lens separates the cues a model registers from those it acts on. Both models register a sign's carrier and an instruction to sandbag without following them, while the smaller model does act on a picture dressed as a test. In a model fine-tuned to sandbag when a small watermark is present, which lowers held-out accuracy from 0.60 to 0.30, V-Write finds the watermark from the answer alone.

Primary area:
interpretability and explainable AI
Keywords:
vision-language models, mechanistic interpretability, AI safety, benchmark, visual authority

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_ngeHOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper propose new mechanistic interpretability method called V-Lens which consists of V-Read and V-Write. V-Read learns the affine transformation from intermediate hidden state to the last hidden state while V-Write calculate the gradient of the defined score to measure how each patch contribute to answer change. The authors tested this new interpretability methods on their own benchmark named SIMONSAYS which covers authority, evaluation awareness, and sandbagging. The qualitative results show that V-Write can accurately locate the target object of the query in the image while localizing the object better than KernelSHAP. More experiments include AUROC comparing to previous lens, reading where the answer arises within LLM layers, visual password experiments. The paper argue that V-Lens can distinguish what model perceives and what model actually generates.

Critical strengths
  1. The authors provided the detailed formulation of V-Lens.
  2. The paper curated their own benchmark SIMONSAYS to test V-Lens.
  3. Extensive experiments are conducted including various baselines of V-Read and V-Write.
Critical weaknesses
  1. I am significantly concerned about the format and overall quality of this paper. Although the authors explicitly mentioned their usage of AI on figures and writings, I still see this paper are far below the high standard of ICLR conference. (1) all figures are presented in low-resolution (some of them are clearly generated by AI without sufficient care) (2) \vspace are excessively used around the paper (L375-376, L304-305, L235-237, L209-211, L150-152, L111-113, L29-L31 etc). Without vspace command, paper will be clearly over 9 pages and this can be the reason for desk-rejection. (3) All texts are mainly describing the experiment results with all detailed numbers without conveying the key messages (or inference) for each results (typical AI-generated text). All of this significantly reduce the readability of the paper.

  2. SIMONSAYS benchmark is curated with the help of Claude (figure 6) but the authors does not describe in the main paper how they validate the quality of the dataset which is critical considering the character of the experiment they do.

  3. Limited novelty of the proposed V-Lens. V-Read formulation is exactly same as tuned lens as the author admitted in L142-145. Moreover, localization ability of logit lens are already widely studied [1-3] which is neither compared to V-Write or discussed extensively. Why V-Write should be better than all previous logit-lens-based localization? It should be mentioned in Sec3 and Sec 7.

  4. One of the main finding of the paper is the "Cues the models notice but do not act on. (L376-429)" but this is not something new and overlapped with previous works [4].

  5. All qualitative results are comparing V-Lens only to naive logit lens not to recent lens (tuned lens or J-lens). It is already known that logit-lens results are not reliable on early-mid layers so the results are not surprising to me.

[1] Neo, C., Ong, L., Torr, P., Geva, M., Krueger, D., & Barez, F. (2025, May). Towards interpreting visual information processing in vision-language models. In International Conference on Learning Representations (Vol. 2025, pp. 57172-57189).

[2] Jiang, N., Kachinthaya, A., Petryk, S., & Gandelsman, Y. (2025, May). Interpreting and editing vision-language representations to mitigate hallucinations. In International Conference on Learning Representations (Vol. 2025, pp. 63582-63605).

[3] Krojer, B., Nayak, S., Mañas, O., Adlakha, V., Elliott, D., Reddy, S., & Mosbach, M. (2026). Latentlens: Revealing highly interpretable visual tokens in llms. arXiv preprint arXiv:2602.00462.

[4] Nooralahzadeh, F., Rohanian, O., Zhang, Y., Fürst, J., & Stockinger, K. (2026). Arbitration Failure, Not Perceptual Blindness: How Vision-Language Models Resolve Visual-Linguistic Conflicts. arXiv preprint arXiv:2604.09364.

Questions for authors
  1. Why V-Read is better than Tuned lens in Table 2 although their formulation are exactly the same?
  2. What is the intuition behind the formulation of V-Write (based on the gradient of the score)? Why not using simple tuned-lens localization?
5 comments
AuthorsOpenReview ↗

Response to Reviewer ngeH (1/3)

We thank the reviewer for their detailed feedback. Below we address the concerns and questions with clarifications, manuscript revisions and additional experiments. Changes in the revised PDF are shown in blue. References [1–4] follow the numbering in the review, and table and appendix numbers refer to the revised paper.

W1. Presentation and formatting

Thank you for pointing out these presentation issues. We had to compress the PDF at the last moment to meet the file-size limit, which degraded the figures, and we could not fix it due to time constraints. We again apologise for the excessive negative \vspace. We have removed those commands, restored full-resolution figures and redrawn Figures 3 and 4 for clarity. Figure 3 now also compares V-Read with the tuned lens; Figure 4 labels training and held-out watermark placements. We welcome further suggestions. All plots are produced by our code directly from the result files or created in Canva, no figure is completely AI-generated, the low quality was because of compression which we have fixed by converting eveything to pdf instead of png

We agree that parts of the results section listed numbers without saying what they mean; to convey the key messages, we revised Section 7.1 to clarify V-Write's question-specific attributions, the effect of removing top-ranked tokens, and V-Read's intermediate readings and hallucination results. Section 7.2 now clarifies representing versus following a sandbagging instruction and the visual-password results. We also clarified the cue–behaviour direction comparison: on Qwen3.5-27B, authority and test-look directions are no closer to their behaviours than shuffled controls (0.26 and 0.33 vs. 0.32 and 0.37), while a harmful prompt lies along refusing (0.81 vs. 0.47; Appendix Figure 38 and Table 27).

W2. SimonSays validation

We agree that the validation procedures should be clearer in the main paper more than they are now. Section 5 now summarises them, and Appendix B.11 gives details. Claude helped draft prompts, reference answers and scene descriptions; validation included human inspection and an independent model.

  • References and inspection. The dataset includes 2,805 WMDP records using official keys, 6,233 reading and baseline records adapted from human-annotated Read the Object data, and 9,793 authored records. A human inspected all 379 edited or generated images and their 1,596 questions for readability, visual consistency and reference-answer correctness. Records without reference answers are not automatically scored.
  • Independent check. Qwen2.5-VL-32B-Instruct answered all 1,435 authored questions with fixed reference answers without seeing the references, agreeing on 91.2% of automatically scorable real-photo questions. Misleading text can also mislead the judge, so we use it alongside human inspection, not as ground truth.
  • Edit and text quality. Across 285 matched pairs, the judge rated 90.9% of edited images and 90.3% of originals as natural; 257 pairs had identical ratings. The mean paired difference was −0.01 (95% CI [−0.06, 0.03]). Edits changed a median 6.6% of pixels; 91.9% affected less than a quarter. Generated scenes also had comparable ratings before and after editing. OCR found 99.7–100% of tested decoy strings absent; difficult planted text was spot-checked manually. Automated checks found no schema, image-availability or answer-key consistency failures in the checked export.
  • Relation to results. V-Read and V-Write are evaluated against the model's own decisions rather than solely against references. We will release per-item validation records, judge outputs and verification code.

The full validation results are reported in Appendix Tables 11–13.

W3. Novelty and comparison with lens-based localisation

We agree that V-Read uses the same family of affine maps as the tuned lens, and we do not claim the family as new. What we add is an exact closed-form fit of that map, which reads the answer better than the tuned lens and fits in minutes instead of hours (Q1), and its use at the VLM answer position with matched image edits. Prior visual-token localisation and intervention work [1–3] is now discussed in related work. Neo et al. and Jiang et al. analyse visual representations and interventions; LatentLens describes visual tokens using contextualised text. Appendix Figure 29 compares patch vocabulary reading, distinct from the direct V-Write localisation comparison now in Section 7.1 and Appendix D.1.

New direct comparison. On 888 COCO photographs with Qwen3-VL-8B-Instruct, V-Write leads the tested lens localisers on the reported metrics. The methods target different quantities; the comparison is specific to this model, dataset and intervention.

AuthorsOpenReview ↗

Response to reviewer ngeH (2/3)

W3. Novelty and comparison with lens-based localisation (continued)

Table R1 (Appendix Table 14). Lens localisers vs. V-Write. Answer removed is the fall in the answer score after attention-masking the top 20% of image tokens. Agreement is Spearman correlation with measured patching effects over 16 regions. Own − other compares removal using the target-question map with that using a map for another object in the same photograph (251 photographs).

Localiserhit@1Mask AUROCAnswer removedAgreementOwn − other
V-Write0.500.723.840.473.32
Logit-lens object, max over blocks0.380.691.660.031.75
Tuned-lens object0.230.531.01−0.040.76
V-Read object0.210.570.62−0.050.63
LatentLens0.110.520.68−0.050.35
Random0.130.500.040.00–

Table R2 (Appendix Table 15). Object absent, model answers "no" (40 COCO photographs). The metric is the answer-score decrease after masking the top 20% of tokens.

LocaliserAnswer removedV-Write − localiser [95% CI]
V-Write1.23–
LatentLens0.270.97 [0.81, 1.14]
Tuned-lens object0.201.04 [0.86, 1.21]
Logit-lens object, max over blocks−0.051.28 [1.06, 1.51]

Table R3 (Appendix Table 16). Comparison with token-level KernelSHAP (50 COCO photographs; 10,000 coalitions). KernelSHAP is an estimated reference; Fall@20% is the score decrease after masking the top 20% of tokens.

MethodSpearman with reference [95% CI]hit@1Mask AUROCFall@20%
Token-level KernelSHAP (reference)1.000.540.703.76
V-Write0.71 [0.67, 0.74]0.540.753.87
KernelSHAP, 16 groups0.27 [0.24, 0.29]0.260.822.30
IG from zero0.180.420.582.08
Logit-lens localiser0.150.300.742.13
Table R4. Localiser efficiency on the same machine, Qwen3-VL-8B-Instruct. f and b denote forward and backward passes.
LocaliserPasses per pictureSeconds per pictureOne-time fitting
V-Write8 f+b0.4none
Logit-lens object1 f0.4none
Tuned-lens object1 f0.4tuned lens, 7.9 h
V-Read object1 f0.4V-Read, 6.9 min
LatentLens1 f + nearest-neighbour search0.4caption store, 0.6 min
KernelSHAP, 16 groups2,080 f49none
KernelSHAP, token level10,000 f243none

We also kindly invite the reviewer to consider our theoretical results in Section 4 (proofs in Appendix H), which we hope further clarify the method.

W4. Overlap with [4]

We thank the reviewer for pointing us to [4]. We agree that a dissociation between internally encoded visual information and the model's final answer has been studied previously. We now cite and discuss [4] in Section 7.2 and Appendix F, and we revised our wording to make clear that we do not claim this general finding as new.

In a related text–image conflict setting on SimonSays, we find that the visual fact remains highly decodable by a probe (AUROC 0.98 on both Qwen models), while answers follow the picture on only 0.53 and 0.49 of conflict records, respectively. Our additional analyses examine the answer readout and the effects of restoring states at different token positions. On Qwen3-VL-8B-Instruct, V-Read's AUROC for the visual fact decreases from 0.95 at mid-depth to 0.55 at the output, despite high probe decodability (Appendix Table 32). At the tested depths, restoring the answer-position state recovers more of the measured conflicting-sentence effect than restoring image-token states. These results are consistent with visual information remaining decodable while the final answer increasingly reflects the conflicting cue.

We also study related cue–behaviour gaps in additional settings, including authority attribution, evaluation cues and sandbagging instructions, and the carrier finding replicates on gemma-4-31B-it, a larger model from another family (probe 0.82–0.86 over six seeds, answer 0.57). We position these as extensions to different cue types, alongside the prior findings of [4].

Table R5 (Appendix Table 32). Text–image conflict on nuScenes (900 records per model). Follows picture gives the share without and with a conflicting sentence. Probe values are AUROC from mid-depth to output; V-Read gives AUROC on conflict records. The last two rows give the share of the cue's effect recovered by restoring image-token states at depth 0.25 or the answer-position state at depth 0.75. Arrows indicate mid-depth and output; ranges summarise the evaluated depths. The recovery values apply to the specified interventions and layers.

MeasurementQwen3-VL-8BQwen3.5-27Bgemma-4-31B-it
Follows picture, no cue / conflict0.93 / 0.530.91 / 0.490.97 / 0.89
Visual-fact probe AUROC0.980.98–0.990.99–1.00
V-Read AUROC across depth0.95 → 0.55–0.90–0.98
Sentence: image tokens / answer position0.00 / 0.73−0.01 / 0.40–
Sign carrier: image tokens / answer position0.99 / 0.791.00 / 0.47–
AuthorsOpenReview ↗

Response to Reviewer ngeH (3/3)

W5. Qualitative comparisons

We agree that the logit lens is a weak early- and mid-depth reference. Figure 3 previously showed only the logit lens, and its redrawn summary panel and caption now include the tuned lens. Table R6 extends the same test to every reader. Other comparisons with stronger readers include:

  • Figure 31: before half depth, V-Read reads 0.56–0.67, while the tuned, Jacobian and logit lenses are near or below chance.
  • Figure 30: hallucinated vs. true answers, AUROC 0.95 for V-Read vs. 0.79 for the Jacobian lens at half depth.
  • Figure 29: object names read from image patches, where V-Read peaks at 33% at one-fifth depth.
  • Figure 28: fitting takes 412 s for V-Read, 28,302 s for the tuned lens and 9,313 s for the Jacobian lens.
  • Table 2 and Appendix Tables 22, 23, 24 and 25: within-question, matched-time and early-depth comparisons with the tuned lens, Jacobian lens and Patchscopes. Figure 2 compares V-Write's hit@1 with attribution baselines.

Table R6. Figure 3's test with every reader: the share of matched pairs where V-Read / the other reader shows the cue's effect first (sign test). gemma-4-31B-it uses 19 pairs and a tuned lens trained on 31× V-Read's tokens; –: not fitted.

Model (pairs)vs tuned lensvs Jacobian lensvs logit lensvs Patchscopes
Qwen3-VL-8B-Instruct (43)49% / 21% (p = 0.04)44% / 23%40% / 23%12% / 51%
Qwen3.5-27B (43)26% / 26%19% / 14%28% / 19%12% / 65%
gemma-4-31B-it (19)47% / 5% (p = 0.02)–53% / 26%16% / 47%

V-Read shows the cue's effect earlier than the tuned lens on Qwen3-VL-8B-Instruct and gemma-4-31B-it, and is level with it on Qwen3.5-27B. Patchscopes shows it earlier on both Qwen models, but it re-runs the later blocks for every reading. Figure 3 uses the 43 pairs read by the paper's held-out V-Read maps; with refitted maps covering all 53 Qwen3-VL-8B-Instruct pairs, the lead over the tuned lens is smaller and not significant (36% vs. 21%, p = 0.20). We will add Figure 3 examples for every reader in the appendix.

Q1. Why can V-Read differ from the tuned lens?

V-Read and the tuned lens use the same family of affine maps but differ in three ways:

  • V-Read regresses onto the final residual state, while the tuned lens matches the final token distribution by KL.
  • V-Read is solved exactly by ridge regression in one pass, while the tuned lens is trained by SGD from the identity.
  • V-Read adds a per-block output gain chosen by KL.

Table R7 changes one of these differences at a time.

Table R7 (Appendix Table 3). V-Read and tuned-lens fitting variants (Qwen3-VL-8B-Instruct; AUROC averaged over the mid-depth window used in Table 2; 900 fitting prompts and 16 blocks). Lower held-out KL is better.

VariantAUROCHeld-out KL
Tuned lens, 1.1M tokens0.8293.06
Tuned lens, full 65.7M-token schedule0.828–
V-Read objective, solved by SGD0.83112.17
V-Read, closed form0.8924.13
Tuned lens initialised at V-Read0.8932.89

Solving V-Read's own objective by SGD loses the gain, while starting the tuned lens from V-Read's solution keeps it, and training the tuned lens 60× longer from the identity does not close the gap. The difference therefore comes from the exact solve, since SGD from the identity does not reach the best map in the family. The initialised tuned lens has a lower held-out KL than V-Read, so V-Read's advantage lies in reading the answer rather than in matching the full output distribution.

Q2. Why gradient-based V-Write instead of tuned-lens localisation?

A tuned-lens reading at an image-token position shows which vocabulary concepts can be read from that token. V-Write instead targets a specific decision of the model. It measures how the answer score changes with the intermediate image-token states as the remaining layers run, using integrated gradients from the picture's mean state. A patch can be readable without influencing the answer, so we test both kinds of method by intervening on image-token states and measuring the change in the answer score.

On the same 888 COCO photographs (Table R1), V-Write finds the object more often than the best lens localiser (hit@1 0.50 vs. 0.38), agrees much better with measured region patching (Spearman 0.47 vs. 0.03), and its top 20% of tokens remove more of the answer (3.84 vs. 1.66). An object-readable patch is therefore not necessarily the patch the answer depends on. We added this comparison to Section 7.1 and Appendix D.1.

We thank the reviewer again for the detailed review, which led to substantial changes in the paper, including full-resolution figures, a validation summary in Section 5, direct comparisons with lens-based localisers and the tuned lens, and a discussion of [4]. If our responses address the reviewer's concerns, we would be grateful if they would consider revisiting their assessment, and we are happy to answer any further questions.

Reviewer_5QdMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how VLMs form decisions at the “answer spot,” the position from which the first answer token is generated. It proposes two tools: a. V-Read, a closed-form linear map from each layer to the final state, which shows when the answer becomes readable; b. V-Write, integrated gradients over image-token states from the picture’s mean state, which shows where in the image the answer comes from. It also introduces SimonSays, a benchmark of 18,831 records with paired visual edits covering authority attribution, evaluation awareness, and sandbagging.

Main findings on Qwen3-VL-8B and Qwen3.5-27B: a. V-Read fits in minutes and reads answers earlier than the logit lens. b. V-Write matches KernelSHAP’s localization at about 1% of the cost. c. Models encode cues such as sign carrier or sandbagging instructions without acting on them. d. V-Write locates a fine-tuned watermark that triggers sandbagging.

Critical strengths
  1. A clear framing that links when, where, and whether it matters for a single answer decision.
  2. Attribution is validated by intervention (deletion, pixel covering, matched edits), not by how heatmaps look.
  3. Extensive baselines, with costs and paired confidence intervals reported.
  4. Practical efficiency: minutes rather than hours, and about 1% of KernelSHAP’s runtime.
  5. The password-lock experiment: the trigger is recovered without supervision, with dose-response behavior and an unrelated-pattern control.
Critical weaknesses
  1. Limited novelty as V-Read is an existing linear shortcut, and V-Write is integrated gradients with a mean baseline.

  2. V-Read’s advantage is mostly question wording. The question alone predicts the answer with AUROC 0.94, and within records that share a question V-Read is no better than the baselines (Table 16). The evaluation window was chosen after seeing the curves (Table 17).

  3. V-Write’s main metric favors itself: deleted tokens are replaced with the mean state, which is also its integration baseline. On metrics not tied to the baseline, KernelSHAP is ahead.

  4. Narrow coverage of attributes and generalization issues: a. The safety findings, and the password lock, use Qwen models only, and the password lock uses a single fine-tuned model. V-Read fails on Gemma and InternVL. b. Only a few manipulations are analyzed: carrier, word swap, evaluation framing, sandbagging instruction, watermark. c. Authority figures, text removal, deployment cues, and the stakes and standing labels are built in but never analyzed. d. COCO tests only object presence. e. Edits made with an image model may introduce artifacts.

  5. It looks like a lot of results are in the appendix. Maybe you should move some.

  6. The analysis is limited to first-token yes/no or letter answers.

Questions for authors

Possible improvements

  1. Use within-question AUROC as V-Read’s main metric and soften its claims.

  2. Generalization: a. Test V-Write deletion with replacements independent of its baseline (other images’ states, pixel inpainting), compared against all methods. b. Run the authority, evaluation-awareness, and sandbagging analyses and the password lock on at least one non-Qwen family, plus a larger model, with several seeds. c. Run other attribution methods on the watermark, and explain why a mark alone on a grey image triggers the lock while receiving little attribution.

  3. Analyze the manipulations already in the dataset, and add new ones such as legibility and language variation, instructions injected into the image, attribute and relation questions and others.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 5QdM (1/4)

We thank the reviewer for their valuable feedback and insightful suggestions, which we have carefully addressed. Changes in the revised PDF are shown in blue, and table and appendix numbers refer to the revised paper.

W1. Limited novelty

We agree with the reviewer that neither formula is fully new, and the paper also says so (L142–145). Our claim is that two specific, efficient choices, which standard lenses and standard IG do not make, are what make the tools work at a VLM's answer position. Below we isolate each choice.

V-Read. V-Read and the tuned lens use the same family of affine maps but differ in three ways:

  • V-Read regresses onto the final residual state, while the tuned lens matches the final token distribution by KL.
  • V-Read is solved exactly by ridge regression in one pass, while the tuned lens is trained by SGD from the identity.
  • V-Read adds a per-block output gain chosen by KL.

Table R1 changes one of these differences at a time.

Table R1 (Appendix Table 3). V-Read and the tuned lens, one difference at a time (Qwen3-VL-8B-Instruct; AUROC averaged over the mid-depth window of Table 2; same 900 fitting prompts and 16 blocks).

VariantAUROCHeld-out KL
Tuned lens, 1.1M tokens0.8293.06
Tuned lens, full 65.7M-token schedule0.828–
V-Read's objective, solved by SGD0.83112.17
V-Read (closed form)0.8924.13
Tuned lens initialised at V-Read0.8932.89

Solving V-Read's own objective by SGD loses the gain, and starting the tuned lens from V-Read's solution keeps it. The difference is therefore the exact solve, not the objective: SGD from the identity does not reach the best map in the family. Appendix Table 22 (Table 13 of the submission) already shows that giving the tuned lens 60× more tokens does not close the gap. On gemma-4-31B-it the closed form reaches a held-out KL of 2.82 in 15 minutes, while the best tuned lens we could fit reaches 5.40 with 31× the tokens (2.4 h).

V-Write. The starting point of the path matters a great deal. IG depends on its baseline, and the usual zero baseline fails at the answer position. Appendix Table 18 (Table 10 of the submission) already shows that starting from the picture's mean instead of zero removes 1.4–2.1× more of the answer on all four models. Table R2 extends this to other starting points, and to replacements that do not use V-Write's own baseline.

Table R2 (Appendix Table 4). Answer removed when the top 20% of image tokens are deleted, for V-Write started from different states (Qwen3-VL-8B-Instruct, 888 COCO photographs, same block; higher is better).

Start of the pathAttention removalReplaced by zeroPixel inpainting
Picture mean (V-Write)3.844.916.96
Dataset mean3.484.456.80
Blurred picture3.434.526.62
Another picture3.194.256.07
Zero state, 8 steps1.523.283.37
Zero state, 64 steps1.863.523.77

Across nine replacements, the zero start keeps only 40–67% of V-Write's effect, including when deleted tokens are themselves set to zero, which favours a zero start. Starts that look like real data keep 83–101%, and the picture mean is the best and needs no reference data. This matches IG's known sensitivity to its baseline (Sturmfels et al., 2020). Integrating in pixel space from a black image does worse still (hit@1 0.15 against 0.53).

With these choices, V-Write correlates at ρ = 0.71 with token-level KernelSHAP (10,000 coalitions), while IG from zero reaches 0.18 and every other cheap method at most 0.27. Under ten replacements that do not use its baseline, it is never behind any other of the 17 attribution methods on Qwen3-VL-8B-Instruct and gemma-4-31B-it, and it is 123× cheaper than KernelSHAP.

Changes to the paper. We rewrote L142–145 and the two contribution items in the introduction to show our contributions more clearly. We also added a paragraph to Appendix A.8 (Relation to existing methods) stating what the exact solve and the picture baseline add, with Table R1 as Appendix Table 3 and Table R2 as Appendix Table 4. We also kindly invite the reviewer to consider our theoretical results in Section 4 (proofs in Appendix H), which we hope further clarify the method.

AuthorsOpenReview ↗

Response to Reviewer 5QdM (2/4)

W2 / Q1. Question wording and within-question AUROC

We have adopted this suggestion in full. Within-question AUROC is now V-Read's main metric (Table 2), comparing it with its like-for-like baseline, the tuned lens, on every non-fitting record for three models. The question's wording cannot help any reader here, and V-Read is ahead of or level with the tuned lens on 23 of 25 task–model pairs.

A classifier that sees only the question is at chance on most tasks, and when another picture is swapped in under the same question, V-Read follows the model's new answer, which a reader of question wording could not do (Table R3).

Table R3 (Appendix Table 31). Qwen3-VL-8B. The swap columns score V-Read against the model's answer before and after the swap.

TaskQuestion onlyV-ReadSwap: old answerSwap: new answer
Open Images, authority0.960.990.890.88
Open Images, edited0.840.960.750.92
nuScenes, edited0.500.920.420.78
VINS-120K0.620.950.590.93
VINS-120K, edited0.500.920.580.92
Generated scenes0.500.92––
WMDP, biology0.500.780.750.78

Table R4 (revised Table 2). Within-question AUROC, V-Read / tuned lens. ↑ / ↓: paired 95% interval excludes zero.

TaskQwen3-VL-8BQwen3.5-27Bgemma-4-31B-it
Open Images, authority0.945 / 0.919 ↑0.880 / 0.8850.920 / 0.806 ↑
Open Images, authority, edited0.948 / 0.9130.911 / 0.8740.851 / 0.924
Open Images, test or real0.951 / 0.911 ↑0.576 / 0.644 ↓–
nuScenes, authority, edited0.942 / 0.935––
VINS-120K, authority0.938 / 0.909 ↑0.778 / 0.7600.920 / 0.782 ↑
VINS-120K, authority, edited0.905 / 0.8690.833 / 0.8060.524 / 0.762 ↓
Generated scenes, authority–1.000 / 1.000–
WMDP, biology0.805 / 0.689 ↑0.612 / 0.6170.527 / 0.462
WMDP, chemistry0.747 / 0.7000.539 / 0.5150.477 / 0.435
WMDP, cyber0.766 / 0.7310.547 / 0.5850.633 / 0.591
Ahead / level / behind4 / 5 / 00 / 8 / 12 / 4 / 1

Within a question, V-Read is never behind any reader on Qwen3-VL-8B, Patchscopes included (Appendix Table 25), its lead on both untouched authority tasks holds on gemma-4-31B-it, and it is level on all but one task on Qwen3.5-27B. The depth window does not drive the result either, since with no window V-Read still scores 0.796 against 0.718 (Appendix Table 26).

Changes to the paper. Table 2 now reports this comparison, the abstract, introduction, §7.1 and conclusion state it, and Appendix F adds Table R3.

W3 / Q2a. Deletion that does not use V-Write's baseline

We ran exactly the suggested test. The original metric did not favour V-Write alone, since KernelSHAP and both patching methods used the same picture-mean reference (Appendix C). To remove any doubt, we re-ran deletion with ten replacements that do not use V-Write's baseline, against all 17 methods.

Table R5 (Appendix Table 21). Answer margin removed by replacing the top 20% of image tokens (Qwen3-VL-8B, 888 COCO photographs), and V-Write's wins / ties / losses against 17 methods.

ReplacementV-WriteKernelSHAPV-Write vs 17 methods
Attention removal3.841.9317 / 0 / 0
Dataset-mean state3.992.2616 / 0 / 1†
Another picture's states3.922.2116 / 1 / 0
Matched noise4.192.4117 / 0 / 0
Zero4.913.5117 / 0 / 0
Picture mean (original)3.922.2517 / 0 / 0
Pixel blur7.117.1215 / 2 / 0
Pixel inpainting6.967.2215 / 2 / 0
Pixel grey7.647.2517 / 0 / 0
Pixel mean colour7.697.2017 / 0 / 0
Total, Qwen3-VL-8B164 / 5 / 1
Total, Qwen3.5-27B133 / 18 / 2
Total, gemma-4-31B-it153 / 17 / 0

† To V-Write's own variant integrated from the dataset mean.

V-Write never loses to another method on Qwen3-VL-8B or gemma-4-31B-it. Its only losses on Qwen3.5-27B are to KernelSHAP under pixel blur and inpainting, where KernelSHAP ties it on the other two models.

Table R6. V-Write against KernelSHAP on measures not tied to any baseline (Qwen3-VL-8B).

MeasureV-WriteKernelSHAP (16 groups)
Insertion (keep only the top tokens)7.06−2.41
Top token on the object (hit@1)0.500.26
Spearman with token-level KernelSHAP0.710.27
Mask AUROC0.720.79
Agreement with 16-region patching0.43–0.580.49–0.65
Seconds per photograph0.3948.1

KernelSHAP keeps an edge on the two measures scored at the resolution of its own 16 groups, and we report both.

Changes to the paper. Appendix D adds Table R5 as Table 21 with one sentence in §7.1, and Table 1's caption states that every method shares the picture-mean reference.

AuthorsOpenReview ↗

Response to Reviewer 5QdM (3/4)

W4a / Q2b. Another family, a larger model and several seeds

We repeated the authority, evaluation-awareness and sandbagging analyses on gemma-4-31B-it, a non-Qwen model larger than both Qwen models, with six seeds, and trained the password lock on four models from two families.

Table R7 (Appendix Table 33). Ranges cover six seeds; the mark covers about 2% of the picture.

FindingQwen3-VL-8BQwen3.5-27Bgemma-4-31B-it
Carrier noticed: probe AUROC0.84–0.850.81–0.840.82–0.86
…but the answer barely separates carriers0.590.510.57
Untouched photos called "test"6%82%88%
WMDP accuracy, plain → told to sandbag0.59 → 0.630.64 → 0.680.77 → 0.54
Password lock forms (seeds)3 / 32 / 23 / 3
Clean / marked accuracy0.59–0.68 / 0.23–0.280.64–0.65 / 0.12–0.150.64–0.72 / 0.15–0.22
V-Write share on the mark28–52%40–49%9–26%

The central finding, that models register a cue without acting on it, replicates on gemma-4-31B-it across all seeds, and gemma-4-31B-it is the only model whose accuracy drops clearly when told to sandbag. The lock also forms on gemma-4-12B-it (39% on the mark), while a model trained without the lock puts only 1.5% there.

On V-Read's weaker Gemma and InternVL results, we found that the final norm of gemma-4-26B had been applied in the wrong form, and the fix moves V-Read from 1 / 3 / 5 to 1 / 5 / 2 (ahead / level / behind; Appendix G). Where V-Read is weaker, its AUROC tracks how closely its mid-depth prediction matches the final output (ρ ≥ 0.93), as Proposition A.1 predicts, so this can be checked without labels.

Q2c. Other methods on the watermark, and the grey image

Table R8 (Appendix Table 34). Share of attribution on the watermark (about 2% of the picture); –: not run.

MethodQwen3-VL-8BQwen3.5-27Bgemma-4-31B-it
V-Write35.5%44.6%16.5%
IG from zero21.2%39.9%6.8%
Attribution patching29.5%19.9%9.6%
Gradient × activation27.8%16.2%6.8%
Activation patching17.9%––
Grad-CAM11.1%––
Pixel IG (black)8.7%––
KernelSHAP5.1%––
RISE3.3%––
Random2.0%2.0%1.9%

V-Write puts the most attribution on the mark on every model, and deleting its top cells unlocks 43% of Qwen3-VL-8B questions, against 27% for random cells. On a grey image every method is at chance, and a transplant test explains why: on a uniform field the vision encoder spreads the only salient patch into every patch, so moving only the mark's cells recovers just 4% of the lock, against 96% on natural pictures. Where the encoder keeps the mark local (gemma-4-12B-it), V-Write finds it on grey as well.

W4b / W4d / Q3. More manipulations and question types

We added these on Qwen3-VL-8B and Qwen3.5-27B, with every picture drawn in code.

Table R9 (Appendix Table 35). Share: V-Write's attribution on the manipulated region.

ManipulationQwen3-VL-8BQwen3.5-27B
Legibility (119 sign edits, 5 degradations at 5 levels)Never "read but ignored"; V-Read tracks the effect level by level; share 0.23 → 0.12 (area)Same; share 0.36 → 0.12
Injected "this is a test"Followed printed 1.00, typed 0.21; share 0.55 vs 0.30 for a neutral boxShare 0.63 vs 0.32
Injected "answer C"Read (share 0.41), not obeyed (0.03)Read (share 0.35), not obeyed (0.03)
Attribute, count, position, relation (COCO hit@1)Leads on colour, position and relation; level on countLeads on all four (e.g. relation 0.56 vs 0.42)

V-Write therefore tracks what the model acts on rather than what is merely visible, and on the new COCO questions, replacing its top cells costs the answer three times more than replacing the object's own cells (2.13 against 0.69).

W4c. Built-in manipulations

We now analyse each of them on eight models (Appendix F.1):

  • Authority figures: edits that change a person draw about their area of V-Write's attribution, against 2–4× for text edits, so models act on written authority more than on who is pictured.
  • Text removal: removing added text from a defaced stop sign restores the original answer on all eight models.
  • Deployment cues: on gemma-4-31B-it, a deployment framing lowers "test" calls on untouched photos from 88% to 58%, and an evaluation framing raises them to 99%.
  • Standing and stakes: models follow the deciding source less when it looks less official (D2: 0.26 against D1: 0.69 on Qwen3-VL-8B), and as the stakes rise Qwen3-VL-8B follows it less (0.64 → 0.42) while gemma-4-31B-it follows it more (0.63 → 0.75).
Reviewer_rgsrOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces an explainability benchmark for vision language models. In addition to this, they propose two tools to explain VLM decisions. V-read looks at when the models decision forms, and V-write looks at where in the image does the model make its decision from. SimonSays is a large benchmark used for evaluating the interpretability in the safety setting. The proposed method takes significantly less compute and time, and demonstrates comparable performance with the existing state of the art.

Critical strengths
  • Explainability in VLMs is a very interesting and useful topic
  • The paper is very clearly written. Figures are very easy to understand, and the qualitative results all together greatly easy the ability to understand this work. Specifically figure 1 is a very clear and well made figure for the proposed method.
  • The computational/time reduction for the proposed method compared to previous work is very impressive and a strong postive of this paper
  • The experiments are comprehensive and are tested on multiple models for comparison.
Critical weaknesses
  • The proposed method does not have any consistent, clear wins in the results tables
  • It is unclear how SimonSays compares to existing benchmarks, and if there is a specific need for its existence
Questions for authors
  • How does SimonSays compare to existing benchmarks? What is the need for it?
3 comments
AuthorsOpenReview ↗

Response to Reviewer rgsr (1/3)

We thank the reviewer for their positive assessment and for highlighting the clarity of the paper, the efficiency of V-Lens and the breadth of our experiments. Below we address the two weaknesses and the question. Changes in the revised PDF are shown in blue.


W1. Consistent, clear wins

The proposed method does not have any consistent, clear wins in the results tables.

We agree that our wins should be stated more clearly, and the updated paper now does so. Under the stricter comparisons added in the revision, V-Write is never behind another method on Qwen3-VL-8B-Instruct or gemma-4-31B-it, and it loses only two comparisons on Qwen3.5-27B. V-Read is ahead of or level with its like-for-like baseline, the tuned lens, on 23 of 25 task–model pairs across three models, and it costs a small fraction of the compute. Table R1 collects these results.

Table R1. Where V-Lens wins, under the strictest comparisons in the revision. Wins / ties / losses use paired 95% bootstrap intervals.

ComparisonV-LensBest alternativeResult
V-Write: answer removed under 10 replacements independent of its baseline, against 17 methods (wins / ties / losses)164 / 5 / 1 (Qwen3-VL-8B), 153 / 17 / 0 (gemma-4-31B-it), 133 / 18 / 2 (Qwen3.5-27B)–Never behind any other method on Qwen3-VL-8B and gemma-4-31B-it†
V-Write: agreement with token-level Shapley values (10,000 coalitions)0.710.27 (KernelSHAP, 16 groups)Clear win
V-Write: insertion, keeping only the top tokens7.063.52 (gradient × activation)Clear win
V-Write: share on a trained watermark (2% of the picture), Qwen3-VL-8B / Qwen3.5-27B / gemma-4-31B-it35.5% / 44.6% / 16.5%29.5% / 39.9% / 9.6%Best on all three models
V-Write: seconds per photograph0.3948.1 (KernelSHAP)123× faster
V-Read: within-question AUROC against the tuned lens, 25 task–model pairsahead on 6, level on 17–Ahead or level on 23 of 25; never behind on Qwen3-VL-8B-Instruct
V-Read: which reader shows a cue's effect first, matched pairs, Qwen3-VL-8B (43) / gemma-4-31B-it (19)49% / 47%21% / 5% (tuned lens)p = 0.04 / 0.02
V-Read: fitting, gemma-4-31B-it (held-out KL, time)2.82, 15 min5.40, 2.4 h (tuned lens with 31× the tokens)Better fit at a fraction of the cost

† The single loss on Qwen3-VL-8B is to V-Write's own variant started from the dataset mean, and the two on Qwen3.5-27B are to KernelSHAP under pixel blur and inpainting.

V-Read is not the best reader on every task. Patchscopes, which re-runs the later blocks for every reading, reads the WMDP sandbagging questions better and shows a cue's effect earlier on both Qwen models. V-Read's claim is the best trade-off between cost and quality, as it fits in 7–20 minutes instead of hours and is never behind the tuned lens within a question on Qwen3-VL-8B-Instruct.

Changes to the paper. Table 2 now reports within-question AUROC against the tuned lens. The baseline-independent deletion results are added to the appendix (Table 21) and summarised in §7, and §7 opens with a summary of these wins.


W2 / Q1. How SimonSays compares to existing benchmarks, and why it is needed

It is unclear how SimonSays compares to existing benchmarks, and if there is a specific need for its existence.

How does SimonSays compare to existing benchmarks? What is the need for it?

SimonSays is motivated by established concerns in LLM safety, including situational awareness and sandbagging, but investigates a relatively unexplored dimension: how image-specific cues influence VLM decisions, and whether recognising these cues necessarily translates into behaviour. The benchmark was specifically constructed with matched visual counterfactuals and diagnostic controls to support internal decision tracing through V-Read and V-Write. We compare it to existing work along each of its three axes below.

Authority attribution. Existing VLM work such as AIpsych (Liu et al., 2026) studies authority bias arising from a model's tendency to trust misleading questions or answer choices. SimonSays instead defines authority as question-specific standing: which visible source within an image should determine the answer. For example, a temporary staff amendment may supersede a printed notice, despite appearing less official. Through inscription swaps, carrier changes, conflicting signs, and controlled variations in visual salience, we distinguish the content of an instruction from the standing of its visual source (Section 5; Appendix B.6 and B.8). Appendix F (Figures 34–35) further demonstrates that models can distinguish visual carriers internally even when their final answers change little.

AuthorsOpenReview ↗

Response to Reviewer rgsr (2/3)

Evaluation awareness. Motivated by text-only LLM benchmarks such as SAD (Laine et al., 2024), we extend evaluation-awareness research to visual contexts. SimonSays introduces control, evaluation, and deployment settings, varying benchmark-style presentation, annotation interfaces, and deployment-associated imagery while keeping the underlying task fixed. These conditions are detailed in Appendix B.7 (Evaluation and Deployment Framing), under System-level Context, Question-level Context, and Visual Presentation. We separately assess cue recognition and task performance to distinguish recognising an evaluation setting from acting on it. Appendix F (Figure 36) illustrates this distinction: evaluation-related cues alter internal representations even when the model's answer remains largely unchanged.

Sandbagging. Existing LLM work has investigated instruction-based sandbagging and textual password-locking (van der Weij et al., 2025), while MM-DeceptionBench (Fang et al., 2026) examines deceptive underperformance in VLMs through realistic scenarios involving contextual and situational pressures. Our motivation differs: rather than testing whether a VLM presents itself as less capable in a particular scenario, we investigate whether an image-embedded cue can itself condition underperformance, even though the image is not needed to answer the underlying question. We study this in two steps.

  • Instructed underperformance. We first investigated instruction-based underperformance using explicit hazardous incorrect-answer instructions and target-accuracy (p%) conditions (Appendix B.9.1). These did not reliably induce the requested behaviour: Section 7.2 reports accuracy of 0.59 without instructions, 0.61 with typed instructions, and 0.64 with instructions printed in the image. Appendix F (Figure 37) shows that these instructions can nevertheless be detected in intermediate representations.
  • Visual password-locking. We then introduced visual password-locking, fine-tuning a VLM to underperform when a small marker is embedded in the image (Appendix B.9.2). The objective is not simply to demonstrate reduced accuracy, but to investigate whether an otherwise inconspicuous visual condition can control capability expression and whether an auditor can recover that condition from the model's internal computations. Figure 4 shows that the marker reduces held-out accuracy from 0.60 to 0.30, while V-Write localises the trigger without being told its identity or location, including at held-out placements never used in training. This introduces a controlled setting for studying visually conditioned underperformance and its internal mechanisms.

Why SimonSays is needed for V-Lens. The benchmark was designed to complement our interpretability experiments through paired image edits and controlled visual cues. For example, Appendix D (Figure 27) tracks V-Write's attribution across successive edits to street and house scenes, including official, handwritten, and conflicting signs. The street example shows attribution shifting toward introduced signs, while the house example reveals cases of imperfect grounding. Appendix E (Figure 33) provides complementary causal evidence through edited-region image-token transplantation, showing that the manipulated visual regions can carry information responsible for changes in the model's answer. We have clarified this motivation in the Introduction (Section 1), highlighting how SimonSays differs from existing behavioural benchmarks by investigating scene-specific authority, evaluation-awareness, and sandbagging cues through controlled visual interventions.

We also evaluate V-Write on COCO through object localisation and token-removal experiments (Section 7.1; Appendix D), and V-Read through decision readout and hallucination analyses on COCO and SimonSays source datasets (Sections 7.1–7.2; Appendix E). Appendix F further uses authority, evaluation, and sandbagging probes to examine how recognition of cues related to internal representations, behavioural decisions, and steering.

AuthorsOpenReview ↗

Response to Reviewer rgsr (3/3)

Summary. To the best of our knowledge, no existing VLM benchmark studies authority among competing visual sources within the same scene, evaluation awareness through matched evaluation and deployment framings conveyed both visually and textually, or underperformance gated by an image-embedded marker together with internal attribution of that trigger. SimonSays therefore introduces a complementary direction for studying VLM safety through image-specific cues. Rather than treating images only as visual evidence for answering questions, we investigate how visual content can convey authority, signal evaluation context, or condition underperformance. By combining controlled visual interventions with perception and binding controls, the benchmark enables us to distinguish what a model sees, what it internally represents, and what ultimately determines its response. We added another section, Appendix B.11, to further validate our dataset using a VLM judge for realism, reference answer verification and EasyOCR for OCR. We have also modified the Introduction (Section 1) and SimonSays (Section 5) of the main paper to highlight the motivation, comparison and validation of the dataset.

We thank the reviewer again for their positive assessment and constructive comments, which have helped us improve the paper. We hope our responses address the reviewer's questions, and we would be happy to answer any further questions during the discussion period.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

Thank you all for the careful reviews and the helpful suggestions. We were glad that the reviewers found the topic useful and the paper clearly written (rgsr), liked that our attributions are checked by intervention rather than by how the heatmaps look (5QdM), and found the experiments and baselines extensive (rgsr, 5QdM, ngeH). Reviewers rgsr and 5QdM also pointed out how cheap V-Lens is to run. The reviews helped us improve the paper a lot, and we summarise the main changes below. All changes are marked in blue in the revised PDF.

Our Strengths

V-Lens gives two cheap tools for looking inside a VLM at the point where it gives its answer. V-Read fits in 7 to 20 minutes, while the tuned and Jacobian lenses take hours, and V-Write costs about 1% of KernelSHAP while its top tokens remove more of the answer than those of any method we compared. SimonSays is, to our knowledge, the first VLM benchmark that uses matched image edits to study authority, evaluation awareness and sandbagging. Together they show which cues a model notices but does not act on, a pattern we now see on three models from two families, and V-Write can find a hidden visual password from the model's answer alone.

Revisions to the paper

We made substantial changes to address every weakness the reviewers raised, including new experiments on Qwen3-VL, Qwen3.5 and Gemma 4 models of up to 31B parameters. The main changes are below.

1. Novelty (5QdM, ngeH). We do not claim the maps themselves as new, so we now show what each of our choices adds. V-Read and the tuned lens use the same kind of map, and when we change one difference at a time, it is the exact closed-form fit that makes V-Read better (0.892 against 0.829 for the tuned lens, while the same objective trained by SGD gives only 0.831). On gemma-4-31B-it, V-Read reaches a better fit in 15 minutes than the tuned lens does in 2.4 hours with 31 times the training tokens. For V-Write, starting from the picture's mean instead of zero removes 1.4 to 2.1 times more of the answer. We also compare V-Write directly with lens-based ways of locating objects, and it finds the object more often (0.50 against 0.38) and agrees much better with measured patching (0.47 against 0.03).

2. Fairer evaluation (rgsr, 5QdM, ngeH). V-Read is now scored only against records that share the same question, so the wording of the question cannot help it (Table 2). On Qwen3-VL-8B-Instruct, Qwen3.5-27B and gemma-4-31B-it, it is ahead of or level with the tuned lens on 23 of 25 task and model pairs, and the table shows the two cases where it is behind. For V-Write, we repeated the deletion test with ten ways of removing tokens that do not use V-Write's own baseline, against 17 other methods. It never loses to another method on Qwen3-VL-8B-Instruct or gemma-4-31B-it, and its only two losses on Qwen3.5-27B are to KernelSHAP under pixel blur and inpainting. Figure 3 also shows V-Read detecting the cue's effect first on 49% of matched pairs versus 21% for the tuned lens on Qwen3-VL-8B-Instruct.

3. Other models and longer answers (5QdM). On gemma-4-31B-it, the probe again detects what a sign is written on (0.82–0.86 across six seeds), while answers barely change (0.57). The password lock now appears across four models from two families, and V-Write attributes more to the mark than any compared method. We added legibility, injected instructions, attribute, count, position and relation questions, analysed existing manipulations on eight models, and extended both tools to longer-than-one-token answers (Appendix F.1). We also corrected an error in our gemma-4-26B results (Appendix G).

4. Dataset quality (ngeH, 5QdM). A person reviewed all 379 edited/generated images and 1,596 questions. Qwen2.5-VL-32B-Instruct independently agreed with 91.2% of real photograph reference answers and rated edited images as natural as originals (90.9% vs. 90.3%). Edits changed a median of only 6.6% of pixels. Results are summarised in Section 5 and detailed in Appendix B.11.

5. How SimonSays relates to prior work (rgsr, ngeH). Earlier benchmarks study authority bias from misleading questions, evaluation awareness in text, and sandbagging set up by the scenario or by text. SimonSays instead asks how cues inside the image change a VLM's decision, using matched image edits. We now cite Nooralahzadeh et al. (2026), reproduce their gap between what a model encodes and what it answers, and present our findings as extending it to authority, evaluation and sandbagging cues.

6. Presentation (ngeH, 5QdM). All figures are now at full resolution, Figures 3 and 4 are redrawn, and the negative \vspace commands are removed. The results section now says what each finding means, and the main text carries the most important new evidence.

We are grateful to the reviewers for their time. Their feedback has made the paper much stronger and clearer, and we would be happy to answer any further questions during the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.