acceptodds
Under review as a conference paper at ICLR 2027

Does RL Post-Training Ground Vision-Language Models? A Causal Measurement

22 views

Abstract

Reinforcement learning (RL) post-training raises the benchmark accuracy of vision-language models (VLMs), but it is unclear whether the gains come from better use of visual evidence or from stronger language priors. Accuracy cannot settle this, since many benchmark items are answerable without the image, and neither can attention, which need not reflect how the image is used. We introduce CARVE, a programmatic benchmark of solver-certified counterfactual edits: each edit changes one piece of visual evidence, certifies the new answer, and records the image region it occupies. We test whether answers follow the edit, whether they follow it when a printed value stays the same but moves to a different element, and whether the edited region's hidden states alone redirect the answer. Across four Qwen-VL backbones trained with GRPO, VPPO, and PRPO, post-training improves edit following modestly on three backbones, yet models that follow a numeral edit keep their original answer after 10.9–39.5% of edits that only move a label, against at most 1.7% of numeral edits, in all 16 configurations; post-training does not remove this failure. The gain is not reliably carried by the evidence: the edited region reproduces 43% of the behavioral gain in our 3B models (95% CI [18, 67]) but 96% in our 7B models ([66, 139]), and the 3B pattern persists with a frozen vision tower, locating the change in the decoder. Answer-changing evidence stays at visual positions rather than moving into the question, and attention to the image rises after training even where the causal role of visual states falls. Grounding claims for VLMs should therefore be tested against evidence whose effect on the answer is known, not inferred from accuracy or attention.

Primary area:
interpretability and explainable AI
Keywords:
vision-language models, reinforcement learning post-training, visual grounding, counterfactual evaluation, activation patching, mechanistic interpretability, evidence binding

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_7BsiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces \textbf{CARVE}, a solver-certified counterfactual benchmark whose edits change one piece of visual evidence (a printed value, or the element that value refers to), certify the new answer symbolically, and record the image region MM carrying the change. By using programmatic visual edit approach, authors propose \textbf{Causal Evidence Following (CEF)}, \text{BEF} and other metrics, carries experiments based on that, this paper claims that RL post-training raises \emph{behavioral} evidence following on three of four backbones but (i) leaves a persistent \emph{prior lock} on referent edits (10.9--39.5% of such edits keep the original answer vs.\ ≤1.7%\le 1.7\% on numeral edits in all 16 configurations), (ii) does not reliably route the gain through the edited region ($\sim$43% gain share at 3B; wide interval at 7B), with the frozen-tower pattern implicating the \emph{decoder} rather than the visual encoder, and (iii) decouples attention-to-image from causal use, cautioning against reading grounding from accuracy or attention alone.

Critical strengths
  • Great cerfification pipeline. Answers are solved from scene parameters before rendering, so they never depend on pixels; the mask is a verified pixel difference with nothing changed outside it. Crucially, the filters audit the post-edit answer distribution, not just the question text, and model correctness is deliberately excluded as a filter. This is a measurement instrument, not another paired-image suite.
  • "Prior lock" is a genuinely new diagnostic that survives the obvious objections. Models follow a changed numeral but keep the old answer when the same value moves to another element. The gap persists within scenes that carry both edit types. Failed referent edits return the original answer rather than a different printed value. Readout probes show that most locked edits are read and located correctly.
Critical weaknesses
  • Limitation in Causal Evidence Following experiments. The patched sequence mixes two global summaries of the image, so it corresponds to no image that could be rendered, and the transferred region carries only a quarter of the edit's effect. A post-training drop in CEF can therefore mean either that the model uses the edit less or that it resists an internally inconsistent patch, which the paper's controls cannot tell apart. The complement region and a consistency-preserving donor would.
  • The gain share is a ratio of two differences of a few points each, and it is not identified at that precision. At 7B the estimate exceeds 100% ([66, 139]), which cannot be read as a share, and the 54 point gap over 3B ([16, 101]) rests on two sizes of one family with one seed. At 3B the frozen-tower value (54.1%) sits above the trainable one (42.7%), so the ordering a localization story would need is not stable. The four underlying accuracies, per objective, would be more informative than the ratio, and the 7B PRPO run, which collapses on Geo3K (40.6 to 24.1, one fixed answer on 11% of items), should be justified before it enters a pooled headline.
Questions for authors
  • For the complement region: what is transfer accuracy at layer 0 when you copy V∖S(M)V \setminus S(M) instead of S(M)S(M), per configuration? If the complement alone recovers most of edited-image accuracy, then the gap between seeing the edit and transferring it reflects how much of the image the patch covers rather than where the evidence sits. Either outcome would change how I read Section 6.3, so I would like the number even if it is unflattering.
  • For the consistency problem: can you rerun CEF with a donor that keeps the figure self-consistent, for instance a MODE-L item where the numeral glyph is rewritten but the geometry it annotates is left unchanged? The current donor differs from the recipient both at the edited tokens and in its global summary, so a smaller CEF after training admits two readings. If the base-to-trained gap survives a donor that is closer to renderable, the weaker reading is ruled out and my main objection goes with it.
  • For the gain share: could you report the four accuracies behind it (edited-image and transfer, base and trained, per objective) rather than the pooled ratio, and state whether the 7B PRPO run is included? With a ratio of two differences of a few points, the per-objective accuracies are what I can actually check against the claim, and I would rather see them than the quotient.
2 comments

We thank the reviewer for the careful review and for recognizing CARVE as “a measurement instrument, not another paired-image suite” and prior lock as “a genuinely new diagnostic that survives the obvious objections.” We ran the three experiments the reviewer asked for; each supports the paper's reading of Section 6.3, which we will sharpen.

W1/Q1. What is transfer accuracy when the complement is patched?

Transfer accuracy when the complement is patched is low in every configuration: copying the complement at layer 0 gives the edited answer on only 12–15% of pairs, against 52–69% for the edited image, although the complement holds 91% of the visual tokens and 71–74% of the edit's representation change. The evidence the model uses sits at and around the edit, not across the image, so the gap reflects where the evidence sits, not how much the patch covers.

Table A. Accuracy (% of the paper's 1,545 numeral pairs on which the run gives the edited answer y'), layer 0, all 11 configurations (published CEF values reproduce exactly). Representation change: edited minus original, squared, summed over visual tokens; range over 3B and 7B. RL = mean of GRPO, VPPO, PRPO.

Edited imageEdited region (CEF)Complement of regionUnrelated tokens
Visual tokens patched (%)1009919
Representation change carried (%)10026–2971–74–
3B base52.440.711.71.7
3B RL, trainable tower61.344.514.22.7
3B RL, frozen tower60.044.813.42.4
7B base62.746.015.31.7
7B RL69.452.512.91.7

W1/Q2. Does the base-to-trained gap survive a self-consistent donor?

Yes. The smaller CEF after training is the 3B pattern in Figure 4: transferring only the edited region loses 27–28% of edited-image accuracy after RL, against 22% for the base model. In 9 of our 26 numeral templates (tables, Venn counts, path weights, number puzzles, sector, number train) nothing else in the figure moves when the numeral changes (pixel difference under 0.3% of the image), so the donor is already the self-consistent figure the reviewer describes: the original with only the numeral glyph rewritten. Rerunning CEF on these 538 pairs reproduces the same pattern: the region loses 24% of the base model's accuracy and 31% of the RL models' with a trainable tower (28% with a frozen tower). The larger drop after training therefore survives a donor with nothing inconsistent in it, which rules out the inconsistency reading. At 7B there is no larger drop to explain, in Figure 4 or here.

Table B. Glyph-only numeral pairs (538), layer 0. Drop = (edited region − edited image) / edited image. Last column: the same drop with the published donor (Figure 4).

ModelEdited imageEdited regionDrop (%), glyph-onlyDrop (%), Figure 4 (all pairs)
3B base52.640.0−24.0−22
3B RL, trainable tower72.350.0−30.8−27 to −28
3B RL, frozen tower69.149.9−27.8−25 to −27
7B base71.757.1−20.4−27
7B RL73.761.3−16.8−24 to −25

W2/Q3. What are the four accuracies behind the gain share, and is 7B PRPO included?

Table C gives the four accuracies behind the gain share for every objective, on the same 1,545 numeral pairs. The 7B PRPO run is included in the paper's pooled 7B gain share and is shown here as its own row.

Table C. The four accuracies per objective, on the paper's 1,545 numeral pairs, layer 0. Accuracy = % of pairs on which the run gives the edited answer y'. Edited image: the model sees the edited figure. Transfer: only the edited region is patched into the original figure (the paper's CEF). The frozen-tower runs start from the same base model.

Objective3B trainable tower: edited image3B trainable tower: transfer3B frozen tower: edited image3B frozen tower: transfer7B: edited image7B: transfer
Base52.440.752.440.762.746.0
GRPO61.445.061.746.569.152.4
VPPO61.443.956.441.467.550.9
PRPO61.044.661.846.571.754.1

7B PRPO. The run does not collapse on CARVE: it has the highest edited-image accuracy of the 7B runs (71.7). The Geo3K drop the reviewer notes is real, and we will report it next to the CARVE numbers.

W2. Frozen tower. The paper does not claim an ordering between frozen and trainable towers; it claims that freezing the tower leaves the same gap between seeing the edit and transferring it (Figure 4: −25% to −27% frozen, −27% to −28% trainable). The pooled shares agree within error (54.1% vs 42.7%, difference +11.4, 95% CI [−5.8, +30.5]). With a frozen tower the edited region's states stay exactly the base model's, so the gap cannot come from the encoder, which is why we locate the change in the decoder.

We will add these controls and the four-accuracy table to Section 6.3, together with a notation table, one running example, and boxed definitions of BEF and CEF. We thank the reviewer again; the complement and consistency questions led directly to a sharper statement of the CEF result.

Reviewer_iueVOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether reinforcement learning (RL) post-training improves visual grounding in vision-language models beyond improvements in benchmark accuracy. The authors introduce CARVE, a synthetic benchmark consisting of solver-certified counterfactual image edits that distinguish changes to visual values from changes to their referents. They also propose behavioral and causal evidence-following metrics to examine whether model predictions respond to these edits and whether representations of the edited regions causally influence the answers. The evaluation covers four Qwen-VL backbones trained using GRPO, VPPO, and PRPO, along with additional analyses using activation patching, frozen vision encoders, and attention measurements. The results show that RL post-training modestly improves behavioral evidence following in most configurations, while referent-binding failures largely persist. The causal analyses further reveal differences in localized evidence use across model sizes and show that increased attention to visual tokens does not necessarily correspond to greater causal reliance on visual evidence.

Critical strengths

The paper investigates an important research question: whether improvements from RL post-training in VLMs reflect better use of visual evidence or stronger reliance on learned priors. It proposes an approach that uses solver-certified counterfactual edits and causal analysis to examine this. The experiments reveal meaningful findings, particularly that models can correctly follow changes to numerical values while failing to associate those values with their visual referents, a limitation that largely persists after RL post-training. These findings provide useful insights into the limitations of current multimodal post-training approaches and the need for more targeted evaluations of visual grounding.

Critical weaknesses

1. Limited insights into how RL post-training affects visual grounding. While the paper identifies interesting failures in visual evidence use, the experiments provide limited insight into why these failures persist or whether RL post-training can address them. All methods are trained on ViRL39K using answer-correctness rewards, whereas grounding is evaluated on a separate synthetic benchmark. Consequently, it is unclear whether the observed limitations arise from RL post-training itself or from training objectives that do not explicitly reward correct visual grounding. An experiment using a grounding-aware reward on a separate set of synthetic training examples would be particularly informative. Without such analysis, the main findings remain largely diagnostic and provide limited guidance on improving visual grounding through RL.

2. Limitations in the causal interpretation of visual evidence use. The proposed causal evidence following metric is interesting, but its interpretation has important limitations. The localized interventions are restricted to numeral edits, and the authors acknowledge that only around 26–29% of the encoder's representation changes are contained within the edited region. Consequently, unsuccessful localized transfer does not necessarily indicate that the model is failing to use the relevant visual evidence, since this information may be distributed across other visual tokens. This makes it difficult to draw strong conclusions about how much of the post-training improvement is actually attributable to the relevant evidence, particularly the reported difference between the 3B and 7B models.

Questions for authors
  1. All RL variants are trained on ViRL39K using answer-correctness rewards, while grounding is evaluated on a separate synthetic benchmark. How do the authors distinguish limitations of RL post-training from a mismatch between the training objectives and the grounding capabilities being evaluated? In particular, to what extent does the training data require explicit value-to-referent binding?
  2. The paper reports that models frequently retain the original answer after referent edits, even when they correctly identify the moved values and their associated elements in the reading and binding probes. Does this necessarily indicate a failure of visual grounding or binding, or could it instead reflect a failure in downstream reasoning or answer updating? How do the authors distinguish these possibilities?
  3. The authors acknowledge that only around 26–29% of the encoder's representation changes are localized within the edited region. Given that visual information may be distributed across other tokens, how confidently can the reported gain shares (43% for 3B versus 96% for 7B) be interpreted as differences in causal reliance on the relevant visual evidence, rather than limitations of the localized intervention?
10 comments

We thank the reviewer for the careful review, and for recognizing that the paper investigates “an important research question” and that its findings provide “useful insights into the limitations of current multimodal post-training approaches.” We address the concerns below, including the experiment the reviewer proposed.

W1. Does the paper explain why grounding failures persist, and can RL address them?

Our paper is diagnostic by design: its goal is not to propose a new grounding objective, but to identify what existing accuracy-based evaluations miss. We show that models can follow changes to numerical values while retaining their original answers when those values move between referents (prior lock), a failure that persists across the 16 evaluated configurations. Identifying this failure through certified edits and causal tests provides a concrete target for future RL methods. Importantly, the new experiment below demonstrates that this diagnosis is actionable.

Q1. Limitation of RL, or a mismatch between the training objective and the evaluated capability?

We respectfully note that this mismatch is the question the paper asks rather than a confound in it. VLMs are built to generalize beyond their training data, and RL post-training is applied precisely to improve generalization to unseen problems; RL-trained models are routinely evaluated on benchmarks they were not trained on. CARVE asks whether that generalization comes from better use of visual evidence. The diagnosis also explains why it does not: let T be the answer from the textbook rule with values in their usual places, and B the answer from binding each value to the element it labels. On standard figures T = B, so reading the figure and applying the template earn the same reward.

The requested experiment. We retrained Qwen2.5-VL-3B with the paper's GRPO run (same ViRL39K data, correctness reward, hyperparameters and seed), adding 3,990 synthetic CARVE figures (about 11% of the data), half with a referent edit so that T ≠ B and only correct binding is rewarded. We tested on 5 CARVE templates not used in training and on 7 figure families from a separately designed generator.

Table A. Qwen2.5-VL-3B, the paper's GRPO run with and without T ≠ B figures (%). Lock = % of edits answered with the old answer, among edits whose original the model solved. Differences: paired, 95% item-bootstrap interval.

GRPOGRPO + T ≠ B figuresDifference
Unseen CARVE templates: referent lock25.617.1−8.5 [−13.1, −4.0]
Unseen CARVE templates: referent accuracy69.674.4+4.7 [+0.9, +8.7]
CARVE numeral edits: lock3.12.3−0.9 [−1.7, −0.1]
Unseen figure families: referent lock56.152.1−5.3 [−10.3, −0.5]
Unseen figure families: new answer21.429.8+11.0 [+7.0, +15.0]
MMK12 / ViRL39K held-out accuracy63.0 / 61.962.8 / 61.5−0.2 / −0.4

Findings. With the same reward and algorithm, T ≠ B figures in the training mix reduce prior lock on templates and figure families the model never saw, while general accuracy is unchanged. The objective, not RL itself, may therefore be a part of the cause. However, it is not a complete solution: referent lock on unseen templates remains 17.1%, seven times the numeral lock (2.3%), and on the unseen families the old answer is still kept on half of the edits. We will add this experiment as a new section.

Q2. Binding failure, or a failure of reasoning or answer updating?

Each alternative predicts a different pattern, so we can check them against what the model actually does:

  • If the model could not read the figure, it would miss the moved label. However, on locked edits (where it keeps the original answer after the label moves), it lists every printed number and names the element that now carries the moved value in 58–71% of cases for trained 3B models and 79–87% at 7B (Table 8). This shows the model sees the label and knows where it sits.
  • If the model reasoned badly, its wrong answers would be scattered. Instead, when it fails a referent edit on a figure it solved, it most often gives exactly the original figure's answer (60–74% of failures), and another number printed in the figure in only 1–8% (Table 7). That answer is the correct calculation, done on the old label positions.
  • If the model failed to update its answers in general, it would also keep the old answer when a number is changed. In fact, it keeps the old answer on at most 1.7% of numeral edits, against 10.9–39.5% when only a label moves (Figure 5).

Together, these results indicate a failure to use the updated value-to-referent mapping in the final prediction, rather than necessarily a failure to perceive or identify that mapping (we will clarify this distinction). For example, in Figure 10, the 7B GRPO model writes “∠C = 42°” and then answers 82, the answer for the original figure.

W2 / Q3. Is localized CEF valid when the edit's representation change is distributed?

CEF measures whether the selected tokens are sufficient to recover the counterfactual prediction, not whether all edit information is confined to them. We added three checks on the paper's numeral pairs for all 11 models.

1. Spatial controls. The complement holds about three-quarters of the representation change but recovers only 15–20% of answers; the region plus one ring of neighbours holds less than half and recovers 79–85%. Size-matched unrelated tokens recover 0–4%. The information the answer uses sits at and next to the edit.

Table B. Among pairs where patching all visual tokens gives the edited answer, the share where the listed patch alone also gives it (%), numeral edits, layer 0, 11 models.

PatchShare of representation changeShare of visual tokensRecovers
Edited region (CEF)26–29%≈9%62–70
Region + one ring of neighbours43–47%≈22%79–85
All tokens except the region72–76%≈90%15–20
Unrelated tokens, same count–9% / 22%0–4

2. The gain shares (43% at 3B, 96% at 7B). With one ring of neighbours, 3B and 7B carry the RL gain at the evidence to the same degree:

Table C. Share of the RL gain in edited-image accuracy reproduced by each patch (%), 95% item-bootstrap interval. First row = the paper's numbers.

Patch3B, trainable tower3B, frozen tower7B
Edited region (paper)42.7 [18, 67]54.1 [27, 83]96.5 [66, 139]
Region + one ring of neighbours92.9 [71, 121]79.2 [55, 109]95.9 [69, 132]

The 3B-versus-7B difference therefore reflects how tightly the exact region fits the 3B evidence, not a different reliance on it. No other conclusion rests on the scale contrast, and prior lock and BEF are behavioural and do not depend on CEF.

3. CEF separates evidence use from disruption. Here we score each patch against the model's own answer on the edited image, right or wrong, and split pairs by whether the model solved the original figure.

  • If CEF only measured how much a patch disturbs the model, both groups would transfer equally.
  • They don't. Patching the edited region plus one ring reproduces the model's answer on 72–78% of pairs whose original it solved, but on only 43–50% of pairs whose original it did not, in every one of the 11 models. The difference is +30 points [28, 32] for the region plus one ring, and +26 [24, 28] for the edited region alone (Table D). Unrelated tokens reproduce almost nothing (under 1%).

Answers on figures the model can solve come from the evidence and its neighbours. The other answers still depend on the image, but draw less on the evidence the question is about. So CEF measures evidence use, not disruption.

Table D. Patch reproduces the model's own edited-image answer (%), numeral edits, layer 0, by whether the model solved the original figure. Brackets: 95% scene-clustered bootstrap.

ModelsOriginal solved?Edited regionRegion + 1 ringUnrelated tokens
3B baseyes / no63.0 / 36.178.1 / 49.90.1 / 0.6
3B RL (6 runs)yes / no55.9 / 31.772.4 / 43.20.3 / 0.6
7B baseyes / no60.1 / 30.777.3 / 46.60.1 / 0.2
7B RL (3 runs)yes / no64.0 / 36.778.1 / 47.20.1 / 0.8
All 11, difference+26.1 [23.8, 28.3]+29.8 [27.6, 32.0]−0.4

We will state the sufficiency reading of CEF, report ring CEF next to exact-region CEF, and revise the 3B–7B discussion in Section 6.3. The revision also adds a notation table, one running example and boxed definitions of BEF, CEF, gain share and prior lock. Thank you again; the question about the objective led to the most useful new result in the revision.

Thanks for the rebuttal, but my intention for W1/Q1 was a bit different. The RL models that the authors evaluated were trained with answer-only GRPO, which doesn't directly incentivize better grounding or better use of visual evidence, as long as the final answer is correct. Since the paper's central premise is "it is unclear whether the gains come from better use of visual evidence or from stronger language priors," it leaves the following question open. If the training did incentivize grounding (there have been several recent works on grounding-aware supervision during RLVR for VLMs), does that change the conclusion for those models under the paper's proposed evaluation setup?

Reviewer_5ifFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CARVE, a controlled visual reasoning benchmark designed to disentangle how well VLMs use visual evidence in reasoning. Specifically, the benchmark constructs paired images with the same question but minimally edited visual content that leads to different correct answers, allowing evaluation of whether changes in visual evidence correctly steer model predictions, using metrics such as Behavioral Evidence Following (BEF). The authors then apply RL post-training to a series of open VLMs and evaluate its effects on visually grounded reasoning. They find that RL generally improves behavioral evidence following, but models are substantially less reliable at following referent edits (e.g., moving labels) than numeral edits (e.g., changing printed values). Furthermore, these improvements are not consistently recovered in a controlled setup that only swaps in hidden states from the edited image regions.

Critical strengths
  1. The controlled evaluation is valuable, as accuracy on many existing VLM reasoning benchmarks is confounded by text-only reasoning that does not rely on visual information.
  2. The different results between referent edits and numeral edits are interesting, suggesting that incorporating spatial information correctly into visual reasoning remains challenging for the evaluated models.
Critical weaknesses
  1. The Behavioral Evidence Following (BEF) metric, as reported in results such as Figure 3, is conditioned on the model correctly answering the original question, so its interpretation may be confounded by differences in baseline accuracy. It would be helpful to also report accuracy on the original images (P(ŷ = y)) and edited images (P(ŷ' = y')) alongside the BEF results.
  2. The use of Causal Evidence Following (CEF) as an analysis metric is not sufficiently justified. As the authors themselves observe, representations of patches outside the edited region can also carry information about the image edit. Replacing hidden states only within the edited region may not constitute a well-controlled localized intervention. At worst, it could disrupt the model's internal representations, while even under more favorable conditions, it may omit important information distributed across other regions.
  3. The evaluated model scales are relatively small. While I think it is reasonable to focus on smaller models for the main experiments, it would be valuable to see at least some validation on frontier models to assess the generalizability of the findings. For example, does the gap between referent edits and numeral edits also persist in frontier models such as GPT, Claude, or Gemini?
Questions for authors

The paper appears to use two terms, "Behavioral Evidence Following" and "conditional edit success," interchangeably for BEF, which is somewhat confusing. Could the authors clarify and standardize the terminology?

3 comments

We thank the reviewer for the careful review, and for recognizing that our approach is “valuable” and “interesting,” particularly since “visual reasoning remains challenging.” We address the three concerns below, including new experiments and clarifications to the paper.

W1. Could BEF improvements simply reflect conditioning on different sets of correctly answered original images?

No. To further look into this, we added unconditional accuracy on both original and edited images for all 5,960 CARVE pairs. All six RL configurations improve edited-image accuracy by 4.4–7.0 points (3B) and 4.7–5.0 points (7B), so the gains are not an artifact of BEF conditioning.

Table A. All 5,960 CARVE Mixed pairs, Qwen2.5-VL. Accuracies and BEF in %; Δ = trained minus base, 95% item-bootstrap interval.

ModelClean accEdited accBEFΔ edited acc
3B base37.435.660.0–
3B GRPO43.842.665.1+7.0 [+5.6, +8.4]
3B VPPO41.740.062.4+4.4 [+3.1, +5.8]
3B PRPO44.741.562.2+5.9 [+4.6, +7.3]
7B base41.741.566.7–
7B GRPO48.646.569.6+5.0 [+3.8, +6.3]
7B VPPO47.246.467.6+4.9 [+3.7, +6.1]
7B PRPO48.946.168.9+4.7 [+3.4, +5.9]

Fixed-set check (existing, Appendix C, Table 4): Restricting evaluation to the same originally solved examples across configurations still yields significant gains: +5.1–7.6 points at 3B and +2.0–3.9 at 7B (Holm-adjusted). We will report unconditional accuracies alongside BEF in the revision.

W2. Is CEF a valid localized intervention if information from the edited region spreads to neighboring visual tokens?

CEF measures whether the selected tokens are sufficient to recover the counterfactual prediction, not whether all edit information is confined to them. We agree that visual encoding can spread information beyond the edited region. We therefore added three checks:

1. Spatial controls. Edited-region tokens (≈9% of the image) recover 62–70% of successful counterfactual predictions, versus 0–4% for unrelated, size-matched regions. Adding one ring of neighbors raises recovery to 79–85%, while transferring everything except the edited region recovers only 15–20%. Thus, the effect is spatially specific but not perfectly confined.

Table B. Percentage of successful full-image counterfactual predictions recovered by each intervention (numeral edits, layer 0). Image share indicates the fraction of visual tokens transferred.

PatchImage share3B base3B RL, trainable tower3B RL, frozen tower7B base7B RL
Edited region (CEF)≈9%6962–6463–686669–70
Region + one ring of neighbours≈22%8282–8479–848383–85
All except edited region≈90%1819–2018–202015–17
Unrelated, size-matched regions9% / 22%12–41–210–2

2. RL-gain attribution. At 3B, the gain recovered by patching increases from 42.7% to 92.9% when neighbors are included (trainable tower); at 7B it remains around 96%. The original 3B–7B difference therefore depends on the patch boundary: some 3B gains are represented in nearby tokens, not necessarily disconnected from visual evidence.

Table C. Share of the RL gain in edited-image accuracy reproduced by each patch (%), 95% item-bootstrap interval. First row = the paper's numbers.

Patch3B, trainable tower3B, frozen tower7B, trainable tower
Edited region (paper)42.7 [18, 67]54.1 [27, 83]96.5 [66, 139]
Region + one ring of neighbours92.9 [71, 121]79.2 [55, 109]95.9 [69, 132]

3. Prediction-specificity check. We score each patch against the model's own edited-image answer, right or wrong, and split pairs by whether the model solved the original figure. If CEF only measured how much a patch disturbs the model, both groups would transfer equally, but they don't. Across 11 configurations, patching the edited region reproduces the model's answer 26.1 points more often when the original was solved; region plus one ring: +29.8. Answers on figures the model can solve draw on the evidence location; the others draw on it much less.

Table D. Patch reproduces the model's own edited-image answer (%), numeral edits, layer 0, by whether the model solved the original figure. Brackets: 95% scene-clustered bootstrap.

ModelsOriginal solved?Edited regionRegion + 1 ringUnrelated tokens
3B baseyes / no63.0 / 36.178.1 / 49.90.1 / 0.6
3B RL (6 runs)yes / no55.9 / 31.772.4 / 43.20.3 / 0.6
7B baseyes / no60.1 / 30.777.3 / 46.60.1 / 0.2
7B RL (3 runs)yes / no64.0 / 36.778.1 / 47.20.1 / 0.8
All 11, difference+26.1 [23.8, 28.3]+29.8 [27.6, 32.0]−0.4

We will clarify this sufficiency-based interpretation of CEF and revise the discussion of the 3B–7B comparison in Section 6.3.

W3. Should the absence of frontier VLMs be considered a critical weakness?

We agree that frontier-model behavior is important, but we respectfully disagree that its absence should be a critical weakness for a study centered on causal analysis of RL post-training. Proprietary frontier models generally do not expose the training checkpoints or internal activations required for controlled pre/post-RL comparisons and activation patching. Open-weight VLMs are therefore the appropriate experimental setting for the paper's central questions, and are widely used in reproducible academic research. Nevertheless, we added the requested behavioral evaluation: Claude Opus 5.5 and GPT-6.1 on 300 matched scenes each, and Gemini 3.1 Pro on 84, using the same prompting and scoring protocol.

Table E. Evaluation on matched scenes. Referent lock measures original-answer retention after referent edits, conditioned on correctly solving the original image. Gap denotes referent lock minus numeral lock (percentage points).

ModelOriginal accReferent-edit accReferent lockGap
Claude Opus 5.599.098.30.0+0.0
GPT-6.1100.099.70.0−0.3
Gemini 3.1 Pro (84 scenes)100.097.60.0+0.0
Qwen2.5-VL-3B base / GRPO61.3 / 75.040.3 / 61.332.6 / 27.1+31.0 / +25.3
Qwen2.5-VL-7B base / GRPO74.3 / 77.762.3 / 64.719.7 / 21.9+19.7 / +21.0

Frontier models achieve 97.6–99.7% referent-edit accuracy and 0% observed referent lock on these subsets, whereas GRPO-trained Qwen2.5-VL-3B/7B still show 27.1%/21.9% referent lock. This shows that the edits are answerable and, importantly, narrows our conclusion: referent-binding failures are substantial in the studied open models, not universal across all VLMs. We will make that scope explicit in the revision.

Thank you for the clarification. In particular, the response to W2 helps strengthen the paper's analysis.

Regarding W3, while I do not expect all findings to remain applicable to frontier models, identifying insights that extend to these models and have practical implications would significantly enhance the contribution.

Overall, I would like to raise my score while encouraging the authors to further identify findings with broader applicability.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Notes to Area Chair and Reviewers

Dear Area Chair and Reviewers,

We sincerely thank all three reviewers for their thoughtful and constructive feedback. All reviewers found the question important and recognized CARVE as a certified measurement instrument and prior lock as a new diagnostic. Your comments have motivated substantial additional experiments strengthening the validity of our causal measurements, the robustness of our behavioral findings, and the scope of our conclusions. We hope the reviewers will consider these new results and engage in the ongoing discussion.

We briefly summarize how we addressed the principal concerns.

Reviewer 5ifF: Unconditional accuracy on all 5,960 CARVE pairs shows that every RL configuration improves edited-image accuracy (+4.4 to +7.0 points at 3B, +4.7 to +5.0 at 7B), so the BEF gains are not an artifact of conditioning on solved originals. New spatial controls show that CEF is spatially specific: the edited region (≈9% of visual tokens) recovers 62–70% of successful counterfactual predictions, the region plus one ring of neighbours 79–85%, and size-matched unrelated tokens only 0–4%. With one ring, the RL gain share is 92.9% at 3B and 95.9% at 7B, so the scale contrast reflects the patch boundary. We also evaluated three frontier models (Claude Opus 5.5, GPT-6.1, Gemini 3.1 Pro): they show no referent lock, which confirms that the edits are answerable, while GRPO-trained Qwen2.5-VL-3B/7B still lock on 27.1%/21.9% of referent edits.

Reviewer iueV: We ran the requested grounding-aware experiment: the paper’s GRPO run with T ≠ B CARVE figures added, with the same data, reward, hyperparameters and seed. Referent lock fell on unseen templates (25.6% → 17.1%) and on unseen figure families (−5.3 points), with general accuracy unchanged (MMK12 63.0 → 62.8). The diagnosis is therefore actionable, but the problem is not solved: lock remains seven times the numeral lock. We further showed that the failure is binding rather than perception, reasoning or answer updating: locked models read and locate the moved label, most often return exactly the original answer, and keep the old answer on at most 1.7% of numeral edits versus 10.9–39.5% of referent edits.

Reviewer 7Bsi: Patching the complement of the edited region gives the edited answer on only 12–15% of pairs (vs 52–69% for the edited image), although it holds 91% of the visual tokens and 71–74% of the edit’s representation change. On 538 glyph-only pairs, a self-consistent donor in which nothing but the numeral changes, the larger post-RL drop at 3B is reproduced (−31% vs −24%). We report the four accuracies behind every gain share, including 7B PRPO as its own row (the highest 7B edited-image accuracy, 71.7%), and show that freezing the vision tower leaves the same gap, which locates the change in the decoder.

We would also like to clarify two points that bear on the overall assessment.

Scope of the contribution. The paper is diagnostic by design. It tests a growing assumption that gains from RL post-training reflect better visual grounding, and shows that accuracy-based evaluations cannot establish it: models follow a changed number yet keep the old answer when the same value moves to another element. Proposing a new training method is outside this scope, but the experiment requested by Reviewer iueV shows that the diagnosis directly points to what training must change.

Frontier models. The paper’s questions are causal and require pre- and post-RL checkpoints and internal activations, which proprietary models do not provide. Open-weight models are therefore a methodological necessity rather than a convenience. Our frontier evaluation scopes the claim explicitly to the open models studied, without affecting the causal findings.

We have carefully addressed every comment and concern raised, including additional experiments and analyses that we believe have further strengthened our work.

We fully understand that reviewing takes considerable time and effort, and we truly appreciate the reviewers’ contributions. While we would have welcomed further engagement during the discussion period to clarify any remaining concerns, we sincerely hope that our detailed responses have satisfactorily addressed the issues raised.

If so, we would be very grateful if the reviewers could reconsider their original scores and support the acceptance of our work. We strongly believe in the importance of this study and hope that our efforts to address all the feedback are reflected in the final evaluation. We would be glad to answer any remaining questions in the time left.

Thank you again for your valuable time and consideration.

Sincerely,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.