acceptodds
Under review as a conference paper at ICLR 2027

Seeing What Matters: Reward-Anchored Visual Credit Assignment via Self-Distilled Reinforcement Learning

Abstract

Reinforcement learning with verifiable rewards improves multimodal reasoning through outcome-level feedback, yet its sequence-level advantage is broadcast across all generated tokens. This obscures which tokens are associated with decisive visual evidence during generation and limits reliable learning from visual evidence. A natural way to address this limitation is to introduce privileged visual evidence and contrast model predictions across different visual conditions, thereby providing finer-grained learning signals for vision-critical tokens. However, the token-level differences induced by these visual conditions do not in themselves constitute reliable credit signals. They may be confounded by factors unrelated to fine-grained visual evidence and may not align with the optimization direction indicated by outcome-level rewards. To this end, we propose **VisCA**, a self-distilled reinforcement learning framework for reward-anchored visual credit assignment. VisCA treats privileged visual differences as candidate evidence for token-level credit assignment rather than directly using them as credit signals. It introduces a matched counterfactual visual condition to assess whether these differences are attributable to fine-grained visual evidence. The verified signal is then calibrated across sibling rollouts. It is subsequently used to modulate the verifier-derived outcome advantage at visually informative tokens, without changing the reward-determined optimization direction. Across six fine-grained visual understanding benchmarks, VisCA improves the average score over the corresponding base models by 7.04 points at 4B scale and 6.21 points at 9B scale. It also maintains strong average performance on held-out tasks that probe broader multimodal capabilities.

Primary area:
reinforcement learning
Keywords:
Multimodal Large Language Model, Self-Distilled Reinforcement Learning, Fine-Grained Visual Understanding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cUVHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors present VisCA, a self-distilled RL framework that breaks down rewards in vision-language models by assigning credit to specific visual regions. Their approach constructs two types of views: evidence-centered crops (privileged) and degraded counterfactual versions. By checking whether these conditions agree, they verify which tokens actually depend on fine-grained visual evidence rather than just correlating with it. This verified signal adjusts outcome-based advantages from a verifier while keeping the reward-driven optimization direction intact. Testing on Qwen3.5 models (4B and 9B parameters) shows 6-7 point average improvements across six fine-grained visual understanding benchmarks, with performance holding steady on separate multimodal tasks. VisCA outperforms GRPO and several self-distillation baselines when trained on the same synthetic dataset.

Critical strengths
  • The counterfactual verification mechanism tackles a real problem in using privileged information for credit assignment. It's a more careful attempt at separating correlation from causation than what I've seen in prior work on this topic.

  • Results hold up across multiple benchmarks and two model scales, and the gains transfer to held-out tasks. This suggests the method isn't just memorizing the training distribution.

  • The semantic analysis in Section 4.4 offers solid evidence that verified credit actually lands on visually grounded tokens instead of linguistic scaffolding. This validates the method's intended behavior rather than just showing aggregate metrics.

Critical weaknesses
  • Training exclusively on 6.2K synthetic examples from Vision-OPD makes me question whether these gains reflect real capability improvements or just overfitting to a narrow distribution. There's no comparison against models trained on more diverse or larger-scale data.

  • The core assumption, that directional agreement between privileged and counterfactual differences indicates visual attribution, lacks rigorous justification. Why does this particular consistency check reliably filter out confounding factors? The paper doesn't really explain the mechanism.

  • Experimental validation sticks to tasks where fine-grained visual details are explicitly required. What about scenarios where attention and reward relevance might diverge, like tasks requiring the model to deliberately ignore salient distractors? Those cases aren't tested.

Questions for authors
  1. How does VisCA perform when the model needs to actively ignore visually salient but task-irrelevant features? Does counterfactual verification still produce reliable credit in those adversarial cases?

  2. Can you compare against simpler spatial decomposition schemes, uniform grid-based reward attribution or gradient-based saliency, for instance, to show whether the counterfactual machinery is actually necessary?

  3. What fraction of tokens typically pass the counterfactual consistency check? How does this vary across different task types or training stages? Does the sparsity of verified credit shift as the model improves?

  4. How sensitive is the method to the choice of privileged region? If the bounding box is slightly misaligned or includes irrelevant background, does the verification mechanism degrade gracefully or collapse?

5 comments

Thank you for your review and constructive feedback. We address generalization from the 6.2K training set, explain the rationale for counterfactual verification, and describe comparisons with simpler attribution methods. We examine verified-credit sparsity across training stages, robustness to task-irrelevant visual features, and sensitivity to the privileged region.

1. Training-Data Rationale and Generalization (W1)

  • Training-data rationale. We use the 6.2K examples released by Vision-OPD [1], which are constructed around questions answerable from localized visual evidence. This design addresses a common difficulty in fine-grained perception, where a model may recognize a detail in an isolated crop but fail to use it effectively in the full image. Pairing full images with privileged regional views aims to improve the model’s use of local evidence, rather than to tailor training examples to the six evaluation benchmarks. VisCA builds on these paired views to derive token-level visual credit, aiming to translate the model’s existing local recognition ability into more effective use of task-relevant evidence during full-image reasoning.

  • Evidence of generalization. To assess whether the gains extend beyond fine-grained visual tasks, we evaluate on three held-out visual tasks, MMVP, CV-Bench, and MMStar, which were not used for VisCA post-training or hyperparameter selection. VisCA achieves higher average scores on the held-out tasks than the base model at both scales, with 80.69 versus 78.99 at 4B and 82.98 versus 82.44 at 9B, and outperforms GRPO, OPSD, RLSD, and Vision-OPD on the same held-out average. All methods are trained on the same 6.2K samples using the same training recipe. If these gains were driven by overfitting to this distribution, the baselines trained on the same data would also show similar improvements, but they do not.

ScaleMethodMMVPCV-BenchMMStarAverage3\mathrm{Average}_3
4BVanilla76.0086.9074.0778.99
GRPO72.6784.4668.5375.22
OPSD80.3386.9869.7379.01
RLSD74.3384.4968.4075.74
Vision-OPD79.3387.1070.7379.05
VisCA (Ours)81.6787.5372.8780.69
9BVanilla81.6788.3777.2782.44
GRPO79.0087.9275.7380.88
OPSD81.0088.1371.9380.35
RLSD80.0088.4274.0080.81
Vision-OPD82.6788.1775.8082.21
VisCA (Ours)83.6788.8776.4082.98

2. Mechanistic Justification of the Consistency Check (W2)

  • Decomposition of the visual contrasts. For a sampled token yty_t, we hold the teacher, question, and response prefix fixed and denote its log-probability under view vv by ℓt(v)\ell_t(v). We compute

    e_t^{\mathrm{vis}} = \ell_t(x^+) - \ell_t(x), \qquad e_t^{\mathrm{ctr}} = \ell_t(x^+) - \ell_t(x^-). \end{xalignat*}$$ These satisfy the exact identity $$\begin{xalignat*}{1} e_t^{\mathrm{vis}} = e_t^{\mathrm{ctr}} + c_t, \qquad c_t = \ell_t(x^-) - \ell_t(x). \end{xalignat*}$$ Here, $e_t^{\mathrm{ctr}}$ measures sensitivity to detail suppression within the same crop, while $c_t$ captures the remaining difference between the degraded crop and the full image, including region and context changes.
  • Why directional agreement. The gate retains a signal only when etvisetctr>0e_t^{\mathrm{vis}} e_t^{\mathrm{ctr}} > 0. If the privileged crop increases a token’s likelihood, retaining positive credit requires that suppressing its fine details decreases that likelihood. Conversely, negative credit is retained when the details suppress the token. In the decomposition above, when ctc_t opposes and outweighs etctre_t^{\mathrm{ctr}}, the privileged–full difference reverses direction and is rejected. The check therefore filters signals whose direction conflicts with the matched detail contrast. Using signs tests directional consistency without requiring the two contrasts to have equal magnitudes.

3. Robustness to Task-Irrelevant Visual Distractors (W3 & Q1)

To evaluate VisCA’s robustness to visually salient but task-irrelevant features, we conduct a distractor test on ZoomBench. Its full images and question-relevant regional crops allow us to place high-contrast solid-color blocks outside the evidence region while keeping the questions and reference answers unchanged. As shown below, VisCA’s accuracy decreases slightly from 60.47% on clean images to 60.00% with distractors, a drop of 0.47 percentage points. VisCA remains effective under the tested distractions, suggesting that salient but irrelevant visual content does not readily disrupt its task performance.

BenchmarkClean+DistractorΔ
ZoomBench60.4760.00-0.47

4. Comparisons with Simpler Attribution Methods (Q2)

Thank you for suggesting these comparisons. We are making every effort to complete comparisons with uniform grid-based reward attribution and gradient-based saliency to assess the added value of counterfactual verification. We will update this response as soon as the results become available.

5. Fraction and Evolution of Verified Credit (Q3)

Across training stages, the fraction of tokens passing the counterfactual consistency check starts higher early in training and gradually decreases to 29.3% at the final checkpoint. This shift in sparsity reflects how the model evolves as it improves. Early in training, the full-image policy remains close to the base model and has not yet internalized fine-grained perception, so differences between full-image and privileged views are spread across many tokens with small yet sign-consistent margins. As training progresses and the policy internalizes localized visual understanding, full-image predictions align more closely with the privileged view on routine tokens, concentrating verified visual credit onto a sparser subset of decisive tokens.

6. Sensitivity to the Privileged Region (Q4)

At inference the model receives only the full image, and all benchmarks are box-free. The privileged region is a training-time construct, so a misaligned or missing box at test time has no effect. During training, the region is used only to build the pair x+x^+ and x−x^-. Since the two views share the same region, comparing them may reduce the influence of mild misalignment or shared irrelevant background when these affect both views similarly. The gate remains a consistency check, and the credit becomes less specific rather than invalid. As shown in the table below, the matched 0.10 degradation is best at 77.73. A random crop gives 76.17, and removing the image gives 76.80. Even the worst setting remains above the Base (72.54) and close to the raw visual-evidence variant (76.27). The mechanism therefore degrades gracefully rather than collapsing, and the reward anchor keeps the update direction unchanged.

Counterfactual ViewV*BenchZoomBenchHR-Bench 4KHR-Bench 8KMME-RW ENMME-RW CNAvg
No Image88.4860.0086.0082.6272.5171.1876.80
Random Crop88.4858.8285.1281.1272.5570.9176.17
Mask89.0157.2887.2583.7572.6072.2577.02
Blur_0.0591.1059.1785.8883.8872.9371.5777.42
Blur_0.1092.1560.4787.2582.6272.6971.1877.73
Blur_0.2588.4860.8384.6283.2571.3870.8176.56

Thank you once again for your thoughtful review and helpful feedback. We hope the explanations above resolve your concerns and clarify the issues you highlighted. We will revise the manuscript to include these clarifications and the additional analyses. We would be glad to provide further details should you have any remaining questions.

[1] Yuan, Q., et al. (2026). Vision-OPD: Learning to See Fine-Grained Details for Multimodal LLMs via On-Policy Self-Distillation. NeurIPS.

4. Comparisons with Simpler Attribution Methods (Q2)

Thank you for suggesting these comparisons. We have now completed experiments with uniform grid attribution and gradient-based saliency to assess the added value of VisCA’s counterfactual verification.

Uniform grid attribution. We partition the full image into a fixed (2×2) grid. For each generated token, we compute the difference between the teacher’s log probability under each grid crop and that under the full image, keeping the question and response prefix fixed. We then average these differences over the four crops to obtain token-level visual credit.

Gradient-based saliency. For each generated token, we compute the gradient of the teacher’s log probability with respect to the visual embeddings under the full-image input. We then average the gradient norms across visual positions to obtain a token-level sensitivity score.

As shown below, VisCA improves the average score by 3.12 and 2.08 points over uniform grid attribution and gradient-based saliency, respectively, highlighting the advantage of counterfactual verification for token-level visual credit assignment.

MethodV*BenchZoomBenchHR-Bench 4KHR-Bench 8KMME-RW ENMME-RW CNAvg
Uniform grid attribution87.9657.9982.0080.2570.6968.7874.61
Gradient-based saliency89.0158.7083.2581.3871.5869.9775.65
VisCA (Ours)92.1560.4787.2582.6272.6971.1877.73
Reviewer_XZLwOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VisCA uses privileged regional views to refine token-level credit in multimodal RLVR. An EMA self-teacher compares sampled-token likelihoods under the full image, an evidence crop, and a degraded version of that crop. A sign-agreement gate filters the visual signal, which is calibrated across sibling rollouts and used to adjust outcome-derived advantages, with clipping to prevent sign reversal. Modulated and unmodulated tokens are normalized separately in the policy objective. The authors train on region-annotated synthetic visual question-answering data and evaluate the method on fine-grained visual understanding benchmarks and held-out multimodal tasks, with component ablations.

Critical strengths
  1. The method adds an explicit visual contrast before using privileged-view differences for credit. Comparing an evidence crop with a degraded counterpart supplies a criterion for filtering the full-image/crop signal. Combining this signal with outcome-derived advantages connects the visual guidance to task-level reinforcement. The verification ablation in Table 3 reports an improvement over the raw visual-evidence variant, supporting the value of this component.

  2. The reported gains are substantial at both evaluated model scales. VisCA exceeds GRPO's six-task average by 3.45 points at 4B and 2.31 points at 9B. The held-out average also improves over the corresponding base models, supporting the practical value of the approach beyond the six fine-grained benchmarks.

Critical weaknesses
  1. The unacknowledged overlap with VPPO raises a serious attribution concern. The analysis in Section 3.1, from Definition 3.1 through Observations 1–2 and Figure 2, is substantially similar in its analytical content, organization, and wording to VPPO [1], Sections 3.2.1–3.2.2, from Definition 3.1 through Insights 1–2 and Figures 3–4. Yet VPPO is not cited or acknowledged. This calls into question the provenance and originality of the motivating analysis. Separately, the model, data, and sampling protocol behind Figure 2 are unspecified.

  2. The baseline for the visual-credit ablation is undefined. Table 3's “Base” scores 72.54, differing from both the untrained model (70.69) and GRPO (74.28) in Table 2. Without its training objective and checkpoint, the gain attributed to adding raw visual credit is difficult to interpret.

  3. The EMA teacher is insufficiently specified and motivated. Section 3.3 introduces an EMA teacher without reporting its decay coefficient or update schedule, or explaining its choice over the current policy. Because all visual-credit scores come from this teacher, these omissions leave the signal-generation procedure incompletely specified and the effect of teacher choice unclear.

[1] Spotlight on Token Perception for Multimodal Reinforcement Learning.

Questions for authors
  1. Please explain the textual and visual overlap with VPPO and identify any material adapted from that work. Why is it not cited, and what model, data, and sampling protocol were used to generate Figure 2?

  2. What training objective and checkpoint does “Base” in Table 3 represent? Is adding raw visual credit the only change from “Base” to “+Visual Evidence”?

  3. What decay coefficient and update schedule were used for the EMA teacher? Why was it chosen over the current policy for computing visual-credit signals?

4 comments

Thank you for your time and effort you have devoted to reviewing our work. Below, we address the overlap with VPPO, clarify the configurations of Base and the ablation variants and how they differ, and explain the EMA teacher settings and the rationale for its use.

1. Relationship to VPPO and Figure 2 Protocol (W1 & Q1)

  • Relationship to VPPO and attribution. The analysis in Section 3.1 follows VPPO’s diagnostic perspective on token-level sparsity and rollout-level heterogeneity. The overlap concerns the organization of these two observations and the histogram presentation in Figure 2. Our analysis uses a different visual-dependency measure, defined in Eq. (1), and the plotted distributions were computed from our own model outputs under the experimental settings described below. VisCA’s method and its associated equations were not adapted from VPPO. We acknowledge that this influence on the diagnostic presentation was not properly attributed and apologize for the missing citation. We will cite VPPO [1] when introducing Figure 2 and the analysis at both levels, explicitly acknowledging that the visualization format was adapted from VPPO to characterize our proposed signal.

  • Model, data, and generation protocol. Figure 2 was computed using the initial Qwen3.5-4B model on the 6,241 training examples from Vision-OPD. For each example, we generated one response through greedy decoding using the full red-box annotated image. The same model then evaluated each generated token under the full image and the privileged regional crop, keeping the response prefix fixed. We computed VDt\mathrm{VD}_t using the rectified log-probability difference in Eq. (1). Figure 2(a) pools these scores across all generated tokens, while Figure 2(b) shows their mean within each response. After excluding two invalid records, the analysis comprises 6,239 responses and 441,353 tokens.

2. Base Configuration and Visual-Evidence Ablation (W2 & Q2)

  • Base objective and checkpoint. Base is the configuration-matched GRPO reference for the component ablation, trained using rule-based verifiable rewards. It shares the same red-box-annotated training images, batch size, rollout count, learning rate, and training duration with VisCA. We evaluate its final checkpoint after one epoch of training.

  • From Base to +Visual Evidence. Raw token-level visual evidence is the only additional signal, and its introduction also activates the dual-path objective. This objective independently normalizes the losses over modulated and unmodulated tokens, with η=1\eta = 1. Sign-consistency verification and rollout-level calibration are not applied in this variant. The dual-path design addresses the sparsity of visual modulation, motivated by the token-level visual dependency distribution in Figure 2. Normalizing over all tokens together can dilute the contribution of the relatively few modulated tokens among the more numerous unmodulated tokens. Separate normalization preserves their contribution while retaining the original verifier-derived advantages for unmodulated tokens. Thus, raw visual evidence is the only newly introduced signal, but the transition also includes the optimization mechanism used to incorporate it.

3. EMA Teacher Configuration and Rationale (W3 & Q3)

  • EMA configuration. We use an EMA decay of 0.95, corresponding to an update rate of ρ=0.05\rho = 0.05:

    \theta_{\mathrm{T}} \leftarrow (1 - \rho)\theta_{\mathrm{T}} + \rho\theta_{\mathrm{S}}. \end{xalignat*}$$ The teacher is initialized from the same base checkpoint as the student, remains fixed during actor optimization, and is updated once after all policy mini-batch updates are completed.
  • Design rationale and control experiment. This design is inspired by Vision-OPD [2], which reports instability with the current policy as teacher and adopts EMA regularization in its self-distillation setting. For VisCA, EMA provides a temporally smoothed reference for the likelihood contrasts underlying visual credit, helping stabilize these signals while tracking policy improvement. We compare EMA with a Current Policy variant that directly synchronizes the teacher with the student, keeping all other settings unchanged. As shown below, Current Policy achieves an Average6\mathrm{Average}_6 of 71.96, compared with 77.73 for EMA. This comparison supports the benefit of EMA regularization for training.

MethodV*BenchZoomBenchHR-Bench 4KHR-Bench 8KMME-RW ENMME-RW CNAvg
Current Policy88.4857.2876.6274.2568.3266.8171.96
EMA92.1560.4787.2582.6272.6971.1877.73

Thank you again for your careful review and constructive feedback. We hope our responses address your concerns and clarify the points you raised. We will incorporate the relevant clarifications and additional analyses into the revised manuscript. If any questions remain, we would be happy to provide further details.

[1]Huang, S., et al. (2026). Spotlight on Token Perception for Multimodal Reinforcement Learning. ICLR.

[2]Yuan, Q., et al. (2026). Vision-OPD: Learning to See Fine-Grained Details for Multimodal LLMs via On-Policy Self-Distillation. NeurIPS.

Thanks for clarifying the attribution, the Base configuration, and the EMA settings. The overlap with prior work should have been acknowledged in the original submission. The new ablation also raises a concern about the method's sensitivity to the teacher choice: with the current policy, it no longer improves over GRPO. Taking the manuscript as a whole, I'll keep my current score for now and continue to follow the discussion and the other reviewers' assessments

Thank you for your further comments. We fully agree that our previous explanation had omissions and will rectify this in the revised manuscript. Specifically, we will add a formal citation to VPPO [1] in Section 2 (Related Work), Section 3.1 (Preliminary), and the caption of Figure 2, to state that our two-level diagnostic analysis and its presentation are inspired by VPPO, and to clarify the connections and distinctions between VisCA and VPPO regarding diagnostic analysis. This will ensure a clear delineation of contributions between the two works and avoid any phrasing that could be construed as ambiguous attribution.

Regarding your concern about the method's sensitivity to the choice of policy teacher, we would like to clarify the underlying mechanism as follows:

  • First, using an EMA teacher is not unique to VisCA, but rather a common regularization practice among OPSD-family methods for this task. For instance, both Vision-OPD [2] and OPD-V [3] adopt this setup. In fact, the policy ablation in Vision-OPD shows that replacing the EMA teacher with an unregularized current policy causes complete training collapse, with the average score across six benchmarks dropping from 79.68% to 0.59%.

  • Second, VisCA relies on a smooth EMA teacher to maintain a consistent token-verification signal over GRPO, while remaining anchored in verifiable rewards. Specifically, VisCA uses the discrepancy between the teacher's predictions across different visual views to adjust advantage values derived from verifiable rewards on selected tokens. When the teacher closely tracks the unregularized current policy, this inter-view discrepancy approaches zero, yet it is precisely these near-zero discrepancies that determine whether a token passes verification. Consequently, a single policy update can flip the sign of the discrepancy and cause the verified token set to jitter from step to step, preventing the advantage adjustment from providing a steady gain over standard GRPO. EMA smooths the teacher's updates and suppresses this jitter, keeping the verified token set stable.

  • Third, this explains why replacing EMA with the current policy degrades gracefully rather than failing. Unlike Vision-OPD, which uses teacher predictions directly as distillation targets, VisCA keeps verifiable rewards as the source of supervision and uses the teacher only to rescale advantages at selected tokens. When the verification signal becomes unstable, the rescaling approaches a neutral value, so the objective remains well defined. Our results match this picture. Without EMA, VisCA stays close to the GRPO baseline with the same red-box input (72.54) and shows no Vision-OPD-style collapse, while still outperforming the vanilla model (70.69).

We hope these additional clarifications address your concerns. We welcome any further feedback and would be grateful if you would reconsider your assessment of the paper.

[1] Huang, S., et al. (2026). Spotlight on Token Perception for Multimodal Reinforcement Learning. ICLR.

[2] Yuan, Q., et al. (2026). Vision-OPD: Learning to See Fine-Grained Details for Multimodal LLMs via On-Policy Self-Distillation. NeurIPS.

[3] Aniri et al. (2026). OPD-V: Visual On-Policy Self-Distillation with Modality Balance. arXiv:2608.05131.

Reviewer_f24ROpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces VisCA, a self-distilled RLVR method that uses differences in token likelihoods across full-image, evidence-crop, and degraded-crop conditions to refine token-level advantages. A counterfactual consistency check filters the visual signal, and rollout-level calibration adjusts its strength. These signals modify the outcome-derived advantage while preserving its sign. Experiments cover two model sizes, six fine-grained visual benchmarks, and three additional multimodal benchmarks.

Critical strengths
  1. The main results are reasonably consistent across model sizes. In Table 2, VisCA improves the six-benchmark average over GRPO and the evaluated self-distillation methods at both scales. The additional evaluations also provide useful evidence that the gains are not accompanied by a broad decline on the tested multimodal tasks.
  2. The paper examines several relevant design choices, including counterfactual construction, token-level modulation, and training overhead. These experiments make the performance–cost trade-off easier to assess.
Critical weaknesses
  1. The ablations do not yet separate the quality of the visual signal from the effect of changing token weights. In Table 3, removing dual-path optimization reduces the average from 76.27 to 73.26, suggesting that independent normalization is a major contributor. However, there is no control that preserves the selected-token count and path weights while replacing the visual signal with a shuffled or random signal. It is therefore difficult to tell how much of the gain comes from identifying the right tokens rather than increasing the effective weight of a sparse subset. The “Base” result in Table 3 is also 72.54, which matches neither the vanilla nor the GRPO result in Table 2.
  2. The evidence for reliable visual attribution remains indirect. Equation (4) checks agreement between the signs of two likelihood differences. Degrading a crop can also change confidence or introduce distribution shifts, so sign agreement alone does not establish that a token depends on the answer-relevant detail. The semantic analysis in Section 4.4 is helpful, but visual words and answer-choice tokens are not necessarily grounded in the correct evidence. Figure 4 also excludes the “Other” category. The results support the filter as a useful heuristic more clearly than they establish the stronger interpretation of verified visual attribution.
Questions for authors
  1. What objective and checkpoint does “Base” in Table 3 refer to, and why does it differ from GRPO in Table 2? With the same dual-path weights and selected-token count, how does VisCA compare with shuffled visual signals?
  2. For Figure 4, how are signed scores aggregated and normalized across categories? What proportion is excluded as “Other”? Is there a controlled or manually annotated evaluation of whether retained tokens actually depend on the correct visual evidence?
  3. Were the hyperparameters in Appendix B and the degradation ratio in Appendix C selected on a separate validation set? Please clarify whether the six evaluation benchmarks influenced these choices.
1 comment

Thank you for your constructive comments on the experimental design and the reliability of visual-credit attribution. Below, we clarify the Base and GRPO settings, provide controlled comparisons to examine whether retained tokens rely on the correct visual evidence, and detail the statistical analysis and hyperparameter selection protocols.

1. Base Configuration and the Shuffled-Signal Control (W1 & Q1)

  • Base configuration. Base in Table 3 serves as the data-matched GRPO reference for the component ablation, rather than the clean-image GRPO baseline in Table 2. Both use the same verifier-only GRPO objective, hyperparameters, and evaluation protocol at the final checkpoint after one epoch, differing only in the training input to the policy. Specifically, Table 2 GRPO uses clean images, whereas Base uses the same red-box annotated training images as VisCA. Because the red box is absent during evaluation and standard GRPO lacks a mechanism to transfer this privileged cue to clean-image reasoning, directly exposing the policy to annotated images can induce a non-transferable shortcut. This mismatch may explain the performance difference between Base in Table 3 and GRPO in Table 2.

  • Shuffled visual signals. As suggested by the reviewer, we keep the same dual-path weights and selected-token count and only permute the finalized gated evidence across valid token positions of the same response. As shown in the table, Token Shuffle lowers the average over six benchmarks from 77.73 to 67.84, a drop of 9.89 points. Shuffling the visual credit across tokens therefore substantially degrades performance, confirming that VisCA benefits from assigning the visual evidence to the corresponding tokens.

MethodV*BenchZoomBenchHR-Bench 4KHR-Bench 8KMME-RW ENMME-RW CNAvg
Token Shuffle78.0153.2573.6271.2566.1264.7867.84
Ours92.1560.4787.2582.6272.6971.1877.73

2. Figure 4 Protocol and Visual-Dependency Evidence (W2 & Q2)

  • Score aggregation and normalization. Inspired by the semantic analysis in VAD [1], we select up to five candidate tokens with the largest absolute scores for each of the three components etvise_t^{vis}, e~tvis\tilde{e}_t^{vis}, and etvis−e~tvise_t^{vis} - \tilde{e}_t^{vis}. Each selected token occurrence contributes one count to its semantic category, with unmatched tokens assigned to Other. We aggregate these counts across all rollouts, exclude Other, and normalize the remaining five category counts so that their shares sum to 100%. Before exclusion, Other accounts for 5.7%, 4.9%, and 9.6% of all selected token occurrences in the etvise_t^{vis}, e~tvis\tilde{e}_t^{vis}, and etvis−e~tvise_t^{vis} - \tilde{e}_t^{vis} components, respectively.

  • Visual-dependency evaluation of retained tokens. We examine this question through a controlled test of visual-credit alignment and qualitative token-level analysis. The Token Shuffle control in Q1 keeps the dual-path weights and selected-token count fixed while permuting the finalized gated evidence across valid positions within each response. This preserves the retained signals but disrupts their correspondence with the original tokens, testing whether their benefit depends on that correspondence. Token Shuffle yields an Average6 of 67.84 compared with 77.73 for VisCA. Figure 6 of the manuscript complements this test by illustrating how credit aligns with the visual cues relevant to the counting query, including “dark”, “two”, and “birds”, alongside spatial cues such as “near” and “left”. Together, these analyses provide complementary support for the effectiveness of assigning retained visual credit to task-relevant token positions.

3. Parameter Selection Protocol (Q3)

No separate validation split was used for the ablations in Appendices B and C. Following common practice in controlled ablation studies, we trained Qwen3.5-4B for one epoch per candidate setting with the remaining configuration fixed, evaluated each final checkpoint on the six fine-grained benchmarks, and selected the setting with the highest Average6\mathrm{Average}_6. At the same time, Figure 5 and Table 4 show that performance varies smoothly across a broad range of parameter values and counterfactual constructions rather than relying on a fragile optimum, and the resulting configuration also maintains strong average performance on the three held-out benchmarks MMVP, CV-Bench, and MMStar, which were never used in parameter selection.

Thank you again for your careful review. We hope our responses clarify the points you raised. We will incorporate the relevant clarifications and analyses into the revised manuscript. If any questions remain, we would be happy to provide further details.

[1] Zhang, K., et al. (2026). VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation. arXiv:2607.28590.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Reviews and Responses

We thank the PCs, SACs, and ACs for coordinating the review process, and the reviewers for their careful reading and constructive feedback.

  1. Prior work and diagnostic analyses (XZLw, f24R). We will cite VPPO in Sections 2 and 3.1 and in the Figure 2 caption, crediting its influence on the token- and rollout-level analysis and histogram presentation. We clarified the model, data, and decoding protocol for Figure 2, as well as score aggregation, normalization, and the treatment of “Other” in Figure 4.

  2. GRPO baseline and attribution controls (f24R, XZLw, cUVH). Table 3’s Base is the final one-epoch GRPO checkpoint trained on the same red-box images as VisCA. Table 2’s GRPO uses clean training images. Adding raw visual evidence also enables the dual-path objective. We described the within-response shuffle control, discussed uniform-grid attribution and gradient-based saliency, and used Figure 6 to illustrate credit at individual token positions.

  3. Verification and teacher choice (XZLw, cUVH). We explained why the gate checks for sign agreement between the two visual contrasts and discussed how often tokens pass verification during training. We also clarified the EMA update schedule and addressed the concern raised by the current-policy comparison. With the EMA teacher and rollout calibration held fixed, adding verification raises the six-benchmark average from 76.70 to 77.73.

  4. Visual distractors and privileged regions (cUVH). We discussed the ZoomBench test with distractors placed outside the annotated evidence region and the comparison of counterfactual views in Table 4. The bounding box defines the privileged view used during training. At inference, VisCA receives only the full image.

  5. Hyperparameter selection and generalization (f24R, cUVH). We clarified how the six fine-grained benchmarks informed hyperparameter and degradation-ratio selection. On MMVP, CV-Bench, and MMStar, which were held out from selection, VisCA improves the average over the pretrained checkpoint from 78.99 to 80.69 at 4B and from 82.44 to 82.98 at 9B.

We will incorporate these clarifications into the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.