acceptodds
Desk rejected at ICLR 2027

VC-OPD: Learning to See What Matters via Visual Cue On-Policy Self-Distillation

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing visual On-Policy Distillation (OPD) methods typically use cropped or zoomed-in image regions as teacher-side privileged information to improve the fine-grained visual perception of small Multimodal Large Language Models (MLLMs). However, such local pixel-level information is insufficient for complex visual reasoning that requires understanding global structures, spatial relationships, and high-level semantics. To address this limitation, we propose Visual Cue On-Policy Self-Distillation (VC-OPD), which extends teacher-side privileged information from local image regions to question-conditioned visual cues. Specifically, we leverage large advanced MLLMs to extract the key visual evidence required to solve a given question, including key visual details, structural relationships, and high-level semantics. A multi-stage quality filtering mechanism is further introduced to retain reliable visual cues as privileged information. During training, these cues are provided only to the teacher model, enabling it to deliver more informative token-level supervision along student-generated trajectories. The student is thereby encouraged to internalize the ability to identify and reason over critical visual evidence directly from the original image, without requiring any additional information at inference time. Since large advanced MLLMs are used only for visual cue generation, VC-OPD is applicable to both open-weight and proprietary black-box models. Experiments on STEM and general visual reasoning benchmarks show that VC-OPD consistently improves visual reasoning performance across different model scales and outperforms a range of reinforcement learning and on-policy distillation baselines. The 4B model trained with VC-OPD surpasses many baselines with 8B or more parameters, demonstrating the effectiveness of question-conditioned visual cues as privileged information for capability transfer. The code will be released after review.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Large Language Models, Visual Reasoning, On-Policy Distillation, Privileged Information, Question-Conditioned Visual Cues

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8P4YOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This submission proposes VC-OPD, an on-policy self-distillation (OPSD) framework for small MLLMs. The teacher receives a training-only, question-conditioned textual visual cue as privileged information, while the student sees only the original image and question. A Visual Cue Agent uses a large MLLM (Gemini-3.5-Flash) to produce the cue, filtered by three gates: Difficulty (Eq. 1), Answer (Eq. 3) and Reliability (Eq. 4). These build a Visual Privilege Library (Eq. 5). A shared backbone is first GRPO-warmed into a frozen teacher (Eq. 6). Distillation then minimizes token-level JSD between the student and the cue-conditioned teacher on student rollouts (Eq. 8). Experiments fine-tune Qwen3-VL-Instruct at 2B, 4B and 8B on ViRL39K and evaluate on STEM and general visual-reasoning benchmarks (Table 1), with ablations over privileged-information type (Table 2) and GRPO warm-up (Table 3).

Critical strengths
  1. The Table 2 dissociation matters. By holding the student, teacher warm-up and configuration fixed and varying only the privileged information (§4.3, Table 2), the submission shows that maximizing teacher capability does not maximize transfer: the Answer/Reasoning teacher scores 95-98 against 72-82 for the Visual Cue teacher, yet Visual Cue wins downstream on average at every scale. This is the result that goes beyond showing that cues help, and it deserves the most weight in the assessment.
  2. Consistency across scales and a clean warm-up ablation. Table 1 and Table 3 report gains at 2B, 4B and 8B on two benchmark groups, and Table 3 isolates the GRPO warm-up contribution (for example +6.48/+3.91/+1.64 STEM at 2B/4B/8B against base+OPD). The pattern is monotone enough to be believable and is not a single lucky cell.
Critical weaknesses
  1. I think the core conceptual claim overlaps substantially with a public work that is not cited, and the closest contemporaneous baseline is cited but not compared empirically. §1 (contributions) and §2 present question-conditioned visual cues as teacher-side privileged information that is task-relevant and recoverable as the novelty. However, the public preprint ViCuR (Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation, arXiv 2606.05718, submitted 4 June 2026) already proposes replacing answer-side privilege with query-related visual cues on recoverability grounds, and it does not appear in the reference list (§References). ViCuR is earlier than VAD, which the submission does cite. Separately, VAD (arXiv 2607.28590) is cited in §2 but is not compared empirically, even though VAD is the closest shared-framework method and reports a matched-budget result at 4B on its own six-benchmark suite. Both ViCuR (4 June 2026) and VAD (30 July 2026) predate the ICLR 2027 deadline of 25 September 2026, so a comparison with both was feasible. As written, the delta over ViCuR (an LLM-generated textual cue, three-gate filtering and API-only cue generation) is stated but not yet demonstrated. It would help to add an explicit differentiation section and a head-to-head comparison under a common evaluation suite.

  2. Two controls would strengthen the argument. The recoverable property is not measured on the student, and run-to-run variability is not reported. The Reliability Gate (Eq. 4) shows only that the teacher answers correctly ≥7/8 with the cue; it does not test whether the student can recover the evidence from the original image, which is the mechanism the paper claims (§3.2-3.3, Eq. 8 discussion). Tables 1-3 report single point estimates with no seeds, no error bars and no significance tests. §4.1 does not say whether the baselines (OPSD, Vision-OPD, VGPO, ReMax, GRPO) were reimplemented or run from official code, nor their tuning budget. Some baseline cells fall below the untuned base (for example Vision-OPD-2B STEM average 40.41 against Qwen3-VL-2B base 43.46, Table 1), so it is difficult to tell whether the comparison is compute fair. It would help to report seeds and variance, an explicit baseline protocol and a student-side recovery probe.

  3. The three gates are central to the method, but the paper does not appear to ablate them. The Difficulty, Answer and Reliability gates (Eqs. 1, 3 and 4; thresholds τd−=1, τd+=7 and τrel=7 in Appendix D) decide which samples and cues enter the Visual Privilege Library, so they change both the training distribution and the amount of supervision. Tables 2 and 3 ablate the privileged-information type and the GRPO warm-up, yet I could not find an experiment that removes or varies a gate, reports how many samples each gate filters, or tests whether the gains survive a simpler filter. The contribution of the gating pipeline, as opposed to the cue content or the reduced training set, is therefore difficult to identify. It would help to report per-gate retention rates and to ablate each gate and its thresholds.

  4. The paper states that the final answer is excluded from the cue to prevent leakage (§3.2), and Fig. 3(b) shows a cue from the tuned model that ends with "The lowest accuracy value reported is 1." This appears to be the final answer. I could not find an automated check for leakage at scale, and the Answer Gate verifies only that the large MLLM's answer is correct, not that the cue omits it. If cues occasionally contain the answer, the teacher's supervision can encode the same shortcut the paper aims to remove, which also complicates the Table 2 comparison with Answer privilege. It would help to run an automated leakage detector over the full Visual Privilege Library, report the rate, and either filter leaking cues or show that the downstream result is unchanged.

Questions for authors
  1. The public VAD release (arXiv 2607.28590) includes an internal run identifier beginning with VC-OPD whose optimization configuration (batch 96, PPO mini-batch 96, 8 rollouts per prompt, lr 2×10⁻⁶, symmetric JSD α=0.5, top-100 student support) matches Table 4 of this submission. The public VAD repository describes target_mode=vc and vc_target_variant=budgeted_asvc as internal identifiers. Please clarify the relationship between this submission and the public VAD (arXiv 2607.28590) and V-Zero (arXiv 2606.25319) work. Is it a shared codebase, a shared data pipeline, or independent work? This materially affects how I weigh the contribution.
  2. ViCuR (arXiv 2606.05718) proposes query-related visual cues as recoverable privilege for multimodal OPD, and it is not cited. What exactly does VC-OPD add over ViCuR? Is it the LLM-generated textual form of the cue, the three-gate filtering, black-box cue generation, or the scale of the empirical study?
  3. VAD (arXiv 2607.28590) is cited in §2 but is not compared empirically, and V-Zero (arXiv 2606.25319), which VAD itself compares against, does not appear in the submission. Why is there no empirical comparison, and would VC-OPD retain its advantage under a common benchmark suite and a matched budget?
  4. See Weakness 4: Could you report the leakage rate and the automated check used at scale? 5.Add seeds and variance, a student-side recoverability measurement, and a train/test contamination analysis between ViRL39K and the evaluation benchmarks.
4 comments
AuthorsOpenReview ↗

Response to Reviewer 8P4Y (Q1)

We thank the reviewer for highlighting our central finding—that transferability of visual cues matters more than teacher improvement alone—and for recognizing consistent gains across model sizes.

Q1. Comparison with newly suggested baselines: ViCuR (June 4, 2026), V-Zero (June 24, 2026), and VAD (July 30, 2026)

Relationship to concurrent work and novelty

We thank the reviewer for pointing out ViCuR, V-Zero, and VAD. These works appeared 1.9 to 3.7 months before the ICLR 2027 deadline and were concurrent with our independent research. We acknowledge that our initial submission lacked sufficient discussion and comparison. We therefore reproduced all three baselines. For ViCuR, we used its released visual cue data and trained with the same configuration as VC-OPD. For V-Zero and VAD, we used their official implementations and original training datasets, changing only the base model. All methods were evaluated under a unified protocol (Tables R5–R6).

We acknowledge that ViCuR has explored question-relevant visual cues as recoverable privileged information. VC-OPD differs in its general cue-generation framework, quality gating, answer leakage control, student architecture compatibility, and pre-distillation teacher enhancement, as detailed below.

1. Cue generation. ViCuR provides limited details on cue generation. VC-OPD uses a Visual-Cue Agent with a unified API for diverse VQA tasks, supported by automated quality checks and gating. Table R4 shows higher teacher scores with VC-OPD cues than with ViCuR cues across 2B, 4B, and 8B models.

2. Answer leakage. According to ViCuR's public issue, cue generation uses full solution steps and answers. In contrast, VC-OPD generates cues using only the image and question, reducing leakage risk by design. Table R4 reports leakage rates of 20.60% for ViCuR and 4.86% for VC-OPD, a 15.74 percentage-point reduction.

3. Student architecture compatibility. ViCuR introduces a SinkTrack-based visual cue recovery module that modifies the student architecture and remains active during inference. This adds inference overhead and complicates adaptation and deployment across different base models. In contrast, VC-OPD transfers the teacher's visual knowledge through on-policy distillation without architectural changes. At inference, the student requires only the original image and question, with no additional modules or associated inference overhead.

4. Teacher enhancement. VC-OPD applies GRPO warm-up before on-policy distillation to improve the teacher's visual STEM reasoning, whereas ViCuR has no corresponding step.

Table R4. Visual-cue quality. Trainset Score is the teacher's training-set accuracy when given visual privileged information.

MetricViCuRVC-OPD
Trainset Score 2B66.6072.68
Trainset Score 4B76.8081.05
Trainset Score 8B78.4782.32
Answer Leaks(Judge model)20.60%4.86%
Average Character Count519.15503.94

Table R5. Results on STEM visual-reasoning benchmarks.

MethodMathVerse$_m^v$MathVerse$_m^t$DynaMathWeMathMathVista$_m$MathVision$_m$Average
VZero-2B35.2851.2751.8630.2961.527.9643.03
VAD-2B36.295053.8530.6764.427.9643.86
ViCuR-2B36.4247.5952.3830.2962.227.9642.81
VC-OPD-2B45.0558.1254.1338.1967.0026.3248.14
VZero-4B30.3348.8663.4546.4872.538.4950.02
VAD-4B31.646.9565.3347.4371.941.1250.72
ViCuR-4B51.2766.6265.3748.8675.238.1657.58
VC-OPD-4B59.3972.0865.8951.6276.8044.0861.64
VZero-8B39.3467.5167.1148.6774.743.0956.74
VAD-8B37.0654.1967.1353.5274.947.3755.69
ViCuR-8B56.7371.4568.1854.176.546.3862.22
VC-OPD-8B61.5573.1068.8255.3376.8045.0763.44

Table R6. Results on general visual-understanding benchmarks.

MethodMMMU$_{pro}^v$MMMU$_{pro}^{std}$HallBenAverage
VZero-2B28.3234.2866.9843.19
VAD-2B28.8436.6567.344.26
ViCuR-2B27.8634.2868.3543.5
VC-OPD-2B31.7337.6369.9346.43
VZero-4B32.2535.5570.9846.26
VAD-4B31.2736.4770.3546.03
ViCuR-4B42.2547.1771.553.64
VC-OPD-4B45.4949.7173.1956.13
VZero-8B44.6840.2372.1352.35
VAD-8B37.2840.6473.550.47
ViCuR-8B50.2353.6473.2959.05
VC-OPD-8B52.4953.7674.8760.37
AuthorsOpenReview ↗

Response to Reviewer 8P4Y (Q1 and Q2)

Relationship to VAD code and experimental settings

VC-OPD was developed independently using the publicly available Vision-OPD codebase, without using VAD's or V-Zero's code or data-processing pipelines. Similar training configurations may arise from the shared Vision-OPD foundation. We cannot explain the VC-OPD identifier used in VAD's public experiments, as it was assigned by its authors. We will clarify our code provenance and methodological differences in the revision.

Empirical comparison with V-Zero, VAD, and ViCuR

We reproduced V-Zero, VAD, and ViCuR and evaluated them alongside VC-OPD on the same benchmark suite. Results appear in Table R5 (STEM visual reasoning) and Table R6 (general visual understanding).

Vision-OPD, V-Zero, and VAD all use local visual crops to support distillation, but in different ways:

Vision-OPD: The student sees the original image; the teacher uses enlarged crops of key regions.

V-Zero: It compares teacher probabilities under relevant- and irrelevant-region views to estimate students' reliance on visual evidence and adjust distillation weights.

VAD: It uses changes in the teacher distribution when key regions are present versus removed to estimate evidence-driven corrections and reconstruct distillation targets.

By contrast, VC-OPD supplies the teacher with question-relevant cues rather than only enlarged local regions, making it suitable for tasks requiring both detailed perception and complex visual reasoning.

Additional experiments support VC-OPD's advantages. Tables R5–R6 show that VC-OPD outperforms concurrent methods V-Zero, VAD, and ViCuR on both STEM and general-vision tasks at 2B, 4B, and 8B. These consistent results indicate improvements in both fine-grained perception and complex reasoning and support the method's effectiveness and generalization.

Table R3 (response to Reviewer uF2w) further compares VC-OPD and Vision-OPD on fine-grained perception and global spatial reasoning. Vision-OPD is stronger on some local-detail tasks but weaker on spatial tasks. VC-OPD maintains strong local perception while improving SpatialEval and PuzzleVQA performance, yielding higher overall averages at all three model scales and better adaptation across tasks.

In summary, the new experiments show better overall performance for VC-OPD across model sizes, supporting our cue-generation and distillation design. The revision will expand the related-work discussion, implementation details, and full comparisons.

Q2. Baseline reproduction details and Qwen3-VL-2B performance degradation

Thank you for the suggestion. We clarify our baseline reproduction protocol and the performance drop observed for the 2B model.

Baseline reproduction and fairness

All baselines share the same evaluation protocol. Their training settings are:

Vision-OPD and VGPO: We used the official open-source code and original hyperparameters, replacing only the base model with the corresponding Qwen3-VL size.

OPSD: We matched VC-OPD's training hyperparameters for fairness.

GRPO and ReMax: We implemented both from official verl examples and matched VC-OPD's base model, batch size, learning rate, and rollout settings.

Using official implementations or aligned training settings helps make the reproduced baselines reliable and comparable.

Why Qwen3-VL-2B performance drops

Some baselines, including Vision-OPD, degrade on Qwen3-VL-2B, whereas VC-OPD improves performance. We view this contrast as an advantage of VC-OPD.

Qwen3-VL-2B has limited initial visual-reasoning ability. Under direct OPSD, the teacher may not use visual privileged information effectively, leading to unchanged or lower performance.

In the original paper, Table 1 shows that Vision-OPD-2B lowers the STEM average from 43.46 to 40.41 (−3.05 points). The ablation in Table 3 similarly shows a drop from 43.46 to 41.66 (−1.80 points) with direct OPD.

The GRPO-warmed model achieves a STEM average of 44.99. Distillation from this teacher with visual cues yields a student score of 48.14, compared with 43.46 for the base student.

These results suggest that direct OPD can hurt weaker 2B models, while GRPO warm-up makes subsequent distillation more effective.

AuthorsOpenReview ↗

Response to Reviewer 8P4Y (Q3 and Q4)

Q3. Ablation of the gating mechanism

Thank you for the suggestion. We added per-stage sample pass rates and an ablation of the complete gating mechanism.

(1) Sample pass rates at each gate. See Table R7.

The Difficulty Gate passes 30.53%, 24.02%, and 24.06% of samples for 2B, 4B, and 8B, respectively. It filters overly easy or hard cases, focusing training near each model's capability boundary.

The Answer Gate passes 91.93%, 90.04%, and 88.85%, suggesting that Gemini-3.5-Flash solves most candidate questions correctly and provides an initial check on cue correctness.

The Reliability Gate passes only 58.34%, 70.89%, and 73.91%, lower than the Answer Gate. A correct answer therefore does not ensure cues consistently help teacher reasoning, justifying additional filtering. The higher pass rates for larger models also suggest more reliable use of cues.

Together, the three gates complement one another by checking sample difficulty, answer correctness, and cue reliability, improving the Visual Privilege Library's quality.

(2) Full gating ablation. Tables R8–R9 compare models with and without gating. Gating improves the STEM averages by 5.73, 3.81, and 2.19 points at 2B, 4B, and 8B, and the general-vision averages by 2.43, 2.07, and 1.08 points, respectively. Gains are consistent across scales and task types.

These experiments support the effectiveness of multistage gating. We will include them in the revision.

Table R7. Sample pass rates for the three gates.

ModelDifficulty GateAnswer GateReliability Gate
2B    30.53%          91.93%      58.34%          
4B    24.02%          90.04%      70.89%          
8B    24.06%          88.85%      73.91%          

Table R8. Gating ablation on STEM visual-reasoning benchmarks. The number of samples without gate matches that with gate, randomly drawn from the dataset.

ModelGateMathVerse$_m^v$MathVerse$_m^t$DynaMathWeMathMathVista$_m$MathVision$_m$Average
2Bwithout gate34.3948.6051.4429.6263.1027.3042.41
2Bwith gate45.0558.1254.1338.1967.0026.3248.14
4Bwithout gate51.2767.7765.0546.0074.8042.1157.83
4Bwith gate59.3972.0865.8951.6276.8044.0861.64
8Bwithout gate57.2370.9468.2851.5276.1043.4261.25
8Bwith gate61.5573.1068.8255.3376.8045.0763.44

Table R9. Gating ablation on general visual-reasoning benchmarks.

ModelGateMMMU$_{pro}^v$MMMU$_{pro}^{std}$HallBenAverage
2Bwithout gate27.9235.4368.6644.00
2Bwith gate31.7337.6369.9346.43
4Bwithout gate42.7247.6371.8254.06
4Bwith gate45.4949.7173.1956.13
8Bwithout gate50.5852.5474.7659.29
8Bwith gate52.4953.7674.8760.37

Q4. Answer leakage in visual cues

Thank you for the careful observation.

(1) Answer leakage in Figure 3(b). The cues in Figure 3(b) were generated by the trained student, solely to examine whether it could recover key visual information from the original image. They were not teacher privileges used during training. The student may include a final answer in its description due to limited ability, but this is unrelated to the training cues.

(2) Leakage in training data. Gemini-3.5-Flash generated the training cues under explicit instructions to describe only question-relevant visual facts, excluding reasoning steps and final answers. We tested the entire Visual Privilege Library for leakage (Table R4).

An LLM judge is used to assesse whether a cue directly revealed answer-bearing information; the resulting leakage rate was 4.86%.

Masking-based robustness check. To assess whether the small fraction of potentially answer-leaking training cues affected the results, we masked answers in all flagged cues by replacing answer-bearing keywords or fields with *** while keeping the ground-truth labels unchanged, and reran VC-OPD with the masked cues. Performance was comparable to that with the original unmasked cues, with no notable difference. This indicates that the limited potential answer leakage in the original cue library is not the primary source of VC-OPD’s gains. Together with the leakage analysis in Table R4, it shows that the improvements persist even after suspected answer-revealing content is masked.

These results suggest that our cue-generation approach reduces answer leakage: most training cues do not contain the answer directly, limiting answer-shortcut learning during distillation.

AuthorsOpenReview ↗

Response to Reviewer 8P4Y (Q5)

Q5. Random-seed variance, student-side recoverability, and train–test contamination

Thank you for the suggestion. We added random-seed experiments, an analysis of student-side recoverability, and checks for train–test contamination.

1. Random-seed variance experiments

To evaluate the stability of VC-OPD, we conducted five independent runs with different random seeds (0–4) on Qwen3-VL-4B, as shown in Tables R10(a)–R10(b). The standard deviations of the aggregate scores are 0.51 and 0.15 for STEM and general visual reasoning, respectively. Most benchmarks have standard deviations below 1.00, indicating that VC-OPD is robust to random-seed variation.

Table R10(a): Random-seed standard deviation on STEM visual-reasoning benchmarks (Qwen3-VL-4B).

MethodMathVerse$_m^v$MathVerse$_m^t$DynaMathWeMathMathVista$_m$MathVision$_m$Average
VC-OPD0.640.320.191.070.632.000.51

Table R10(b): Random-seed standard deviation on general visual-reasoning benchmarks (Qwen3-VL-4B).

MethodMMMU$_{pro}^v$MMMU$_{pro}^{std}$HallBenAverage
VC-OPD0.570.400.770.15

2. Student-side recoverability

As Figures 3(b) and 5 show, after VC-OPD training, the student can recognize and use visual information it previously struggled to interpret without access to visual cues. This provides qualitative evidence of student-side recoverability.

Student-side recoverability is difficult to measure directly, and no unified reliable metric is yet available. Concurrent state-of-the-art methods, including VAD, Vision-OPD, and ViCuR, also lack an established quantitative measure. We therefore assess it indirectly through pre- versus post-training capability changes.

3. Train–test contamination analysis

We examined potential overlap between ViRL39K and the evaluation benchmarks by matching both image content and question semantics (Table R11). Among 22,301 evaluation samples, only 115 showed potential overlap (0.5157%), indicating limited train–test overlap.

Overall, the low overlap rate suggests a limited risk of train–test contamination and minimal potential impact on the reported results.

Table R11. Potential data contamination between ViRL39K and evaluation benchmarks. Both image content and question semantics are matched.

BenchmarksBenchmark SamplesSimilar SamplesProportion of Similar Samples
MathVerse_MINI3,940210.5330%
DynaMath5,010701.3972%
WeMath1,74070.4023%
MathVista_MINI1,000171.7000%
MathVision_MINI30400.0000%
MMMU_Pro_V1,73000.0000%
HallusionBench95100.0000%
VStarBench19100.0000%
HRBench4K80000.0000%
SpatialEval4,63500.0000%
PuzzleVQA2,00000.0000%
All22,3011150.5157%
Reviewer_uF2wOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes VC-OPD, a framework for improving compact multimodal language models using question-conditioned textual descriptions of relevant visual details, relationships, and semantics as teacher-only privileged information. A large MLLM generates candidate cues, which are filtered through difficulty, answer-correctness, and reliability gates; a GRPO-warmed model then serves as the privileged teacher, and the student matches its token distributions using Jensen–Shannon divergence along student-generated trajectories. The cues and large model are unnecessary at inference time. Experiments on Qwen3-VL models at 2B, 4B, and 8B scales report consistent improvements across STEM and general visual-reasoning benchmarks compared with base models.

Critical strengths
  • Thoughtful data-quality pipeline; the Difficulty, Answer, and Reliability Gates attempt to focus training on informative examples and remove incorrect or unusable synthetic cues is intuitive and makes sense.

  • The paper evaluates Qwen3-VL models at 2B, 4B, and 8B scales across STEM, mathematical reasoning, general multimodal reasoning, and hallucination-oriented benchmarks.

Critical weaknesses
  • It feels like the major weakness of the paper is lack of novelty. On-policy self-distillation (OPSD) being a framework where having its own model with privileged information become a teacher, this paper seems like an application of such framework on the vision-language domain. The difference would of course be what the privileged information is, but that alone seems an incremental contribution.

  • The central motivation is that question-conditioned textual cues provide more transferable global and relational information than pixel-level visual privileges such as crops. However, this claim is not isolated experimentally; no controlled experiment. Introducing a new form of privileged information within OPSD could constitute a contribution, but the manuscript does not isolate why this privilege is superior to existing visual privileges. The experiments show that the bundled VC-OPD pipeline works, rather than establishing the proposed principle that question-conditioned textual cues are more transferable than image-based privileged information.

  • The paper provides no analysis separating tasks requiring local details from those requiring global structures or spatial relationships which is the motivation to use textual cues.

Questions for authors

see weaknesses above.

2 comments
AuthorsOpenReview ↗

Response to Reviewer uF2w

We thank the reviewer for recognizing our data-quality filtering and extensive evaluation across model scales and visual-reasoning benchmarks.

Q1. Method novelty

Thank you for focusing on novelty. Our main contribution is not merely applying an existing approach to vision-language models. It lies in how visual privileged information is constructed and how well it generalizes across tasks, in two respects:

1. Visual Cue Agent. We propose a unified cue-generation framework compatible with open- and closed-source models. Difficulty, answer, and reliability gates control training-sample and privilege quality. The gating ablation in Table R8 (response to Reviewer 8P4Y) supports this design.

2. Visual cues as privileged information. Unlike local pixel-level privileges, question-conditioned visual cues support fine-grained perception, global spatial reasoning, and complex STEM reasoning across more tasks. Controlled comparisons with pixel privileges (Tables R1–R2) and focused local/global tests (Table R3) support this advantage.

Thus, our novelty lies not only in a new form of privileged information, but also in a reliable, general cue-generation mechanism with demonstrated effectiveness and generalization across visual-reasoning tasks.

Q2. Advantages of visual cues over visual pixels

To address this concern directly, we ran controlled experiments comparing visual pixels and visual cues as privileged information. Gemini-3.5-Flash generated bounding boxes for ViRL39K, from which we made enlarged crops for pixel-based privileges. Both approaches used the same training data, GRPO warm-up, and OPSD procedure, isolating the effect of privilege type.

Better task transfer. In Tables R1–R2, visual cues outperform visual pixels on average by 1.38, 2.29, and 1.03 percentage points on STEM tasks, and by 0.21, 0.50, and 1.48 points on general-vision tasks, for 2B, 4B, and 8B models, respectively. These consistent gains suggest better transfer across tasks and scales.

Stronger teachers. On the training set, teachers with visual cues outperform those with visual pixels by 15.18, 11.48, and 11.47 percentage points at 2B, 4B, and 8B, respectively. Both privilege types are recoverable, but cues offer more useful support for teacher reasoning and may provide better student supervision under OPSD.

Broader visual-task coverage. We also evaluated fine-grained recognition and global spatial understanding (Table R3 in Q3). Vision-OPD has small advantages on some local-detail tasks, whereas VC-OPD retains competitive fine-grained perception and substantially improves global spatial understanding. Its mean scores exceed Vision-OPD by 9.94, 9.32, and 7.68 percentage points at 2B, 4B, and 8B.

Under the same OPSD framework, these controlled results show that visual cues are effective privileges. Compared with local pixels, they better strengthen the teacher and produce broader, more transferable visual improvements, supporting our design rationale.

Table R1. STEM benchmark results. Models using visual pixels as privileged information are trained on ViRL39K.

Privileged InformationTrainset ScoreMathVerse$_m^v$MathVerse$_m^t$DynaMathWeMathMathVista$_m$MathVision$_m$STEM Average
Visual Pixels-2B57.5043.5358.553.6934.2965.225.3346.76
Visual Cues-2B72.6845.0558.1254.1338.1967.0026.3248.14
Visual Pixels-4B69.5756.2270.6963.7347.2476.142.1159.35
Visual Cues-4B81.0559.3972.0865.8951.6276.8044.0861.64
Visual Pixels-8B70.8562.5674.1166.6550.7676.344.0862.41
Visual Cues-8B82.3261.5573.1068.8255.3376.8045.0763.44

Table R2. General visual benchmark results. Models using visual pixels as privileged information are trained on ViRL39K.

Privileged InformationMMMU$_{pro}^v$MMMU$_{pro}^{std}$HallBenGeneral Average
Visual Pixels-2B30.1238.6169.9346.22
Visual Cues-2B31.7337.6369.9346.43
Visual Pixels-4B45.3848.9672.5655.63
Visual Cues-4B45.4949.7173.1956.13
Visual Pixels-8B49.6552.8974.1358.89
Visual Cues-8B52.4953.7674.8760.37
AuthorsOpenReview ↗

Response to Reviewer uF2w

Q3. Fine-grained perception versus global spatial understanding

Thank you for this constructive suggestion. We added comparisons on both types of visual tasks.

We compared VC-OPD (visual-cue privileges) with Vision-OPD (local-pixel privileges) on fine-grained visual perception benchmarks VStarBench and HRBench4K, and global visual reasoning benchmarks SpatialEval (spatial understanding) and PuzzleVQA (abstract visual-pattern reasoning), as shown in Table R3.

Relative to the base model, VC-OPD improves both fine-grained perception and global spatial understanding across model sizes. Vision-OPD's gains focus on local details, while its spatial scores decline. Although their fine-grained results are comparable, VC-OPD is consistently stronger on spatial tasks. Its overall mean exceeds Vision-OPD by 9.94, 9.32, and 7.68 percentage points at 2B, 4B, and 8B.

These results suggest that enlarged local crops mainly benefit fine-grained recognition, whereas visual cues preserve local semantic details while conveying global structure and spatial relations. This supports question-conditioned textual cues as a more broadly useful form of privileged information.

Table R3. Fine-grained perception and global spatial understanding results.

ModelVStarBenchHRBench4KSpatialEvalPuzzleVQAAverage
2B64.4059.7551.245.2045.15
Vision-OPD-2B64.4062.3843.064.9543.70
VC-OPD-2B66.4959.5060.8027.7553.64
4B67.0262.1272.7933.0058.73
Vision-OPD-4B75.3966.2563.0030.7058.84
VC-OPD-4B71.2065.6274.4661.3568.16
8B71.7365.2575.1252.6066.18
Vision-OPD-8B76.9669.5063.9546.0064.10
VC-OPD-8B76.9670.3876.3363.4571.78
Reviewer_Y8LFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes VC-OPD, an on-policy distillation method for small MLLMs. The teacher is a GRPO-warmed, frozen copy of the base model. It receives a question-conditioned textual “visual cue” generated by Gemini-3.5-Flash, while the student sees only the image and question. Cues are filtered by three gates (student difficulty, Gemini answer correctness, teacher reliability), and the student is trained with token-level JSD on its own rollouts. On Qwen3-VL 2B/4B/8B, the paper reports consistent gains over the base models and over RL/OPD baselines on STEM and general benchmarks. An ablation suggests that a stronger privileged teacher (answer/reasoning) does not necessarily distill better than a cue-conditioned one.

Critical strengths
  • The core observation is interesting and, as far as I know, not well documented for MLLMs. In Table 2, answer- and reasoning-conditioned teachers reach >95% on the training set but distill worse than the cue-conditioned teacher at 72–82%. The “recoverability” framing, i.e. privileged information should be something the student could in principle extract from its own input, is a useful way to think about designing privileged context in OPD, and is somewhat more general than the specific method.
  • Using a textual cue as the interface to the large model is practical. It avoids needing logits from the large model, so a proprietary API can serve as the knowledge source. The gating pipeline is simple and easy to reproduce, and the hyperparameters are reported in reasonable detail (Appendix D).
  • The experimental scope is decent for a method paper. Three model scales, nine benchmarks, five post-training baselines, plus ablations on teacher warm-up and on the form of privileged information. The numbers in the tables are internally consistent; I recomputed the averages and the deltas quoted in the text and they match.
Critical weaknesses
  • The reviewer questions the reproduce process. VGPO has significant performance improvement on both reasoning and general benchmarks on Qwen2.5-VL series. It is strange that the reported counterpart VGPO results on Qwen3-VL improve slightly or even worse with the same training dataset/configuration and a stronger base model.
  • There is a huge gap between the InternVL3.5 technical report and the corresponding reported figures in this paper on MathVerse-V and Wemath, both on 8B and 38B models. The author should point out if there is a different setting while evaluation.
  • To make fair comparison with Vision-OPD, the reviewer would suggest doing experiment on the Qwem3.5 series model.
Questions for authors

See weaknesses

2 comments
AuthorsOpenReview ↗

Response to Reviewer Y8LF

Q1. Why does VGPO perform poorly on Qwen3-VL?

Thank you for raising this issue. We reproduced VGPO on Qwen3-VL using its official code and found no implementation problems after repeated checks and experiments. Further examination of the literature and public reproductions suggests that our observation is plausible and independently supported.

Differences between base models. VGPO was mainly validated on Qwen2.5-VL (3B, 7B, and 32B), so its gains need not transfer to other architectures. The Qwen3-VL Technical Report (Bai et al., 2025) describes changes to vision-language fusion and multimodal representation, including DeepStack and Interleaved-MRoPE. VGPO relies on hidden-state similarity between generated and visual tokens and reweights advantages according to visual activation. Changes in multimodal representations and activation distributions may therefore affect its visual reward signal and optimization.

Independent reproduction. The open-source project Awesome Perception-Aware RLVR reproduced several perception-aware RL methods, including VGPO, on Qwen3-VL-4B-Instruct under common settings. It reported average scores of 70.6 for GRPO and 63.9 for VGPO, a decline of about 6.7 percentage points. It also found a notable drop from VGPO's best checkpoint to its final checkpoint, suggesting sensitivity to optimization settings and checkpoint selection. Thus, VGPO's gains on Qwen3-VL appear unstable.

This agrees with Table 1 of our paper: on Qwen3-VL-4B, VGPO and GRPO achieve STEM averages of 50.30 and 53.90, respectively. Absolute scores cannot be directly compared across studies with different benchmarks and protocols, but the relative trends agree.

Our VGPO results should therefore not be dismissed as reproduction errors; they reflect performance on Qwen3-VL under our settings. They do not contradict the gains reported on Qwen2.5-VL. We will clarify VGPO's implementation and training configuration and discuss independent reproductions in the revision.

References and public reproduction evidence

[1] Bai et al. (2025). Qwen3-VL Technical Report. arXiv:2511.21631. https://arxiv.org/abs/2511.21631

[2] Awesome Perception-Aware RLVR: Unified Reproductions and Controlled Comparisons. GitHub. https://github.com/zsxm1998/awesome-perception-aware-rlvr/blob/main/examples/comparison/rlvr_qwen3_vl_4b/README.md

Q2. InternVL3.5 baseline settings

Thank you for the comment. The InternVL3.5-8B and 38B results in Table 1 were measured independently under our unified evaluation protocol; they were not taken from the official technical report. We consider these comparisons rigorous and reliable.

We evaluated with VLMEvalKit and vLLM, a maximum of 4,096 generated tokens, and Thinking Mode disabled. For mathematical reasoning, the official InternVL3.5 setup recommends Thinking Mode, including a dedicated R1-style reasoning prompt, sampling (do_sample=True, temperature=0.6), and up to 65,536 generated tokens. Thinking Mode and longer outputs can substantially improve STEM results. InternVL developers also linked MathVerse and WeMath reproduction differences to Thinking Mode in GitHub Issue #1185.

The evaluation protocols therefore differ substantially, which largely explains the score gap without implying an error in our independent evaluation. We will make these differences explicit in the revision to prevent confusion and aid reproducibility.

AuthorsOpenReview ↗

Response to Reviewer Y8LF

Q3. Fair comparison with Vision-OPD

We appreciate the suggestion to evaluate VC-OPD on Qwen3.5 models for a fair comparison with Vision-OPD. However, we already established a direct, controlled comparison by reimplementing Vision-OPD on the same base models. We also added experiments with matched training data to further strengthen fairness.

Existing experiments use the same base models and evaluation protocol

Rather than copying the results reported in the Vision-OPD paper, we reimplemented Vision-OPD on Qwen3-VL-Instruct and compared it with VC-OPD at 2B, 4B, and 8B scales.

At each scale, both methods start from identical initial weights and use the same benchmarks and evaluation protocol. This controls for model initialization and baseline capability, allowing a more direct comparison of the distillation methods.

Additional experiments: matching training data

ViRL39K lacks the visual privileges required by Vision-OPD, so the results in Table 1 used Vision-OPD's original training dataset. For a fairer comparison, we used Gemini-3.5-Flash to generate bounding boxes for ViRL39K, created enlarged crops as visual privileged information, and trained Qwen3-VL-2B/4B/8B-Instruct models.

Tables R1 and R2 show that VC-OPD outperforms Vision-OPD at all tested scales after aligning base models, training data, and evaluation. Teachers given visual cues also score over 10 points higher on the training set than teachers given enlarged bounding-box crops.

These results suggest that VC-OPD provides stronger teacher supervision for STEM tasks and better transfers visual understanding and reasoning to the student.

Table R1. Results on STEM benchmarks. Vision-OPD is trained on ViRL39K.

MethodTrainset ScoreMathVerse$_m^v$MathVerse$_m^t$DynaMathWeMathMathVista$_m$MathVision$_m$STEM Average
Vision-OPD-2B57.5043.5358.553.6934.2965.225.3346.76
Ours-2B72.6845.0558.1254.1338.1967.0026.3248.14
Vision-OPD-4B69.5756.2270.6963.7347.2476.142.1159.35
Ours-4B81.0559.3972.0865.8951.6276.8044.0861.64
Vision-OPD-8B70.8562.5674.1166.6550.7676.344.0862.41
Ours-8B82.3261.5573.1068.8255.3376.8045.0763.44

Table R2. Results on general visual benchmarks. Vision-OPD is trained on ViRL39K.

MethodMMMU$_{pro}^v$MMMU$_{pro}^{std}$HallBenGeneral Average
Vision-OPD-2B30.1238.6169.9346.22
Ours-2B31.7337.6369.9346.43
Vision-OPD-4B45.3848.9672.5655.63
Ours-4B45.4949.7173.1956.13
Vision-OPD-8B49.6552.8974.1358.89
Ours-8B52.4953.7674.8760.37

We agree that experiments on Qwen3.5 models would further test cross-model generalization. We plan to include these results in a future revision.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall response

We thank all reviewers for their recognition and constructive feedback. We conducted additional controlled comparisons, ablations, and reliability experiments to address novelty, fairness, visual-cue effectiveness, and robustness. Our main responses are as follows:

1. Method novelty. VC-OPD introduces a unified Visual-Cue Agent for diverse VQA tasks, with difficulty, answer, and reliability gates to ensure high-quality privileged information. Unlike local visual pixels, question-conditioned visual cues support a broader range of tasks, including fine-grained recognition, global spatial understanding, and complex STEM reasoning (Tables R1–R3).

2. Fair comparison. We evaluated all methods under a unified evaluation protocol, and added controlled comparisons with Vision-OPD, ViCuR, V-Zero, and VAD. VC-OPD achieves better overall STEM and general-vision performance at 2B, 4B, and 8B scales (Tables R1–R2 and R5–R6). We also clarify reproduction discrepancies involving VGPO and InternVL3.5.

3. Benefits of visual cues. Visual cues strengthen the teacher while supporting both fine-grained perception and global spatial understanding. Compared with Vision-OPD, VC-OPD improves average performance by 9.94, 9.32, and 7.68 percentage points at 2B, 4B, and 8B, respectively (Table R3).

4. Training strategy and gating. Direct OPSD provides limited benefit for the weaker 2B model. Combining GRPO warm-up with OPD raises its average STEM score from 43.46 to 48.14 (original Table 3). Triple gating also improves combined average scores across STEM and general-vision tasks by 1.82–4.63 points across model sizes (Tables R8–R9), supporting both design choices.

5. Reliability and data quality. VC-OPD's answer-leakage rate is 4.86%, versus 20.60% for ViCuR (Table R4). Student-side recoverability is supported by qualitative evidence (Figures 3(b) and 5). Only 0.5157% of evaluation samples are potentially similar to ViRL39K samples (Tables R11). We also ran independent experiments with multiple random seeds to assess stability.

Overall, the additional experiments further support VC-OPD's effectiveness, novelty, and cross-task generalization. We will expand the related work, experimental details, and supplementary results in the revision to improve transparency and reproducibility.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.