acceptodds
Withdrawn from ICLR 2027

Distill the Visual Evidence, Not Just the Answer: Cross-World On-Policy Distillation for Vision-Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A central goal of vision-language model (VLM) distillation is to transfer both the teacher's language capabilities and its visual understanding. However, existing methods primarily supervise the student's output, leaving visual understanding implicit. Our analysis reveals that a student can match the teacher's answer without relying on the same visual evidence, raising the question: how can we ensure the student responds to the visual information that actually determines the answer? To this end, we propose Cross-World On-Policy Distillation (CW-OPD), which explicitly supervises the student's response to changes in visual evidence. For each example, CW-OPD constructs two visual worlds that share the question and scene context but differ in answer-critical evidence, yielding different answers. We perform on-policy distillation in both worlds and distill the teacher's cross-world belief transition, encouraging the student to match not only what the teacher predicts but also why its prediction changes with the evidence. A gradient analysis shows that this term is invariant to errors shared by both worlds and supplies a corrective signal invisible to endpoint matching alone. In this way, CW-OPD makes reliance on the relevant visual evidence an explicit distillation target rather than an implicit consequence of output matching. To diagnose whether a model truly grounds its answers in visual evidence, we introduce CWBench, which measures cross-world consistency via Cross-World Pair Accuracy (CWPA). Experiments on Qwen3.5-4B show that CW-OPD outperforms the strongest baseline by 1.2 points on average, and the 4B student exceeds DeepSeek-V4.1 (552B) by 22.4 CWPA points on CWBench. Code is included in the supplementary material.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-language models, on-policy distillation, visual evidence grounding, counterfactual visual worlds, cross-world distillation, multimodal knowledge distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bC7DOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CW-OPD, a cross-world on-policy distillation method for vision-language models. The authors observe that a student can match the teacher’s answer without relying on the same visual evidence. To address this misalignment, they construct dual-world pairs that share the question and scene but differ in answer-critical evidence through a plan–edit–verify pipeline. CW-OPD performs on-policy distillation in both worlds and additionally distills the teacher’s cross-world belief transition, so that the student learns not only what the teacher predicts but also why its prediction changes. The authors also introduce CWBench with new metrics (CWPA, CWFR), and show improvements on standard benchmarks and CWBench with Qwen3.5 students.

Critical strengths
  1. Intriguing motivation with a logically connected method. The motivation is intriguing and the method is logically connected to it. Extensive experiments, including ablations and design choices, support the detailed loss design and can guide future research.

  2. Methodological novelty. The idea of distilling the transition from teacher to student, not just each logit, is novel and convincing. Also, the transition loss is well designed and supported by the proposition.

Critical weaknesses
  1. Some baselines could be unfair. (1) Some methods, such as Vision-OPD, require a privileged hint in the form of a cropped image, but the paper does not explain how these methods are handled on CWBench training. Vision-OPD is especially important, since the motivational problem (student–teacher misalignment in visual evidence) is closely related to its approach. If the implementation differs from the original paper, this should be clearly explained. (2) Also, the OPD and OPSD results in Table 1 are identical to those in Table 3, where only LOPDAL^A_{OPD} is used. This implies that these baselines are trained only on (I, q, A), which is half of the data samples, without (I′, q, B). If the other baselines are also trained in this way, the comparison could be unfair, since CW-OPD additionally uses the edited worlds.

  2. The asymmetric gain between the two worlds lacks a logical explanation. The transition loss is defined symmetrically on the two worlds, so it should in principle affect the responses in both worlds. However, in Table 3, adding the transition loss improves mainly A Acc (original world), while B Acc (edited world) barely changes. The paper does not explain why the transition supervision benefits only one side. In addition, almost all models, including frontier models, show much higher accuracy on world B than on world A, which suggests that the edited images may be easier, for example due to overly salient edits or the verifier favoring easy pairs. Without explaining this asymmetry, the CWPA gain could also be interpreted as better solving the original questions rather than responding better to evidence changes.

Questions for authors
  1. Figure 1 averages the results of the Qwen3.5-9B teacher (OPD) and the 4B EMA teacher (OPSD). Could you report them separately? Since the EMA teacher is conditioned on an answer hint, its stronger sensitivity in Figure 1(c) might come from the hint rather than from better visual grounding.
Reviewer_xD3hOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors believe that on-policy distillation (OPD) for VLMs matches the teacher's output distribution without constraining which visual evidence the student uses. CW-OPD builds, for each multiple-choice VQA example, an edited image in which a chosen distractor becomes the correct answer. The student rolls out once on the original image, and the same prefix is scored in both "worlds". The paper introduces CWBench with Cross-World Pair Accuracy (CWPA) and flip rate (CWFR). Then the paper evaluates student models with different sizes and multiple external benchmarks, and includes a dual-world control without transition supervision and a larger-teacher variant.

Critical strengths
  • The problem setting is clear, and the objective is simple yet targeted.
  • I like the paired, answer-changing interventions, which are more convincing than ordinary single-image accuracy.
  • I appreciate the DW-OPD ablation study, and the reported clear CWPA improvements over DW-OPD in Table 3.
Critical weaknesses
  • The decisive comparisons cannot currently be reconciled. DW-OPD averages 79.43 in Table 3, while the corresponding OPSD-based lambda=0 sweep reports 77.73 in Table 8. Table 4 identifies a frozen 9B teacher but reports 65.82 CWPA / 80.97 Avg; Table 5 reports 55.44 / 80.37 for that teacher setting. Correct me if I'm wrong, but with the standard teacher, the cross-world benefit isn't very promising. I have a concern that maybe most of the CWPA gain therefore tracks the hint-conditioned teacher, and authors might need to prove that the teacher's belief transition is induced by visual evidence (not text alone).
  • CWBench's independent validity is insufficiently established. The supplement includes answer-checking prompts (for edit faithfulness), but the verifiers, acceptance procedure and human-audited label error rate remain unclear. Also, training and testing share an editing pipeline.
  • Some images are blurred, some details are hard to recognize; the spacing between images/tables and their captions are sometimes strange.
Questions for authors
  • Please reconcile Tables 3/8 and 4/5 with Appendix E, including per-run scores, evaluated pair counts and aggregation. How does the implemented teacher top-100 union objective relate to Eq. 6?
  • Can you provide the actual, full and clear configurations and run variability, to make sure the gains are reliably attributed to visual transition supervision?
  • Can you redraw and replace figures in the future?
Reviewer_ZqSeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that on-policy distillation (OPD) for VLMs supervises only outputs, so a student can match the teacher’s answers without relying on the same visual evidence. It constructs dual-world pairs (an image and an edited counterpart with a flipped answer) and proposes CW-OPD: OPD in both worlds plus a loss matching the student’s and teacher’s cross-world log-ratios. It also introduces CWBench and a pair-accuracy metric (CWPA), and reports gains on CWPA and standard VQA benchmarks.

Critical strengths

Timely problem and setting. OPD is currently one of the most widely used approaches for post-training models, and whether it transfers the teacher’s visual grounding, not just its answers, is a relevant question for VLMs. Adapting prior ideas on counterfactual image pairs into a form that works within OPD is a useful contribution if done well. This is the main consideration in the paper’s favor: the direction is worthwhile, even if the current execution and framing can be improved.

Critical weaknesses
  1. Novelty and framing. The motivating problem, that correct outputs do not imply correct grounding, is not specific to OPD. It applies equally to SFT or GRPO. The proposed fix, training on counterfactual image pairs with a consistency or contrastive objective, has therefore also been explored before, and minimal-change image benchmarks also already exist (e.g., VisMin, VLind-Bench). Since the paper's contribution and framing rest on identifying and fixing an OPD-specific limitation, and on introducing a new benchmark, the related work needs to properly address how this builds on prior ideas, honestly re-position the method against them, and clarify why a new QA benchmark is needed.

  2. The claimed effect is not demonstrated. The paper never shows that the student actually learns to rely on the right visual evidence. All gains are accuracy-based, and the motivating diagnostics (e.g., the blur test in Fig. 1) are never re-run after training. On top of that, key details needed to attribute the gains are missing: what the hint contains, and whether the baselines had the same hints and both cross-world images. This matters because without grounding evidence and equalized baselines, it is impossible to tell whether the improvements come from the cross-world objective, from the hint, or from extra data. The central claim is therefore unsupported.

Questions for authors
  • Clarifications

    • What exactly is the hint (prompt and examples)? If it is a pointer to the edited region, please state this explicitly.
    • Were SFT, GRPO and the other compared methods trained with the same hints and on both cross-world images, i.e., were the settings data-equalized?
    • Several tables are inconsistent about which teacher and setting each result uses. Please state the teacher (9B or EMA), whether the hint is used, and which worlds are trained on for each row, and resolve the following:
      • Table 3’s “DW-OPD” row appears to use the OPSD (EMA + hint) teacher, yet it is named OPD.
      • Table 8’s λ = 0 row, defined as DW-OPD, reports 77.73 Avg., while Table 3 reports 79.43 for DW-OPD.
      • Table 4’s caption says a Qwen3.5-9B teacher, but its “clean” numbers (e.g., 80.97 Avg., 65.82 CWPA) match the OPSD setting. Table 5 gives 80.37 Avg. for CW-OPD with the 9B teacher.
      • App. E says the OPD baselines (VGS, FP-OPD, Vision-OPD) use the EMA teacher, while Table 5 presents the same numbers as baselines with a 9B teacher.
    • Can Fig. 1(c) be reported separately for the 9B and EMA teachers?
    • Why are sample rollouts at temperature 2.0? Standard choice is 1.0. Are the KL terms computed at the rollout temperature?
  • Suggestions

    • Re-run the motivating diagnostics after training: attention to the critical region (Fig. 1a) and sensitivity under blurring (Fig. 1b), for CW-OPD versus the baselines.
    • An explanation and perhaps evaluation on existing counterfactual benchmarks (e.g., VLMs are Biased, VLind-Bench, VisMin) to show the gains generalize beyond CWBench, and clarify what CWBench offers over them.
    • If the baselines were not data-equalized, compare against them under the same information, e.g., SFT or GRPO on both worlds with the hints.
    • Report some results with rollouts sampled at the standard temperature of 1.0, to show the gains do not depend on the unusual choice of 2.0.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.